Request GPU capacity →

Your data

AWS query (Athena)

Replace cur_table. Save as CSV.

SELECT date_trunc('hour', line_item_usage_start_date) AS usage_hour,
       product_instance_type AS instance_type,
       SUM(line_item_usage_amount) AS hours,
       SUM(CASE WHEN line_item_line_item_type <> 'Credit' THEN line_item_unblended_cost ELSE 0 END) AS cost,
       SUM(CASE WHEN line_item_line_item_type = 'Credit' THEN -line_item_unblended_cost ELSE 0 END) AS credits
FROM cur_table
WHERE line_item_usage_start_date >= date_add('month', -3, date_trunc('month', current_date))
GROUP BY 1, 2
ORDER BY 1;
Google Cloud query (BigQuery)

Replace the table name. Save as CSV.

SELECT TIMESTAMP_TRUNC(usage_start_time, HOUR) AS usage_hour,
       sku.description AS instance_type,
       SUM(usage.amount_in_pricing_units) AS hours,
       SUM(cost) AS cost,
       -SUM(IFNULL((SELECT SUM(c.amount) FROM UNNEST(credits) c), 0)) AS credits
FROM `project.dataset.gcp_billing_export_v1_XXXXXX`
WHERE usage_start_time >= TIMESTAMP(DATE_SUB(DATE_TRUNC(CURRENT_DATE(), MONTH), INTERVAL 3 MONTH))
GROUP BY 1, 2
ORDER BY 1;

Measure GPU use

Run on each GPU machine. It samples nvidia-smi and writes gpu_use.csv. No network calls, no hostnames.

curl -O https://check.ashibaresearch.com/gpu_use.py
python3 gpu_use.py --test
nohup python3 gpu_use.py --hours 168 &

After a week, drop every gpu_use.csv here. Read the script first.