A small project read 33 billion D1 rows: where the reads came from and a pre-release checklist
Sylwair reviews how a small content site on Workers Paid used more D1 rows read than the monthly allowance, explains how D1 counts scanned rows, names the four query patterns that multiplied reads, and lists the fixes and a checklist to run before shipping.
- Products
- D1Workers
- Source
- Author experienceSylwair blog
- Author
- Sylwair
Key points
- The author writes that his billing page showed about 33 billion D1 rows read against the 25 billion included, adding a D1 Rows Read line of $8.22, and that the cause was not user growth but queries repeatedly scanning large ranges of data.
- He explains that D1 counts rows scanned, not rows returned, and gives a rough formula: total reads = requests × rows scanned per query × queries per request; crawler traffic on the sitemap, scheduled URL-submission queues and background statistics keep running even when no person visits. Cloudflare’s current D1 pricing page also says rows read counts rows scanned rather than rows returned.
- The four amplifiers he found: a search or results page that recomputed historical statistics on every visit; the same request querying the same data more than once; a dynamic sitemap recomputed from the database on each fetch; and an SEO submission queue that ran too often with large batches.
- His fixes: precompute summary tables, reuse data already queried in a request, serve the sitemap from a periodically refreshed snapshot, lower cron frequency and batch size, and index columns used for filtering, sorting and status; he found the D1 Metrics view and the Query Insights ranking by rows read the most useful way to find the problem, and says he will not claim an exact saving from screenshots.
- Before release he now asks: will this endpoint be crawled or triggered by a schedule, how many rows does each query scan versus return, are the right indexes in place, are there duplicate queries, can the sitemap be a snapshot, can statistics be pre-aggregated, how often does the queue run, and is the billable usage trend normal — and every cron job should state its frequency, maximum items per run, estimated rows scanned and how to pause it.
Original sources
Published
A small project read 33 billion D1 rows: where the reads came from and a pre-release checklist
Open original sourceSylwair reviews how a small content site on Workers Paid used more D1 rows read than the monthly allowance, explains how D1 counts scanned rows, names the four query patterns that multiplied reads, and lists the fixes and a checklist to run before shipping.
More to read
Introducing on-demand profiling: Cloudflare production examples
Dominik Picheta introduces on-demand CPU and memory profiling for Workers and Durable Objects, sharing Cloudflare examples of removing duplicate work and unnecessary allocation paths.
babynarc: a Durable Objects alarm loop and a surprise bill that grew from $300 to $500.
@babynarc says an alarm loop in an old Durable Object he thought was inactive left him badly burned: he watched a surprise $300 bill grow to $500 even after killing it. Replying to Cloudflare's Dane Knecht and Kenton Varda, he suggests prepending budget-alert details to every `cf` result by default.
Cloudflare Medivh: a self-hosted near-real-time bill estimator
An open-source, self-hosted monitor that adds today’s estimated usage cost to Cloudflare’s daily-updated billable usage, sends summaries, and calls a webhook when the current cycle crosses a budget.