Cosmos DB: request units, capacity modes and 429 errors
What an RU is, provisioned manual, autoscale and serverless, reading a query's cost, 429 throttling and cost reflexes.
On the SQL Server side I size in vCores, DTUs or memory. In Cosmos DB everything is measured in Request Units (RU). It is the notion that took me longest to digest. Here are my notes on what an RU is, the capacity modes and cost reflexes.
What an RU is
An RU is a "performance currency" that abstracts CPU, IOPS and memory. Every operation (read, write, query) has an RU cost, whatever the API. A few reference points from the docs:
- reading a roughly 1 KB item by
id+ partition key (point read): 1 RU; for 100 KB: 10 RU; - inserting a 1 KB item without indexing: about 5.5 RU; replacing an item costs twice the insert;
- the same query on the same data always costs the same number of RUs;
- the strong and bounded staleness consistency levels double the cost of reads.
What moves the cost: item size, number of indexed properties, indexing policy, number of predicates, result size. Point reads cost less than queries.
Capacity modes
| Mode | Principle | When |
|---|---|---|
| Provisioned manual | You set RU/s (steps of 100, minimum 400 per container) and are billed hourly | Stable, predictable load |
| Provisioned autoscale | You set a maximum Tmax; throughput varies between 0.1 x Tmax and Tmax; billed hourly on the highest level reached |
Variable or unpredictable load |
| Serverless | No throughput to provision; you pay for consumed RUs and storage | Dev/test, intermittent traffic |
Details checked in the docs:
- Autoscale: minimum
Tmaxof 1000 RU/s, in steps of 1000. It can be cheaper if you useTmaxfor 66% or fewer of the hours in a month. You can switch between manual and autoscale at any time (portal, CLI, PowerShell). - Serverless: a serverless account runs in a single region, you can't pass throughput when creating a container, nor read or update throughput. Each physical partition handles up to 5,000 RU/s. The docs recommend it for getting started, bursty traffic (average-to-peak ratio under 10%) and test environments.
- Throughput is set at container level (dedicated) or database level (shared across up to 25 containers). The docs advise against shared throughput for most workloads: a noisy container can starve the others.
- With several regions, provisioned RUs are available in each region: global throughput is R x N.
Reading the cost of a query
The cost comes back in the x-ms-request-charge response header. In the portal: Data Explorer, new SQL query, execute, then "Query Stats". With the Python SDK:
items = list(container.query_items(
query="SELECT * FROM c WHERE c.customerId = @cid",
parameters=[{"name": "@cid", "value": "C42"}],
partition_key="C42"))
charge = container.client_connection.last_response_headers["x-ms-request-charge"]
print(charge)
A paged query is charged per page; summing the pages gives the total cost.
429 errors and retries
When you consume more RUs than the throughput available in a given second, the service returns 429 (Request rate too large). The counter resets every second. The SDKs retry automatically (typically up to nine times), so the application often sees nothing.
- 1 to 5% of requests in 429 with acceptable latency: healthy, the throughput is fully used.
- Autoscale can also return 429s, if you exceed
Tmaxor a hot partition exceeds its share. - 429s on metadata (creating or listing containers) aren't fixed by adding RUs: use a single SDK client and cache names.
To see who is throttled:
CDBDataPlaneRequests
| where TimeGenerated >= ago(24h)
| summarize throttled = dcountif(ActivityId, StatusCode == 429),
total = dcount(ActivityId),
ru = sum(RequestCharge)
by CollectionName, OperationName, bin(TimeGenerated, 1min)
| extend fractionOf429s = 1.0 * throttled / total
| order by fractionOf429s desc
The "Normalized RU Consumption" metric tells you whether you are over- or under-provisioned: constantly at 100% with 429s means you need more RUs or a better partition key.
Cost reflexes
- Prefer point reads (
id+ partition key) over queries. - Include the partition key in filters.
- Tune the indexing policy for production: by default every property is indexed, which costs RUs on writes.
- Keep items small; put large binary content in Blob Storage and store a link.
- Pick autoscale for variable loads, serverless for dev/test, manual for stable ones.
- Set alerts on RU consumption.
- Estimate with the Cost Estimator or the capacity planner before deciding.
Lessons learned
- Raising RUs is not the first answer to a 429: look first for a hot partition or a badly targeted query.
- The biggest reflex for a SQL DBA to acquire: measure the RU cost of every typical query during development.