LLI Wiki — My Digital Knowledge Base

Cosmos DB: request units, capacity modes and 429 errors

What an RU is, provisioned manual, autoscale and serverless, reading a query's cost, 429 throttling and cost reflexes.

On the SQL Server side I size in vCores, DTUs or memory. In Cosmos DB everything is measured in Request Units (RU). It is the notion that took me longest to digest. Here are my notes on what an RU is, the capacity modes and cost reflexes.

What an RU is

An RU is a "performance currency" that abstracts CPU, IOPS and memory. Every operation (read, write, query) has an RU cost, whatever the API. A few reference points from the docs:

What moves the cost: item size, number of indexed properties, indexing policy, number of predicates, result size. Point reads cost less than queries.

Capacity modes

Mode Principle When
Provisioned manual You set RU/s (steps of 100, minimum 400 per container) and are billed hourly Stable, predictable load
Provisioned autoscale You set a maximum Tmax; throughput varies between 0.1 x Tmax and Tmax; billed hourly on the highest level reached Variable or unpredictable load
Serverless No throughput to provision; you pay for consumed RUs and storage Dev/test, intermittent traffic

Details checked in the docs:

Reading the cost of a query

The cost comes back in the x-ms-request-charge response header. In the portal: Data Explorer, new SQL query, execute, then "Query Stats". With the Python SDK:

items = list(container.query_items(
    query="SELECT * FROM c WHERE c.customerId = @cid",
    parameters=[{"name": "@cid", "value": "C42"}],
    partition_key="C42"))
charge = container.client_connection.last_response_headers["x-ms-request-charge"]
print(charge)

A paged query is charged per page; summing the pages gives the total cost.

429 errors and retries

When you consume more RUs than the throughput available in a given second, the service returns 429 (Request rate too large). The counter resets every second. The SDKs retry automatically (typically up to nine times), so the application often sees nothing.

To see who is throttled:

CDBDataPlaneRequests
| where TimeGenerated >= ago(24h)
| summarize throttled = dcountif(ActivityId, StatusCode == 429),
            total = dcount(ActivityId),
            ru = sum(RequestCharge)
  by CollectionName, OperationName, bin(TimeGenerated, 1min)
| extend fractionOf429s = 1.0 * throttled / total
| order by fractionOf429s desc

The "Normalized RU Consumption" metric tells you whether you are over- or under-provisioned: constantly at 100% with 429s means you need more RUs or a better partition key.

Cost reflexes

  1. Prefer point reads (id + partition key) over queries.
  2. Include the partition key in filters.
  3. Tune the indexing policy for production: by default every property is indexed, which costs RUs on writes.
  4. Keep items small; put large binary content in Blob Storage and store a link.
  5. Pick autoscale for variable loads, serverless for dev/test, manual for stable ones.
  6. Set alerts on RU consumption.
  7. Estimate with the Cost Estimator or the capacity planner before deciding.

Lessons learned

Further reading

Mastodon LLIMastodon SocialFosstodon