DynamoDB trivia: reads throttled but under capacity

When the throttle happens

I get woken up to an alert about read capacity on one of my DynamoDB tables. Here is the error being logged.

ProvisionedThroughputExceededException: The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API.

When I load my CloudWatch graphs, and it appears I’m well below capacity.

These graphs show consumed capacity (blue on the left) well below provisioned capacity (red on the left), while our throttled requests spike for a bit then go down.

Our first thought is that DynamoDB is doing something wrong. Maybe it had an issue at that time. Before I go on, try to think and see if you can brainstorm what the issue was.