Scan with filter returns zero results in DynamoDB

Question

I have a table with approximately 300 000 rows in DynamoDB. It contains three columns: source, data and cycles. The source attribute is a hash key and data is a range key. The cycles attribute is a number which is added by the application each time a row is to be added. I used to scan these rows with the following filter:

However, this scanfilter spontaneously seems to have decided not to work properly anymore. Anytime I execute this request it will return zero results. When I remove the scanfilter it returns 20 results, so I assume it has nothing to do with the limit or tableName attributes. The funny thing is that when I set the filter to find items with cycles equals "1" it actually works, maybe because there are 10 rows with cycles = 1 at the start of the table.

My question is simply how to fix this. Should I add a new index, or should I change this scanfilter? I can also try to use the query command instead of scan, but my previous attempts to do this didn't work out either.

Answer 1

By accessing all items in a table or a secondary index, the Scan action returns one or more items and item attributes. A FilterExpression action can be used to tell DynamoDB to return fewer entries. If the total number of scanned items exceeds the 1 MB maximum dataset size limit, the scan is terminated and the results are provided to the user as a LastEvaluatedKey value so that the scan can be resumed later. The number of items beyond the limit is also included in the results. If no table data meets the filter requirements after a scan, the scan will fail. A single Scan operation reads up to the maximum number of items set (if using the Limit option) or a maximum of 1 MB of data, after which any filtering is applied.

You must paginate the result set if LastEvaluatedKey appears in the response. In the Amazon DynamoDB Developer Guide, see Paginating the Results for more information.

Applications can request a parallel Scan operation by giving the Segment and TotalSegments options for quicker performance on a large table or secondary index. See the Amazon DynamoDB Developer Guide's Parallel Scan for more information.

When reading data in a table, Scan utilises eventually consistent reads; as a result, the return set may not include changes to the table data that occurred before the operation began. Set the ConsistentRead parameter to true if you need a consistent copy of the data when the Scan starts.