TRIM HORIZON AWS definition for DynamoDB Triggers Streams Lambda

Question

I have an IoT device sending irregular updates that I store in a DynamoDB Table. Now, I want to process just the recent updates onto another DynamoDB table and save it there. So, I need to use 'n' number of latest updates from the first table to compute an update in my second Table for that same device and ensure that they are in sync with all my original updates, sort of like a sliding window. And, while the DynamoDB Triggers (Streams + Lambda) suit my needs perfectly, I just can't seem to find any clear definition or documentation of TRIM_HORIZON. After referring to a few other docs, I figured it could be the oldest data in my first Table, which is a lot. But some docs also suggest that it is just for the past 24hr. Now, I don't know if it means the oldest data in the stream for the last 24hr, but I can only guess. So, can anyone please tell me which is actually true about TRIM_HORIZON, like will I be able to configure it?

Now, if it's not possible and I cannot use TRIM_HORIZON, I'd have to switch to using LATEST for performing a query on the first Table. But, that would just defeat the entire purpose of my streams. Please, advice!

Answer 1

Here's what I found in the DynamoDB's documentation:

(https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html)
All data in DynamoDB Streams is subject to a 24 hour lifetime. You can retrieve and analyze the last 24 hours of activity for any given table

(https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/Welcome.html)
TRIM_HORIZON - Start reading at the last (untrimmed) stream record, which is the oldest record in the shard. In DynamoDB Streams, there is a 24 hour limit on data retention. Stream records whose age exceeds this limit are subject to removal (trimming) from the stream.

This means if you have a Lambda continuously processing your stream updates, you could just go on with using LATEST. And, because you must use 'n' last updates to compute an update in your second Table, you'd as it is, need to query the first Table for each and every update in order to 'merge' current update with previous ones from your device. So, I think its highly unlikely for just about anyone to do something that using TRIM_HORIZON.