how do I decide the throughput of the data stream

Question

I am trying out Amazon's Kinesis stream for the first time. What I do not understand is, how do I decide the throughput of the data stream?

Answer 1

Hi @Hannah,

1. Estimate the average size of the record written to the data stream in kilobytes
2. Estimate the number of records written to the data stream per second
3. Decide the number of Amazon Kinesis Applications consuming data concurrently and independently from the data stream
4. Calculate the incoming write bandwidth
5. Calculate the outgoing read bandwidth

Then use the following formula:

number_of_shards = max (incoming_write_bandwidth_in_KB/1000, outgoing_read_bandwidth_in_KB/2000)