How to list all the objects in Amazon S3

Question

S3Client.ListObjects return only 1000 of objects. How to retrieve list of all existing objects using Amazon C# library?

Answer 1

As stated already, Amazon S3 indeed requires Listing Keys Using the AWS SDK for .NET:

As buckets can contain a virtually unlimited number of keys, the complete results of a list query can be extremely large. To manage large result sets, Amazon S3 uses pagination to split them into multiple responses. Each list keys response returns a page of up to 1,000 keys with an indicator indicating if the response is truncated. You send a series of list keys requests until you have received all the keys.

The mentioned indicator is the NextMarker property from the ObjectsResponse Class - its usage is illustrated in the complete example Listing Keys Using the AWS SDK for .NET, with the relevant fragment being:

static AmazonS3 client;
client = Amazon.AWSClientFactory.CreateAmazonS3Client(
                    accessKeyID, secretAccessKeyID);

ListObjectsRequest request = new ListObjectsRequest();
request.BucketName = bucketName;
do
{
   ListObjectsResponse response = client.ListObjects(request);

   // Process response.
   // ...

   // If response is truncated, set the marker to get the next 
   // set of keys.
   if (response.IsTruncated)
   {
        request.Marker = response.NextMarker;
   }
   else
   {
        request = null;
   }
} while (request != null);