How to access files in S3 bucket from R

Question

I have installed R on one of my Amazon EC2 Instance. And I have uploaded few csv files into a S3 bucket. I was wondering if there is a way to access these csv files in the S3 bucket from R.

Any help  would be appreciated.

Answer 1

Enter the following command: install.packages("AWS.tools")

Then, use the s3.get() command from there. The Help tab should tell you what goes in for arguments.

Answer 2

You can take a look at the cloudyr aws.s3 package in Github ( https://github.com/cloudyr/aws.s3 ), it is quite related to what you need. Unfortunately, this package is quite early-stage & a little unstable.

I've have had good success simply using R's system() command to make a call to the AWS CLI. This is relatively easy to get started on, very robust, and very well supported.

Start here: https://aws.amazon.com/cli/

List objects using S3 API: https://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects.html

Get objects using S3 API: https://docs.aws.amazon.com/cli/latest/reference/s3api/get-object.html

So, for example, on command-line try following:

pip install awscli aws configure aws s3 help aws s3api list-objects --bucket some-bucket --query 'Contents[].{Key: Key}' aws s3api get-object --bucket some-bucket --key some_file.csv new_file_name.csv

In R, can just do something like:

system("aws s3api list-objects --bucket some-bucket --query 'Contents[].{Key: Key}' > my_bucket.json")