User Data script to call aws cli

Question

I am trying to get some files from S3 on startup in an EC2 instance by using a User Data script and the command

/usr/bin/aws s3 cp ...

The log tells me that permission was denied and I believe it is due to aws cli not finding any credentials when executing the user data script.

Running the command with sudo after the instance has started works fine.

I have run aws configure both with sudo and without.

I do not want to use cronjob to run something on startup since I am working with an AMI and often need to change the script, therefore it is more convenient for me to change the user data instead of creating a new AMI everytime the script changes.

If possible, I would also like to avoid writing the credentials into the script.

How can I configure awscli in such a way that the credentials are used when running a user data script?

Answer 1

I suggest you remove the AWS credentials from the instance/AMI. Your userdata script will be supplied with temporary credentials when needed by the AWS metadata server.

See: IAM Roles for Amazon EC2

• Clear/delete AWS credentials configurations from your instance and create an AMI

• Create a policy that has the minimum privileges to run your script

• Create a IAM role and attach the policy you just created

• Attach the IAM role when you launch the instance (very important)

Have your userdata script call /usr/bin/aws s3 cp ... without supplying credentials explicitly or using credentials file

Answer 2

I'm assuming you are already using an IAM role?

If you aren't running Amazon Linux you will need to make sure export AWS_CA_BUNDLE is run, it normally runs from /etc/profile.d/aws-cli-default-config.sh   this isn't run by user-data. You may have other things in /etc/profile.d/  or /root/.aws/config neither of which I believe user-data reads.