S3fs-fuse: Does not check ECS task role location

[countergram]

Is it possible to use s3fs from within ECS with an IAM role? I can't get it to work.

s3fs has a hardcoded URL for roles: http://169.254.169.254/latest/meta-data/iam/security-credentials/

ECS container specific roles (Task roles) are at http://169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI

[psyvision]

Hi @ggtakec - We've stumbled across this issue today and I'd like to get a pull request in to use the alternate URL to use the ECS task role rather than the EC2 instance role.

How do you think would be best to implement this? I.e. an additional command line switch to change the URL used or an alternative command line option e.g. task_iam_role?

[ggtakec]

@psyvision @countergram
I'm sorry for not care about ECS.

If I implement this function, consider the following:
1) Add an argument indicating that it is ECS as the startup option (ex. "-ecs" etc)
2) Use the value of the environment variable $ AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
3) Required for (1) and (2), if an exclusion option such as iamrole is specified, an error

It maybe fixed S3fsCurl::CheckIAMCredentialUpdate, I think that it will be a way to add a review per function.
This function will be called from s3fs_check_service.

It seems that we need to aupport ECS on s3fs.

[psyvision]

Hi @ggtakec no worries! I'm happy to work on implementing that, I had already had a good look over the code base but anted to check best practices.

I'll submit a pull request as I start getting things implemented.

[psyvision]

Nearly got this working... :/

[psyvision]

It works! I need to do a code tidy up to remove some of my debug logging.

I have a IAM role created with S3FullAccess. My ECS task is assigned this role. I can then mount a bucket with the following command:

s3fs -o allow_other -o ecs -o use_sse=1 my-bucket-name /mnt

Note
I've used jsoncpp to parse the JSON metadata returned by AWS because the previous parser wouldn't handle the ECS specific endpoint. At the moment this appears to be dynamically linked so I have to run apt-get install libjsoncpp-dev before I can use s3fs.

My knowledge of the build setup here isn't great (I'm not really a C++ programmer) so I haven't worked out how to static link it yet. If I do I'll commit that change.

[ggtakec]

@psyvision I merged your PR #671
If you found something wrong about it, please let me know.
Thanks all.