Containers-roadmap: [Fargate] [request]: Mount S3 buckets as filesystem on Fargate, e.g. via FUSE

[michalc]

Tell us about your request
A way of mounting (folders of) S3 buckets as a filesystem on Fargate containers, potentially via FUSE.

Which service(s) is this request for?
Fargate

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
We launch JupyterLab and RStudio servers on demand and per-user in Fargate containers, which are shut down when idle. Without file-persistance, source code and output data can be lost, and any previously uploaded data would have to be re-uploaded.

We also use AppStream for other analysis tools, which in our configuration stores the home folders on S3: so mounting these S3 folders on Fargate would be ideal to present a unified file-space for each user.

At the moment this isn't possible in Fargate since --cap-add SYS_ADMIN is not supported (and there may also be other requirements I'm not aware of).

While the below work-arounds are possible, they can be manual with a too high chance of mistakes.

Are you currently working around this issue?
Yes:

• For JupyterLab, we use an S3 contents manager that can save and retrieve notebooks to/from S3 transparently.
• For other files / RStudio, users explicitly save/retrieve files from S3 using R or Python code, e.g. using Boto3 in Python

[whereisaaron]

Allowing SYS_ADMIN was also discussed in the context of mount EFS filesystems for FarGate: https://github.com/aws/containers-roadmap/issues/53#issuecomment-491848677

And in the context of allowing SYS_PTRACE (https://github.com/aws/containers-roadmap/issues/409#issuecomment-510717648) @ejc3 pointed out a tweet that said that it might possible if there were demand: https://twitter.com/nathankpeck/status/1122188790452371456

[jantman]

For what it's worth, this would be useful to me also but in a much simpler use case... I run a bunch of open source projects in containers that are stateless but require config files to be mounted into the container. I currently store those config files in S3, and have only 2 options:

1. Run an EC2 host instead of Fargate, just to download a handful (i.e. a dozen or so) of trivially small text files from S3 onto the instance and mount them into the containers.
2. Re-package the existing public/FOSS docker images with awscli and a hacked-together entrypoint script that downloads the config files from S3 in the container itself... which also means a container that shouldn't need IAM privileges at all needs to be able to pull from S3, and I need to constantly rebuild these containers myself to keep up with upstream.

I suppose my use case is a bit more constrained than FUSE, as I really just need a way to pull files/prefixes from S3 into a container when it starts.

[jcharnley]

For what it's worth, this would be useful to me also but in a much simpler use case... I run a bunch of open source projects in containers that are stateless but require config files to be mounted into the container. I currently store those config files in S3, and have only 2 options:

Hi did you find any other solutions? this is the same use case as myself but am not using EC2 only ECS Fargate. and am not sure how to re-package the docker image...

[mwarkentin]

@jantman fargate supports EFS now - would that work as an alternative to what you’re asking for?

[jcharnley]

am looking into that now thanks

[joaquin386]

But EFS on Fargate is not yet available via CF, I am specially waiting for that extension.

[jcharnley]

amn going to give it a try, i just need to mount a file when starting a docker container on ECS fargate

[jcharnley]

think it will be easier for me to extend my image and put that on ECR with mounted volume. thanks tho

[KMahankali]

So this feature request is for ECS-EC2 as well? I am seeing below error when tried to mount s3fs file system, when i initially changed my directory permissions to 755 before mounting it was good however when i mount my directory to S3 that is when my directory permissions got changed and restricted
Appreciate if this feature request releases soon for both ECS-EC2 & ECS fargate

Before mounting --> drwxrwxrwx 2 root root 6 Jul 24 04:40 mounttest2
After mounting --> drwx------ 1 root root 0 Jan 1 1970 mounttest2

stat /home/ec2-user/mounttest: permission denied\

[smetj]

Having the ability to mount S3 files/directories into a Fargate container would be very useful to mount configuration files into the container. One could currently achieve this using EFS for Fargate, however putting config files on EFS as part of a deployment is not ideal compared to speaking to the S3 API.

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

First of all EFS is not a suitable solution for this particular use case because copying files to EFS can't be done through the AWS API.
You will either have to mount the network share on the system on which the deployment is executed or figure out some indirect approach which just moves the problem to a different space. (for example a container/lambda which fetches said config file from S3 and copies it to EFS or some variation on that.)

If Fargate could mount (config)files from S3 into containers it would allow me to use unmodified/original containers:

I don't need to either:

• build a new container which bases the original one and includes the config file(s)
• build a new container which includes some custom entrypoint pulling in a config from wherever.

Both these approaches, or variations on them, require me to have an environment to build and manage the builds of the containers. The deployment of the stack depending on these containers then also depend upon the building of these containers. More infrastructure means more complexity, more problem solving, more diverted attention, more cost ...

When Fargate could mount S3 files/dirs, I could:

• use Terraform to deploy the stack and generate config files during deployment.
• Let Terraform upload the config files to S3 as part of the stack deployment
• mount config files into the necessary Fargate containers.

For many use cases we could just use vanilla containers without any customization hence supporting infrastructure required.

• Whether this is achieved using Fuse or another fashion I have no preference for that.
• Performance isn't an issue for me. It's about simplifying deployments and cutting deployment supporting infrastructure.

Are you currently working around this issue?
Not really because I build new containers with custom entrypoints pulling in the config from elsewhere hence I need infrastructure to organize that.

[gdbassett]

Going to add my chip to the pile as well. We use Fargate to spin up jupyter notebooks with jupyterhub. We mount some relatively static data using EFS, but our big data stores belong in S3. The inability to mount the S3 buckets as filesystems complicates our ability to interact with our files through the various jupyter kernels.