Amazon-ecs-agent: tmpfs and dev shm for Fargate

[taktakpeops]

Summary

This is more a question than an issue. I am currently trying to run a Selenium Grid with Chrome and Firefox node in ECS Fargate.

My container are able to start but all my tests are failing because /dev/shm cannot be set as in an ECS launch type.

I was wondering if tmpfs and dev shm will be accessible in Fargate at some point. If not, I would like to understand the why (which I believe is related to the architecture of Fargate).

Description

Currently, for giving access to /dev/shm to my container inside a task, I am doing the following in my Cloudformation manifest:

SeleniumNodeChromeTask:
    Type: "AWS::ECS::TaskDefinition"
    DependsOn: TalpaAccountSeleniumGrid
    Properties:
      Volumes:
        - Name: "shm"
    ContainerDefinitions:
    - MountPoints:
           - ContainerPath: /dev/shm
              SourceVolume: shm
              ReadOnly: false

The volume is mounted correctly but Firefox and Chrome keep failing.

Here is an error message I got from my Firefox container:

WARNING: failed to open shm: Permission denied: file /builds/worker/workspace/build/src/ipc/chromium/src/base/shared_memory_posix.cc, line 250

Expected Behavior

I am expecting to be able to setup tmpfs and dev shm for my container and get my headless browsers to work as expected

Environment Details

ECS Cluster: latest
Selenium Grid Docker images: 3.141.59-yttrium (official images)

[fenxiong]

Hi @taktakpeops ,
I think you are right that tmpfs and shm are not supported on ECS Fargate (either via docker run parameter or a bind mount), and it's reflected in our documentation as well. As to whether it can be supported in the future, I recommend creating a feature request in the containers-roadmap repository (as far as I know, there isn't any such request yet).

Let us know if you have any other question. Thanks.

[taktakpeops]

Hi @fenxiong,

It's indeed in your documentation. I was wondering the why of this choice.

I will open this issue in the repo you suggested.

Thank you !

[sudopla]

I created a volume for a Fargate task, mounted on /dev/shm and it worked for me. See the CDK code below

// Create Task Definition
const taskDefinition = new ecs.FargateTaskDefinition(this, "Task", {
    cpu: 1024, 
    memoryLimitMiB: 6144, 
    taskRole,
    volumes: [ { name: 'shm_volume'} ] // Selenium/Firefox needs a volume mounted in /dev/shm. This is the --shm-size option in docker run that is not suported in Fargate
    })

// Add shm volume to container definition
containerDefinition.addMountPoints({ sourceVolume: 'shm_volume', containerPath: '/dev/shm', readOnly: false})

[taktakpeops]

The problem here is that you aren't mounting a shared memory but a volume named "/dev/shm". In theory, Firefox won't be able to run still.

For chrome, there is a flag to prevent it from using the shared memory.