Aws-sdk-ruby: ECS client's run_task not correctly using ephemeral_storage property

[another-dave]

• [x] I've gone though Developer Guide for v3 and API reference
• [x] I've checked AWS Forums and StackOverflow for answers — I've just raised a Question on Stack Overflow, but no existing answers to the issue
• [x] I've searched for previous similar issues and didn't find any solution

Describe the bug

The ephemeral_storage property does not take effect when calling AWS::ECS::Client.run_task — task is successfully launched in AWS, but storage isn't increased beyond the default.

Gem name and its version

• aws-sdk-ecs (1.78.0)

Version of Ruby, OS environment

$ ruby -v
ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]

To Reproduce (observed behavior)

Calling task with the following format:

client = Aws::ECS::Client.new(region: region)
client.run_task({
  cluster: fargate_cluster_name,
  task_definition: task_definition,
  launch_type: 'FARGATE',
  network_configuration: {
    awsvpc_configuration: {
      subnets: my_subnets,
        security_groups: my_security_groups,
        assign_public_ip: 'ENABLED',
      },
  },
  overrides: {
    container_overrides: [
      {
        name: my_container_name,
        command: command_params
      }
    ],
    ephemeral_storage: {
      size_in_gi_b: 200
    }
  }
})

Will successfully launch the task in AWS — it runs fine (until it runs out of disk space :slightly_smiling_face:). I've added df -H to the entrypoint script which shows the disk space with the same values, regardless of ephemeral_storage param (screenshot below).

I've tried setting the value to lower amounts that 200 either (e.g. 40/50), but no effect.

Expected behavior
Can successfully increase disk space via the Ruby SDK property.

Screenshots

Just a screenshot showing the output of df -H from within the task, when size_in_gi_b was set to 200. (Though it was unusual here that total size for overlay was 32G — I thought the default was 20G, _including_ the space taken by Docker, but this is the same regardless.)

Additional context

• Have double-checked that the task is using platform v 1.4.0 — it is.
• Using the SDK from within Rails 6.1
• Docker image is ~ 470mb, based on ubuntu:latest

[ajredniwja]

Reached out to the ECS team for them to investigate this, will update once I hear back from them.

[another-dave]

Reached out to the ECS team for them to investigate this, will update once I hear back from them.

Brilliant, thanks very much Ajwinder!

[ajredniwja]

@another-dave: Soft update: the ECS team has pushed out the changes which should be reflected soon.

[another-dave]

@another-dave: Soft update: the ECS team has pushed out the changes which should be reflected soon.

Hi @ajredniwja, that's great — thanks for following it up. I'll allow some time for it to take effect & check it out some time next week :+1:

[mullermp]

Has this been fixed @ajredniwja @another-dave ?

[another-dave]

Has this been fixed @ajredniwja @another-dave ?

Hi @mullermp, @ajredniwja,

sorry for the delay getting back on this — didn't have a chance to test last week in the end. Just to confirm that this is all working as expected now. Just did a test run to bump up the ephemeral storage to 100GB and all worked fine :tada:

Thanks for your help!

Best
Dave

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.