Shrine: S3 KMS encryption support

Created on 15 Feb 2019  Â·  36Comments  Â·  Source: shrinerb/shrine

Brief Description

How can we use Shrine for S3 KMS encrypted uploads?

This had a workaround until recently so I didn't bother reporting, but with the latest changes starting v2.14 (62705baf928bc7709ed8f88b48cdd7a4b4e9266c) it's no longer an option so I'm reaching out for help.

Thanks in advance!

Expected behavior

We are using the KMS encryption support to upload/download files from S3. I managed to make it work by overwriting the S3 client to let the storage use the Aws::S3::Encryption::Client instead.

It would look like this:

require 'shrine/storage/s3'

class S3Storage < Shrine::Storage::S3
  attr_writer :client

  def object(id)
    Aws::S3::Object.new(
      # The S3 encryption client is a thin wrapper to generic s3 client.
      client: client.try(:client) || client,
      bucket_name: bucket.name,
      key: [*prefix, id].join('/')
    )
  end
end

Shrine.storages[:encrypted] = S3Storage.new(prefix: 'encrypted', bucket: ENV['MY_ENC_BUC'])
Shrine.storages[:encrypted].client = Aws::S3::Encryption::Client.new(
    kms_client: Aws::KMS::Client.new(region: 'us-east-1'),
    kms_key_id: ENV['KMS_ID']
)

This little patch, allowed us to have a pretty clean KMS support for Shrine.

Actual behavior

With the latest changeset, we're getting errors while trying to download the file, due to (I believe) the different APIs for AWS S3 clients....

Simplest self-contained example code to demonstrate issue

> upload.asset.read
NoMethodError: undefined method `head_object' for #<Aws::S3::Encryption::Client:0x000055b916c75a18>
    from /usr/local/bundle/gems/shrine-2.14.0/lib/shrine/storage/s3.rb:576:in `load_data'
    from /usr/local/bundle/gems/shrine-2.14.0/lib/shrine/storage/s3.rb:383:in `open'
    from /usr/local/bundle/gems/shrine-2.14.0/lib/shrine/uploaded_file.rb:96:in `open'
    from /usr/local/bundle/gems/shrine-2.14.0/lib/shrine/uploaded_file.rb:254:in `io'
    from /usr/local/bundle/gems/shrine-2.14.0/lib/shrine/uploaded_file.rb:156:in `read'
    from (irb):17
   ...

System configuration

Ruby version: 2.3

Shrine version: 2.14

Most helpful comment

@rmalkevy I've made a mistake in my comment about delegation; it said to delegate [:head_object, :get_object], but it should have said [:head_object, :delete_object]. Aws::S3::Encryption::Client#get_object is already implemented and it's what does the decryption, so adding that #get_object delegate would use the regular Aws::S3::Client#get_object which doesn't do any decryption.

With this, downloads are automatically decrypted:


Script

require "shrine"
require "shrine/storage/s3"
require "forwardable"

Aws::S3::Encryption::Client.extend Forwardable
Aws::S3::Encryption::Client.delegate [:head_object, :delete_object, :config, :build_request] => :client

client = Aws::S3::Encryption::Client.new(
  region:            "...",
  access_key_id:     "...",
  secret_access_key: "...",
  encryption_key:    OpenSSL::PKey::RSA.new(1024),
)

Shrine.storages[:encrypted] = Shrine::Storage::S3.new(
  client: client,
  bucket: "...",
)

uploader      = Shrine.new(:encrypted)
uploaded_file = uploader.upload(StringIO.new("content"))
uploaded_file.download.read # => "content"

All 36 comments

If your workaround was was working, then I'd say the shrine storage needs an option to use that dependent S3 client, so it didn't require a monkey-patch.

But if the monkey-patch stopped working because APIs are different between Aws::S3::Encryption::Client and the standard... maybe it needs a separate shrine storage class?

Any interest in contributing one?

(I am not previously familiar with "S3 KMS encryption", you probably know more about it and it's ruby SDK support than I do)

Note that Shrine 2.14.0 "officially" added support for Aws::S3::Encryption::Client, so you don't have to use a custom subclass:

client = Aws::S3::Encryption::Client.new(
  kms_client: Aws::KMS::Client.new(region: 'us-east-1'),
  kms_key_id: ENV['KMS_ID']
)

Shrine.storages[:encrypted] = Shrine::Storage::S3.new(
  client: client,
  prefix: "encrypted",
  bucket: ENV['MY_ENC_BUC'],
)

However, I didn't foresee that Aws::S3::Encryption::Client only implements #get_object and #put_object operations, so some parts of Shrine::Storage::S3 won't work correctly. Note that there is an open PR for adding #head_object and #delete_object to Aws::S3::Encryption::Client – https://github.com/aws/aws-sdk-ruby/pull/1932 – so you can give a 👍there.

However, it still sucks that Aws::S3::Encryption::Client doesn't implement multipart uploads, which Shrine uses in certain cases. I will try to avoid multipart uploads in this case. I'll also add proper tests for Shrine::Storage::S3 operations with Aws::S3::Encryption::Client.

For now you should be able to work around this issue by delegating #head_object (and #delete_object so that deletions work) to the underlying Aws::S3::Client instance:

require "shrine/storage/s3"
require "forwardable"

Aws::S3::Encryption::Client.extend Forwardable
Aws::S3::Encryption::Client.delegate [:head_object, :delete_object] => :client

# ...

Note that just in case you should also increase the multipart upload threshold to some high number, so that Shrine::Storage::S3 doesn't attempt to use multipart uploads:

Shrine::Storage::S3.new(multipart_threshold: { upload: 1024**3 }, **options) # sets threshold to 1 GB

@janko thanks for detailed explanations!!!

Unfortunately, I did some tests and it's not looking good.
From reading the encryption client code, it looks like it extends the headers to include the signatures as part of the requests, that is why I don't think https://github.com/aws/aws-sdk-ruby/pull/1932 will help.

Correct me if I'm wrong:
https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-s3/lib/aws-sdk-s3/encryption/client.rb#L278-L284

Basically, the problem is that delegating any encrypted operation to the regular S3 client will lead to an Aws::S3::Errors::NotFound. For the same reason I had to overwrite the Shrine::Storage::S3#object.

I believe we should either ask AWS S3 team to provide a consistent S3 Encryption Client interface or write a custom storage which doesn't call the methods they don't implement.

From what I understand (and I could be wrong, as I hadn't used client-side encryption personally), Aws::S3::Client::Encryption doesn't use any "encryption features" from S3 itself, it performs all encryption logic on the "client" side. From the requests that I have seen, it seems that files are still uploaded to the specified location, so I don't really understand why a plain (non-encrypted) #head_object or #delete_object request wouldn't work.

I haven't tried it out with KMS as in your code, but just using a simple encryption key, and the operations worked just fine:

require "aws-sdk-s3"
require "openssl"

client = Aws::S3::Encryption::Client.new(
  region:            "<REGION>",
  access_key_id:     "<ACCESS_KEY_ID>",
  secret_access_key: "<SECRET_ACCESS_KEY>",
  encryption_key:    OpenSSL::PKey::RSA.new(1024),
)

bucket = "<BUCKET>"

client.put_object(bucket: bucket, key: "foo", body: "content")
client.get_object(bucket: bucket, key: "foo").body.string # => "content"

client.client.head_object(bucket: bucket, key: "foo") # => #<struct Aws::S3::Types::HeadObjectOutput ...>
client.client.delete_object(bucket: bucket, key: "foo") # => #<struct Aws::S3::Types::DeleteObjectOutput ...>

Could you help me understand the problem better, possibly by posting a script which produces the Aws::S3::Errors::NotFound errors you've been seeing?

@janko your example is good, except you need a set of KMS keys instead of the local encryption_key.

Example:

client = Aws::S3::Encryption::Client.new(
  kms_client: Aws::KMS::Client.new(region: 'us-east-1'),
  kms_key_id: ENV['KMS_ID']
)

The backtrace I'm getting looks like this:

> Shrine.storages[:encrypted].client.head_object(bucket: 'bucket', key: MyUpload.first.asset.id)
Aws::S3::Errors::NotFound: 
    from /usr/local/bundle/gems/aws-sdk-core-3.43.0/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call'
    from /usr/local/bundle/gems/aws-sdk-s3-1.30.1/lib/aws-sdk-s3/plugins/sse_cpk.rb:22:in `call'
    from /usr/local/bundle/gems/aws-sdk-s3-1.30.1/lib/aws-sdk-s3/plugins/dualstack.rb:26:in `call'
    from /usr/local/bundle/gems/aws-sdk-s3-1.30.1/lib/aws-sdk-s3/plugins/accelerate.rb:35:in `call'
    from /usr/local/bundle/gems/aws-sdk-core-3.43.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
    from /usr/local/bundle/gems/aws-sdk-core-3.43.0/lib/aws-sdk-core/plugins/idempotency_token.rb:17:in `call'
    from /usr/local/bundle/gems/aws-sdk-core-3.43.0/lib/aws-sdk-core/plugins/param_converter.rb:24:in `call'
    from /usr/local/bundle/gems/aws-sdk-core-3.43.0/lib/aws-sdk-core/plugins/response_paging.rb:10:in `call'
    from /usr/local/bundle/gems/aws-sdk-core-3.43.0/lib/seahorse/client/plugins/response_target.rb:23:in `call'
    from /usr/local/bundle/gems/aws-sdk-core-3.43.0/lib/seahorse/client/request.rb:70:in `send_request'
    from /usr/local/bundle/gems/aws-sdk-s3-1.30.1/lib/aws-sdk-s3/client.rb:3466:in `head_object'
    from /usr/local/lib/ruby/2.3.0/forwardable.rb:204:in `head_object'
    from (irb):4
    from /usr/local/bundle/gems/railties-4.2.11/lib/rails/commands/console.rb:110:in `start'
    from /usr/local/bundle/gems/railties-4.2.11/lib/rails/commands/console.rb:9:in `start'
    from /usr/local/bundle/gems/railties-4.2.11/lib/rails/commands/commands_tasks.rb:68:in `console'
    from /usr/local/bundle/gems/railties-4.2.11/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
    from /usr/local/bundle/gems/railties-4.2.11/lib/rails/commands.rb:17:in `<top (required)>'
    from bin/rails:9:in `require'
    from bin/rails:9:in `<main>'

Shrine here is v2.15

The script still works for me when I use KMS. I had to set up Aws::S3::Encryption::Client in the following way in order for it work:

require "aws-sdk-s3"
require "aws-sdk-kms"

client = Aws::S3::Encryption::Client.new(
  region:            "...",
  access_key_id:     "...",
  secret_access_key: "...",
  kms_key_id:        "...",
  kms_client:        Aws::KMS::Client.new(
    region:            "...",
    access_key_id:     "...",
    secret_access_key: "...",
  ),
)

# ...

The script I posted previously still executes without errors with the above setup. If you still believe this causes issues, could you post a self-contained example that shows the issue?

@janko apologies! I was using the wrong bucket.

Testing it out indeed helped to fix the issue with a minor addition.
To be able to generate the URLs, two more methods need to be forwarded to the S3 client (config and build_request.

So the final result looks like:

require "forwardable"
require "shrine"
require "aws-sdk-kms"
require "aws-sdk-s3"

Aws::S3::Encryption::Client.extend Forwardable
Aws::S3::Encryption::Client.delegate(
  [:head_object, :get_object, :config, :build_request] => :client
)

Shrine.storages[:encrypted] = Shrine::Storage::S3.new(
  prefix: 'encrypted',
  bucket: ENV['S3_BUCKETS'],
  # Sets the threshold to 1GB to not use multipart...
  multipart_threshold: { upload: 1024**3 },
  client: Aws::S3::Encryption::Client.new(
    # Use default region since IAM defaults to a single region.
    kms_client: Aws::KMS::Client.new(region: 'us-east-1'),
    kms_key_id: ENV['KMS_ID']
  )
)

Thanks a lot for helping with this!!!
This can be closed, or, if you want, I can send a PR with some docs.

The:

Aws::S3::Encryption::Client.extend Forwardable
Aws::S3::Encryption::Client.delegate(
[:head_object, :get_object, :config, :build_request] => :client
)

business seems obtuse enough that Is still wonder if code changes should be done in shrine to centralize this logic and support it out of the box. Whether a separate shrine storage class, or otherwise.

Unless there's been a misunderstanding and that isn't truly required.

Are you two getting two different results from the same reproduction script? I could try it out myself, knowing pretty much nothing about KMS, to see what happens for me.

I'm happy that it works! Ok, in that case I think that Shrine::Storage::S3 could handle that automatically. So, if :client is a Aws::S3::Encryption::Client, we could make the following changes:

  • S3#open – call #head_object on the underlying Aws::S3::Client
  • S3#delete – call #delete_object on the underlying Aws::S3::Client
  • S3#upload – avoid multipart uploads
  • S3#url – somehow delegate URL generation to Aws::S3::Client (this one will be tricky)

The issue is just that Shrine::Storage::S3 is already pretty complex as it is. I think we need to start simplifying it in general. But I think we'll be able to find a clean way to add support for Aws::S3::Encryption::Client, so that these monkey patches are not necessary.

Generally speaking, it looks more of an aws-sdk-s3 concern... It would be great to get an update from them before we move forward with S3 client specific updates...

@stas I also use s3 storage and KMS for client-side encryption. This issue helps me a lot.
But I can't understand how to decrypt the downloaded file.
@janko you added instruction for client-side encryption via AWS KMS, but there no information about downloading and decryption. Here some information about downloading server-side encrypted files.

Could you @stas @janko help me to understand how to decrypt the file?

Could you @stas @janko help me to understand how to decrypt the file?

We implemented a _downloads_ endpoint, which calls shrine asset.download and then passes it to send_file.

@stas Like this one?

class DownloadsController < ApplicationController
  def documents
    document = Document.find(id) if Document.exists?(id: params[:id])

    if document.nil? || !document&.file
      redirect_to root_path and return
    end

    file      = document.file.download { |tempfile| tempfile.read }
    name      = "#{document.name}.pdf"
    type      = 'application/pdf'

    send_file file, filename: name, type: type, disposition: 'attachment'
  end
end

I added a question to the StackOverflow with more information.

You mean that asset.download decrypt file automatically?

@rmalkevy :100:

Minor suggestion, consider serving the file directly after the download, instead of loading it into the memory since Shrine will create a tempfile already which you can pass to send_file. Something like:

    tfile = document.file.download
    send_file(tfile)
    tfile.close

Also, you might have the original file name in the metadata already:
https://github.com/shrinerb/shrine/blob/v2.16.0/doc/metadata.md#extracting-metadata

@stas thank you for minor suggestion, it saves me memory) But my file is still don't open.

When I use default :store without encrypting - my file downloaded like pdf.
But when I use in the model encrypting - my file downloaded like bytes.

class Document < ApplicationRecord
  # include DocumentsUploader::Attachment.new(:file) # -> downloaded like pdf
  include DocumentsUploader::Attachment.new(:file, {store: :encrypted}) # -> downloaded like bytes
end

Are you sure that file has to decrypt automatically?

@rmalkevy I've made a mistake in my comment about delegation; it said to delegate [:head_object, :get_object], but it should have said [:head_object, :delete_object]. Aws::S3::Encryption::Client#get_object is already implemented and it's what does the decryption, so adding that #get_object delegate would use the regular Aws::S3::Client#get_object which doesn't do any decryption.

With this, downloads are automatically decrypted:


Script

require "shrine"
require "shrine/storage/s3"
require "forwardable"

Aws::S3::Encryption::Client.extend Forwardable
Aws::S3::Encryption::Client.delegate [:head_object, :delete_object, :config, :build_request] => :client

client = Aws::S3::Encryption::Client.new(
  region:            "...",
  access_key_id:     "...",
  secret_access_key: "...",
  encryption_key:    OpenSSL::PKey::RSA.new(1024),
)

Shrine.storages[:encrypted] = Shrine::Storage::S3.new(
  client: client,
  bucket: "...",
)

uploader      = Shrine.new(:encrypted)
uploaded_file = uploader.upload(StringIO.new("content"))
uploaded_file.download.read # => "content"

@janko You saved my life :)))

Unfortunately support for KMS is completely broken in Shrine v3.
The file downloads, but it's still encrypted.

I patched the S3 storage class to skip support for chunked downloads to have it working:

Shrine::Storage::S3.class_eval do
  def open(id, rewindable: true, **options)
    obj = client.get_object(key: object(id).key, bucket: bucket.name, **options)
    Down::ChunkedIO.new(chunks: obj.body.enum_for(:each))
  end
end

It looks like this commit changed the way it was working before: https://github.com/shrinerb/shrine/commit/a36c5fe1e182c19307d1a003d9453fddbf477e70

@janko I could try to submit a patch, but I'm not sure I understand why Shrine::Storage::S3#get_object is so custom, also the docs don't help a lot understanding what is going on there. Do you think you can help us understand the purpose of it please?

While playing with the aws-sdk-s3, I discovered the following:

> object(id).data
=> #<struct Aws::S3::Types::HeadObjectOutput delete_marker=nil, accept_ranges="bytes", expiration=nil, restore=nil, last_modified=2020-01-20 21:03:06 +0000, content_length=96...>

> object(id).get
=> #<struct Aws::S3::Types::GetObjectOutput body=#<StringIO:0x0000560c518a7150>,...>

Would it be enough to use the first call to get the content_length and follow with the next one to get the body of the object? Sorry, if it sounds stupid, it's just not obvious what's going on there...

Thanks in advance!

Sorry to break the support, let me give some background, and then we can talk about possible solutions.

The reason we need to fetch the object's content length is so that the IO object returned by S3#open has a non-empty #size attribute. This is used when refreshing metadata in restore_cached_data plugin, which is used to prevent metadata tampering by fetching the actual file info.

Initially we were just doing object.content_length, which calls object.data, which makes a HEAD request via #head_object. This was working well until someone reported that server-side encryption parameters don't work. Eventually I found that it's because :sse_* options need to be forwarded to #head_object as well. However, not all #get_object params are supported by #head_object, so we had to filter params out.

Before releasing Shine 3.0, I was comparing Shrine's on-the-fly processing performance with ActiveStorage's. In that process I realized that the additional HEAD request can make a difference in performance. So I eventually found a way to eliminate it, with the code you can see today.

Another complication is that at some point the aws-sdk-s3 gem broke initializing Aws::S3::Bucket with an Aws::S3::Encryption::Client instance, which breaks usage with Shrine:

require "aws-sdk-s3"
client = Aws::S3::Encryption::Client.new(encryption_key: "a" * 16, stub_responses: true)
Aws::S3::Bucket.new(name: "shrine-testing", client: client)
NoMethodError: undefined method `config' for #<Aws::S3::Encryption::Client:0x00007fcddfc3f6b8>
from /Users/janko/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/aws-sdk-s3-1.60.1/lib/aws-sdk-s3/customizations/bucket.rb:13:in `block in <class:Bucket>'

I will try adding back support for client-side encryption, this time with proper tests. It will now be a module that's automatically prepended to Shrine::Storage::S3, so that the encryption-specific logic is separate from default one.

Thanks @janko, confirming this has a fix.

Here's how the configuration looks like:

Shrine.storages[:encrypted] = Shrine::Storage::S3.new(
  prefix: 'encrypted',
  bucket: ENV['AWS_S3_BUCKET'],
  client: Aws::S3::Encryption::Client.new(kms_key_id: ENV['AWS_KMS_ID'])
)

Good, I might release this in a patch version shortly.

Could you also verify that it works without any method forwarding patches (those shouldn’t be needed anymore)?

Could you also verify that it works without any method forwarding patches (those shouldn’t be needed anymore)?

Yep, looks good! Feel free to release the changes :bowing_man:

this is a great feature to have supported out of the box, thanks to all!

Just a side question.. I've tried implementing the encryption as described here, and it seems to work, kinda.
I cannot display the file locally after upload. If I call url it generates an s3 link, but the image it corrupted.

Also, the amazon s3 console doesn't say anything about the file being encrypted.

I've tried looking through the documentation, and I cannot see how to use the low level s3.upload and s3.open methods inside a normal EncryptedUploader < Shrine class.

Can someone point me the right direction?

I cannot display the file locally after upload. If I call url it generates an s3 link, but the image it corrupted.

The generated URL doesn't include the right headers to decrypt the file. You need to use #download to fetch the right file.

Also, the amazon s3 console doesn't say anything about the file being encrypted.

They don't have support for all the encryption functionality...

The short answer, is basically to use #download, and serve it with something like #send_file, which is up to you how to implement it.

okay.. cool.. i'll try to see if i can get it working.. thank! :)

Well, using the download and copying the tempfile somewhere on the computer shows that the file is corrupted.. So I dont think the decryption is working..

original filesize is 9928 bytes, downloaded file is 9936 bytes.. the KMS key is set up to be synchronous.. weird..

Could you create a self-contained example that reproduces the issue? Client-side encryption should be working correctly, if there any bugs it would be good to get to the bottom of them.

I've tried the "content" example above..

the read produces "\xCB>4\x01\xAD\xFA~v\xEFMR\xEE\xF3\aD\xEB"

# Config
require 'shrine'
require 'shrine/storage/s3'
require 'shrine/storage/file_system'
require 'forwardable'

Shrine.plugin :activerecord
Shrine.plugin :determine_mime_type

Aws::S3::Encryption::Client.extend Forwardable
Aws::S3::Encryption::Client.delegate [:head_object, :delete_object, :config, :build_request] => :client

encrypted_s3_client = Aws::S3::Encryption::Client.new(
    kms_client: Aws::KMS::Client.new(region: ENV.fetch("AWS_REGION")),
    kms_key_id: ENV.fetch("AWS_KMS_KEY"),
    access_key_id: ENV.fetch("AWS_ACCESS_KEY_ID"),
    secret_access_key: ENV.fetch("AWS_SECRET_ACCESS_KEY"),
    region: ENV.fetch("AWS_REGION")
)

public_s3 = Shrine::Storage::S3.new(
    prefix: 'store',
    access_key_id: ENV.fetch("AWS_ACCESS_KEY_ID"),
    secret_access_key: ENV.fetch("AWS_SECRET_ACCESS_KEY"),
    bucket: ENV.fetch("S3_BUCKET_NAME"),
    region: ENV.fetch("AWS_REGION")
)

encrypted_s3 = Shrine::Storage::S3.new(
    client: encrypted_s3_client,
    prefix: 'crypt',
    bucket: ENV.fetch("S3_BUCKET_NAME"),
    multipart_threshold: { upload: 1024**3 }
)

Shrine.storages = {
    cache: Shrine::Storage::FileSystem.new('public/uploads', prefix: 'cache'),
    store: public_s3,
    encrypted: encrypted_s3
}

# code

uploader = Shrine.new(:encrypted)
uploaded_file = uploader.upload(StringIO.new("content"))
uploaded_file.download.read # => "\xBAb8w\x11M S45y\x17\x8ET\x03\xB6"

# log
irb(main):001:0> uploader = Shrine.new(:encrypted)
=> #<Shrine:0x00007f8892776510 @storage_key=:encrypted>
irb(main):002:0> uploader.upload(StringIO.new("content"))
[Aws::KMS::Client 200 2.653704 0 retries] generate_data_key(key_id:"[FILTERED]",encryption_context:{"kms_cmk_id"=>"[FILTERED]"},key_spec:"AES_256")  

[Aws::S3::Client 200 2.923491 0 retries] put_object(body:"[FILTERED]",content_type:"text/plain",bucket:"mybucket",key:"crypt/772b436067e281e82f744c1d178902b6",metadata:"[FILTERED]")  

=> #<Shrine::UploadedFile storage=:encrypted id="772b436067e281e82f744c1d178902b6" metadata={"filename"=>nil, "size"=>7, "mime_type"=>"text/plain"}>
irb(main):003:0> f = _
=> #<Shrine::UploadedFile storage=:encrypted id="772b436067e281e82f744c1d178902b6" metadata={"filename"=>nil, "size"=>7, "mime_type"=>"text/plain"}>
irb(main):004:0> f.download.read
[Aws::S3::Client 200 0.196672 0 retries] get_object(bucket:"mybucket",key:"crypt/772b436067e281e82f744c1d178902b6")  

=> "\xBAb8w\x11M S45y\x17\x8ET\x03\xB6"

@mikkelwf what version of shrine are you using?

The latest shrine configuration example you can use is this here:
https://github.com/shrinerb/shrine/issues/348#issuecomment-576824523

@mikkelwf what version of shrine are you using?

The latest shrine configuration example you can use is this here:
#348 (comment)

I use version 3.2.1

i've tested with the setup you referenced, no change.

i also updated aws-sdk-core from 3.92.0 to 3.94.0, still no change..

Hi there,
unfortunately I have to report back that the S3 encryption is not working anymore (again).

I'm using the latest released version of shrine:

Using aws-sdk-s3 1.78.0
Using shrine 3.2.2

And this is the configuration:

Shrine.storages[:encrypted] = Shrine::Storage::S3.new(
  prefix: 'encrypted',
  bucket: ENV['AWS_S3_BUCKET'],
  client: Aws::S3::EncryptionV2::Client.new(
    kms_key_id: ENV['AWS_KMS_ID'],
    key_wrap_schema: :kms_context,
    content_encryption_schema: :aes_gcm_no_padding,
    security_profile: :v2_and_legacy
  )
)

Uploading and download of the files works, but the resulted file is garbage... I suspect it's not decrypted.
Anybody else having similar issues?

It doesn't work with encryption client V2, we'll need to add support for it separately.

Shrine 3.3.0 has been released, which adds support for Aws::S3::EncryptionV2::Client.

Thanks @janko, everything works now! :bow:

Was this page helpful?
0 / 5 - 0 ratings

Related issues

pmackay picture pmackay  Â·  4Comments

waclock picture waclock  Â·  8Comments

printercu picture printercu  Â·  10Comments

bmedici picture bmedici  Â·  3Comments

technodrome picture technodrome  Â·  4Comments