Aws-sdk-ruby: AWS::S3::Object upload_file doesn't support StringIO

Created on 2 Dec 2016  路  3Comments  路  Source: aws/aws-sdk-ruby

I have a use case where I have some in memory data that I'd like to send to S3 over multipart or not depending on size and take advantage of AWS::S3::Object.upload_file on.

I was hoping to do this and use StringIO to have a "fake" file object:

    data = "...................."
    io_obj = StringIO.new(data)       
    s3_resource = Aws::S3::Resource.new(client: s3)
    obj = s3_resource.bucket(bucket_name).object(key)
    obj.upload_file(io_obj)

It all works until it gets this to this line then throws an exception as File.size doesn't support StringIO (even though File.size claims to support any IO object)

https://github.com/aws/aws-sdk-ruby/blob/master/aws-sdk-resources/lib/aws-sdk-resources/services/s3/file_uploader.rb#L31

The File.size issue is the only part of the flow blocking this from working - any chance you can check for source.size on the object itself (which works for both IO and StringIO objects) instead of passing through File.size to check? (or else check for StringIO as a type and do .size then)

I could skip using upload_file but then I don't get multipart support in case the in-memory data is large (rare), and I'm hoping to avoid having to write this data to disk.

feature-request

Most helpful comment

can't you just use Object#put(body: io_object)? that supports both file and stringIO objects

All 3 comments

There are additional reasons that #upload_file will not support a StringIO object. The #upload_file avoids buffering multiple file parts into memory by open multiple file objects. I suppose that the method could be abstracted away from file by adding a wrapper interface that returns "file parts".

can't you just use Object#put(body: io_object)? that supports both file and stringIO objects

Added to feature request backlog.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

pchaganti picture pchaganti  路  6Comments

thegreyfellow picture thegreyfellow  路  3Comments

domcleal picture domcleal  路  5Comments

robbkidd picture robbkidd  路  5Comments

jtruelove picture jtruelove  路  5Comments