Shrine: NoMethodError (undefined method `bytesize' for #<Array:0x000000000a721be0>):

[thebravoman]

---

Brief Description

Uploading a picture. Performing

class GenerateShrineDerivativesJob < ApplicationJob

  def perform record
    record.image_derivatives!
    record.save
  end

end

in a Delayed::Job.

The following error occurs:

d"=>"gid://fll-casts/ContentPicture/3835"}] (id=1482666) (queue=default) RUNNING
[ActiveJob] [GenerateShrineDerivativesJob] [474a896b-9bcc-45d8-84ba-602e306834c8] Performing GenerateShrineDerivativesJob (Job ID: 474a896b-9bcc-45d8-84ba-602e306834c8) from DelayedJob(default) enqueued at 2020-07-22T18:54:30Z with arguments: #<GlobalID:0x000000000a6b35f0 @uri=#<URI::GID gid://fll-casts/ContentPicture/3835>>
[ActiveJob] [GenerateShrineDerivativesJob] [474a896b-9bcc-45d8-84ba-602e306834c8] Error performing GenerateShrineDerivativesJob (Job ID: 474a896b-9bcc-45d8-84ba-602e306834c8) from DelayedJob(default) in 97.74ms: NoMethodError (undefined method `bytesize' for #<Array:0x000000000a721be0>):
/app/vendor/bundle/ruby/2.6.0/gems/down-5.1.1/lib/down/chunked_io.rb:173:in `readpartial'
/app/vendor/bundle/ruby/2.6.0/gems/shrine-3.2.1/lib/shrine/uploaded_file.rb:146:in `copy_stream'
/app/vendor/bundle/ruby/2.6.0/gems/shrine-3.2.1/lib/shrine/uploaded_file.rb:146:in `block in stream'
/app/vendor/bundle/ruby/2.6.0/gems/shrine-3.2.1/lib/shrine/uploaded_file.rb:98:in `open'
/app/vendor/bundle/ruby/2.6.0/gems/shrine-3.2.1/lib/shrine/uploaded_file.rb:146:in `stream'
/app/vendor/bundle/ruby/2.6.0/gems/shrine-3.2.1/lib/shrine/uploaded_file.rb:122:in `download'
/app/vendor/bundle/ruby/2.6.0/gems/shrine-3.2.1/lib/shrine/plugins/derivatives.rb:272:in `process_derivatives'
/app/vendor/bundle/ruby/2.6.0/gems/shrine-3.2.1/lib/shrine/plugins/derivatives.rb:184:in `create_derivatives'
/app/vendor/bundle/ruby/2.6.0/gems/shrine-3.2.1/lib/shrine/plugins/derivatives.rb:44:in `block in define_model_methods'
/app/app/jobs/generate_shrine_derivatives_job.rb:7:in `perform'

Expected behavior

This was working a few days ago, now it just stopped.

Actual behavior

Exception occurred - shown above.

Simplest self-contained example code to demonstrate issue

Upload is on heroku. Not sure if this is important
Image uploaded has

require "image_processing/mini_magick"
class ImageUploader < Shrine
...
  Attacher.derivatives do |original|
    magick = ImageProcessing::MiniMagick.source(original)
    { 
      p55:    magick.resize_to_limit!(98, 55),
      p169:   magick.resize_to_limit!(300, 169),
      p360:   magick.resize_to_limit!(640, 360),
      p720:   magick.resize_to_limit!(1280, 720),
      p1080:  magick.resize_to_limit!(1920, 1080)
    }
  end
...
end

System configuration

Ruby version: 2.6.5

Shrine version: shrine (3.2.1)

[janko]

The aws-sdk-core version 3.104.0 introduced a change which broke the Shrine::Storage::S3#open method.

This has been fixed in Shrine master, so you can pull directly from there (I will make a release in the next few days). Alternatively you can downgrade the aws-sdk-core gem to version 3.103.0 or below.

[thebravoman]

Thanks @janko

For anyone out there I finally decided to run a test on production. I've done much to improve the specs for a picture upload, but between aws, shrine, heroku, active record, image_magic, delayed_job, the weather in France and many others this is the piece of code that is run every day on production in a rake job to make sure users could upload pictures and if they can't we will get some proper feedback.

Hope it could help

  # I've done much to improve the specs for a picture upload but between aws, 
  # shrine, heroku, active record, image_magic, delayed_job, the weather in France and many others.
  # Yet there are still errors from time to time. 
  # 
  # This is the piece of code that is run every reqularly on production in a rake job 
  # to make sure users could upload pictures and if they can't we will get some timely feedback.
  # 
  # Inspired after https://github.com/shrinerb/shrine/issues/492
  # 
  # This is a production test. It depends on actuall records available on production. It tries
  # to minimize its impact and that's why it is not creating and removing objects. It is working
  # on the same content pictures. 
  # 
  # If a 'test fails' the method throws and exception. 
  def picture_could_be_uploaded_and_derivatives_generated
    source_content_picture = ContentPicture.find(3836)
    target_content_picture = ContentPicture.find(3837)
    target_content_picture.image_attacher.destroy_attached
    throw "Picture_could_be_uploaded_and_derivatives_generated; save of target content picture after destroying the attachment failed" unless target_content_picture.save
    target_content_picture.image_attacher.attach(open(source_content_picture.image_url))
    target_content_picture.save
    job = Delayed::Job.where("handler like '%GenerateShrineDerivativesJob%'").order(created_at: :desc).first
    throw "Picture_could_be_uploaded_and_derivatives_generated; no job in scheduled" if job == nil
    throw "Picture_could_be_uploaded_and_derivatives_generated; job is not the right job. " unless job.handler.include? "gid://fll-casts/ContentPicture/3837"
    Delayed::Worker.new.run(job)
    throw "Picture_could_be_uploaded_and_derivatives_generated; job still exists #{job}" if Delayed::Job.exists?(job.id)
    target_content_picture.reload
    throw "Picture_could_be_uploaded_and_derivatives_generated; not all derivative are available" if [:p55,:p169,:p360,:p720,:p1080].map{|a| target_content_picture.image_derivatives[a] != nil}.include?(false)
  end

[janko]

Since this is a pretty important bug, I've made a hotfix release 3.2.2, which only includes a fix for this issue.

/cc @brettwgreen

[jrochkind]

I was curious to see the fix, and there wasn't a PR or commit linked to in this issue. I'm not sure if there was a PR, but I found the commit by doing a v.3.2.1...v.3.2.2 compare.

https://github.com/shrinerb/shrine/commit/83f2e6a4fdb5753171eb9316dfc23ec0d1d6eff9

@janko , you might want to consider (or not!) making all/most commits via PR, even when it's you the committer doing them. I find it aids in visibility and self-documentation of the codebase, and most of my projects I make most (if not all) commits via PR, even when I'm the only one doing them. (or not! you do you! just a thought!)

You can also/alternately of course make a bare commit (without a PR) show up linked to an issue on github just by mentioning the issue number in the commit message. Another way to make the code more discoverable/documented!

[thebravoman]

Can confirm. Our test passed over this with 3.2.2. Thanks @janko

[brettwgreen]

Since this is a pretty important bug, I've made a hotfix release 3.2.2, which only includes a fix for this issue.

/cc @brettwgreen

Awesome... I will pull and run my tests with latest AWS SDK and let you know a bit later this evening. Thank you!!

[janko]

Glad to hear the fix is working 👍

@jrochkind:

you might want to consider (or not!) making all/most commits via PR, even when it's you the committer doing them. I find it aids in visibility and self-documentation of the codebase, and most of my projects I make most (if not all) commits via PR, even when I'm the only one doing them. (or not! you do you! just a thought!)

I do make some changes PRs, but usually when I want them to be reviewed. I still prefer to commit directly to master for these kind of fixes.

You can also/alternately of course make a bare commit (without a PR) show up linked to an issue on github just by mentioning the issue number in the commit message. Another way to make the code more discoverable/documented!

In this case, this GH issue was created after I already committed the fix to master. For the hotfix I've branched from v3.2.1, cherry-picked the commit from master, tagged v3.2.2, and then did git push --tags. I didn't think it made sense to mention the issue in the cherry-picked commit, since I've already mentioned the original master commit in https://github.com/shrinerb/shrine/issues/492#issuecomment-662711923.

[brettwgreen]

Since this is a pretty important bug, I've made a hotfix release 3.2.2, which only includes a fix for this issue.
/cc @brettwgreen

Awesome... I will pull and run my tests with latest AWS SDK and let you know a bit later this evening. Thank you!!

Updated to Shrine 3.2.2 and aws-sdk-core 3.104.3 and all tests are now passing that were previously getting this error. Thanks for the patch release!