Etcher: "rawr i'm a dinosaur" when writing bz2 file

[imrehg]

• Etcher version: git head at e47a70606ce017a14a3ab29a1edeee5e6399b3b1
• Operating system and architecture: ArchLinux x64
• Do you see any meaningful error information on DevTools?

tracker.js:10 Error: rawr i'm a dinosaur
    at ChildProcess.child.on (/home/greg/resin/prog/etcher/lib/src/child-writer/index.js:92)
    at emitTwo (events.js:106)
    at ChildProcess.emit (events.js:191)
    at handleMessage (internal/child_process.js:718)
    at Pipe.channel.onread (internal/child_process.js:444)
a.(anonymous function)                       @ tracker.js:10
(anonymous function)                         @ /home/greg/resin/prog/etcher/node_modules/angular/angular.js:13708
logException                                 @ /home/greg/resin/prog/etcher/lib/gui/modules/analytics.js:157
reportException                              @ /home/greg/resin/prog/etcher/lib/gui/modules/error.js:45
ImageWriterService.flash.then.catch          @ /home/greg/resin/prog/etcher/lib/gui/pages/main/controllers/flash.js:82
processQueue                                 @ /home/greg/resin/prog/etcher/node_modules/angular/angular.js:16170
(anonymous function)                         @ /home/greg/resin/prog/etcher/node_modules/angular/angular.js:16186
$eval                                        @ /home/greg/resin/prog/etcher/node_modules/angular/angular.js:17444
$digest                                      @ /home/greg/resin/prog/etcher/node_modules/angular/angular.js:17257
(anonymous function)                         @ /home/greg/resin/prog/etcher/node_modules/angular/angular.js:17483
completeOutstandingRequest                   @ /home/greg/resin/prog/etcher/node_modules/angular/angular.js:5955
(anonymous function)                         @ /home/greg/resin/prog/etcher/node_modules/angular/angular.js:6234
d.util.hasFunction.f.(anonymous function)    @ tracker.js:3

Get this rawr i'm a dinosaurerror while writing the bz2 compressed version of the Raspbian Jessie image. Very slow writing (~3x slower than other compressed images), and breaks around ~95%.

Happens both with 1.0.0-beta15 AppImage, and git head npm start.

[jviotti]

Haha wtf? This must be coming from the bz2 decompression module we use (can't think of anything else). Could be a funny way to say a package is deprecated?

[jviotti]

Indeed: https://github.com/regular/unbzip2-stream/blob/845023bb9b032706b6527704334cb0403a865f6c/lib/bzip2.js#L274

Do you mind creating a new issue on that project, probably linking to the problematic image? The maintainer is very responsive and I honestly don't have the knowledge to debug such as bz2 algorithm issue.

[imrehg]

Sure thing :) Filed bug: https://github.com/regular/unbzip2-stream/issues/10

[regular]

Ha ha, I knew this would at some point show up in an application window. I did not write the actual decompression code (and neither did I author the error messages). I just wrapped it into a stream interface and packaged it. So, your guess is as good as mine, I'm afraid. The code is not very .. accessible unfortunately.

[regular]

PR would be appreciated though!

[jviotti]

No worries! I'll study how bzip2 works, and will try to fix or at least make sense out of it whenever I can allocate some time for it!

[WasabiFan]

@jviotti @lurch actually seems to have figured it out already: https://github.com/regular/unbzip2-stream/issues/10

[alexandrosm]

@jviotti where are we with this one?

[jviotti]

We have https://github.com/cscott/compressjs, which is supposed to have
a fix for this issue, however they don't expose a standard NodeJS
implementation (they roll their own, which I can't get to play with
NodeJS, since its all done inside a single function), so we don't have a
good bzip2 decompression alternative yet.

On Thu, Dec 01, 2016 at 08:02:09PM -0800, Alexandros Marinos wrote:

@jviotti where are we with this one?

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/resin-io/etcher/issues/734#issuecomment-264368271

--
Juan Cruz Viotti
Software Engineer

[lurch]

Maybe it's worth nudging the author of unbzip2-stream and asking him to pick up the bzip2 fixes from compressjs ? I could do that if you like?

[jviotti]

@lurch That would be amazing! Thanks a lot!

[regular]

@lurch Even better than nudging: a PR with test. I don't use that module in any of my projects and I am no expert on bzip at all, so I'm in no way more qualified to apply that patch. I will however merge a PR asap and republish the npm module.

[lurch]

Even better than nudging...

Haha, I'll see when I have time to put something together :smile_cat:

[alexandrosm]

@jviotti how much work would be to either port the fix or use another library? This is our second oldest bug :)

[jviotti]

I started hacking on it at some point, got too deep into the rabbit hole, and ended up abandoning it. I'm happy to give it another go.

[alexandrosm]

I guess we could "won't fix" this. Not sure it's worth your time.

--

Alexandros Marinos

Founder & CEO, Resin.io

[phone]

@alexandrosm

On Thu, Mar 2, 2017 at 8:18 AM, Juan Cruz Viotti [e-mail]
wrote:

I started hacking on it at some point, got too deep into the rabbit hole,
and ended up abandoning it. I'm happy to give it another go.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/734#issuecomment-283699639,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCJjmnAqgk5P_1x198moV5rSDXgG4ks5rhuvAgaJpZM4KK2Ef
.

[jviotti]

I guess there's no reason to not keep it open until a clearer way forward appears.

[regular]

This might be fixed in [email protected] Let me know if it is!

[jviotti]

Amazing stuff @regular , thanks a lot for this. I can't find any image that reproduces the issue, so I'll upgrade this on Etcher, and we'll see if it ever happens again, unless @imrehg still has the problematic image somewhere.

[lurch]

Unfortunately I no longer have a copy of @imrehg 's 2016-05-27-raspbian-jessie.img.bz2 file - I must have deleted it due to lack of disk space :-( (and @imrehg 's dropbox download link is no longer working)
So I instead attempted to recreate it by downloading, extracting and recompressing http://downloads.raspberrypi.org/raspbian/images/raspbian-2016-05-31/2016-05-27-raspbian-jessie.zip and that did then recreate the infamous dinosaur-error with [email protected] ;-)
And upgrading to [email protected] _did_ then eliminate the bug, so I can confirm v1.0.11 fixes this problem :smile:

[jviotti]

Amazing news!!