Etcher: regression: writing image with bmap results in corrupt data

[dlech]

• Etcher version: 1.5.30
• Operating system and architecture: Win 10 1809 x64
• Image flashed: https://education.lego.com/en-us/support/mindstorms-ev3/python-for-ev3
• Do you see any meaningful error information in the DevTools? No

This image uses a bmap to speed up flash time.

Flashing this image using Etcher 1.5.30 results in an unbootable SD card.

Flashing the same image with Etcher 1.4.9 does not corrupt the image and the SD card boots as expected.

Everything other than the Etcher version was the same between the two tests (same computer, same SD card, etc.).

[dlech]

Here is a binary diff showing where a file (uEnv.txt in the FAT partition) is written to a different location from where it should be.

   Visuel HexDiff v 0.0.53 by tTh 2007                             dec   7bits  
 4538416   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 4538432   00 00 03 18 00 00 00 00 23 20 42 6f 6f 74 20 73            # Boot s
 4538448   63 72 69 70 74 20 74 6f 20 63 68 61 69 6e 2d 6c    cript to chain-l
 4538464   6f 61 64 20 6e 65 77 65 72 20 55 2d 42 6f 6f 74    oad newer U-Boot
 4538480   20 66 72 6f 6d 20 6f 66 66 69 63 69 61 6c 20 4c     from official L
 4538496   45 47 4f 20 66 69 72 6d 77 61 72 65 0a 23 0a 23    EGO firmware # #
 4538512   20 54 68 65 20 55 2d 42 6f 6f 74 20 74 68 61 74     The U-Boot that
 4538528   20 73 68 69 70 73 20 77 69 74 68 20 74 68 65 20     ships with the 
 4538544   6f 66 66 69 63 69 61 6c 20 66 69 72 6d 77 61 72    official firmwar
 4538560   65 20 66 6f 72 20 4c 45 47 4f 20 4d 49 4e 44 53    e for LEGO MINDS
** /media/VMs/ev3dev-stretch-ev3-generic-2019-03-03.1887436800  4538416   0%    
 4538416   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 4538432   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 4538448   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 4538464   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 4538480   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 4538496   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 4538512   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 4538528   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 4538544   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 4538560   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   /media/VMs/bad-ev3dev.img                        15931539456  4538416   0%   

   Visuel HexDiff v 0.0.53 by tTh 2007                             dec   7bits  
 4538928   73 63 72 69 70 74 2e 0a 23 0a 23 20 47 65 6e 65    script. # # Gene
 4538944   72 61 74 65 20 62 6f 6f 74 2e 73 63 72 20 62 79    rate boot.scr by
 4538960   20 72 75 6e 6e 69 6e 67 3a 0a 23 20 2e 2f 74 6f     running: # ./to
 4538976   6f 6c 73 2f 6d 6b 69 6d 61 67 65 20 2d 63 20 6e    ols/mkimage -c n
 4538992   6f 6e 65 20 2d 41 20 61 72 6d 20 2d 54 20 73 63    one -A arm -T sc
 4539008   72 69 70 74 20 2d 64 20 62 6f 61 72 64 2f 6c 65    ript -d board/le
 4539024   67 6f 2f 65 76 33 2f 63 68 61 69 6e 2d 62 6f 6f    go/ev3/chain-boo
 4539040   74 2e 63 6d 64 20 62 6f 6f 74 2e 73 63 72 0a 23    t.cmd boot.scr #
 4539056   0a 0a 73 65 74 65 6e 76 20 75 62 6f 6f 74 61 64      setenv ubootad
 4539072   64 72 20 30 78 63 31 30 63 30 30 30 30 0a 73 65    dr 0xc10c0000 se
** /media/VMs/ev3dev-stretch-ev3-generic-2019-03-03.1887436800  4538928   0%    
 4538928   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 4538944   00 00 03 18 00 00 00 00 23 20 42 6f 6f 74 20 73            # Boot s
 4538960   63 72 69 70 74 20 74 6f 20 63 68 61 69 6e 2d 6c    cript to chain-l
 4538976   6f 61 64 20 6e 65 77 65 72 20 55 2d 42 6f 6f 74    oad newer U-Boot
 4538992   20 66 72 6f 6d 20 6f 66 66 69 63 69 61 6c 20 4c     from official L
 4539008   45 47 4f 20 66 69 72 6d 77 61 72 65 0a 23 0a 23    EGO firmware # #
 4539024   20 54 68 65 20 55 2d 42 6f 6f 74 20 74 68 61 74     The U-Boot that
 4539040   20 73 68 69 70 73 20 77 69 74 68 20 74 68 65 20     ships with the
 4539056   6f 66 66 69 63 69 61 6c 20 66 69 72 6d 77 61 72    official firmwar
 4539072   65 20 66 6f 72 20 4c 45 47 4f 20 4d 49 4e 44 53    e for LEGO MINDS
   /media/VMs/bad-ev3dev.img                        15931539456  4538928   0%   

[dlech]

The same image flashed with v1.4.9 has the file in the right place:

   Visuel HexDiff v 0.0.53 by tTh 2007                             dec   7bits  
 4538416   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 4538432   00 00 03 18 00 00 00 00 23 20 42 6f 6f 74 20 73            # Boot s
 4538448   63 72 69 70 74 20 74 6f 20 63 68 61 69 6e 2d 6c    cript to chain-l
 4538464   6f 61 64 20 6e 65 77 65 72 20 55 2d 42 6f 6f 74    oad newer U-Boot
 4538480   20 66 72 6f 6d 20 6f 66 66 69 63 69 61 6c 20 4c     from official L
 4538496   45 47 4f 20 66 69 72 6d 77 61 72 65 0a 23 0a 23    EGO firmware # #
 4538512   20 54 68 65 20 55 2d 42 6f 6f 74 20 74 68 61 74     The U-Boot that
 4538528   20 73 68 69 70 73 20 77 69 74 68 20 74 68 65 20     ships with the
 4538544   6f 66 66 69 63 69 61 6c 20 66 69 72 6d 77 61 72    official firmwar
 4538560   65 20 66 6f 72 20 4c 45 47 4f 20 4d 49 4e 44 53    e for LEGO MINDS
** /media/VMs/ev3dev-stretch-ev3-generic-2019-03-03.1887436800  4538416   0%    
 4538416   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 4538432   00 00 03 18 00 00 00 00 23 20 42 6f 6f 74 20 73            # Boot s
 4538448   63 72 69 70 74 20 74 6f 20 63 68 61 69 6e 2d 6c    cript to chain-l
 4538464   6f 61 64 20 6e 65 77 65 72 20 55 2d 42 6f 6f 74    oad newer U-Boot
 4538480   20 66 72 6f 6d 20 6f 66 66 69 63 69 61 6c 20 4c     from official L
 4538496   45 47 4f 20 66 69 72 6d 77 61 72 65 0a 23 0a 23    EGO firmware # #
 4538512   20 54 68 65 20 55 2d 42 6f 6f 74 20 74 68 61 74     The U-Boot that
 4538528   20 73 68 69 70 73 20 77 69 74 68 20 74 68 65 20     ships with the
 4538544   6f 66 66 69 63 69 61 6c 20 66 69 72 6d 77 61 72    official firmwar
 4538560   65 20 66 6f 72 20 4c 45 47 4f 20 4d 49 4e 44 53    e for LEGO MINDS
   /media/VMs/good-ev3dev.img                       7948206080  4538416   0%    

[dlech]

I've narrowed down the version a bit. 1.5.26 seems OK and 1.5.28 is bad (there are no assets for testing 1.5.27).

https://github.com/balena-io/etcher/compare/v1.5.26...v1.5.28

This leads me to believe that 2c835437e9f2c9e688bb713af64c7cd083c469a8 introduced the problem.

[thundron]

@dlech Thank you for the thorough analysis, we'll look into this

[dlech]

Thanks, I hope that it can be prioritized since it is affecting many users (it is being reported on Facebook quite a bit).

[lurch]

4538944 - 4538432 = 512 (i.e. the size of one disk-block), so maybe it's a simple off-by-one error? :man_shrugging:

[dlech]

Good call:

https://github.com/balena-io-modules/etcher-sdk/blob/2811c7be2597706361778aca6932b70f41d141fd/lib/source-destination/zip.ts#L50

[laurensvalk]

Thanks for fixing this so quickly. Is there an estimated timeline for when this will make it into the next public release?

[zvin]

@laurensvalk it's in 1.5.31, you can grab it from here https://github.com/balena-io/etcher/releases

[laurensvalk]

That's awesome, thanks! Keep up the great work :+1: