Build: .img file corrupt after build

[multivac61]

Running the command ./compile.sh BRANCH=dev BOARD=nanopineo KERNEL_ONLY=no PROGRESS_DISPLAY=plain RELEASE=xenial BUILD_DESKTOP=yes NO_APT_CACHER=yes builds an .img file that does stalls when booting up uboot. I get the following error through my serial debug pins:

U-Boot 2016.09-armbian (Oct 11 2016 - 11:26:23 +0000) Allwinner Technology                                                                  

CPU:   Allwinner H3 (SUN8I 1680)
Model: Xunlong Orange Pi One
DRAM:  512 MiB
MMC:   SUNXI SD/MMC: 0
*** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   phy interface0
eth0: ethernet@1c30000
Hit any key to stop autoboot:  0 
switch to partitions #0, OK
mmc0 is current device
Failed to mount ext2 filesystem...
** Unrecognized filesystem type **
starting USB...
USB0:   USB EHCI 1.00
USB1:   USB OHCI 1.0
scanning bus 0 for devices... 1 USB Device(s) found
scanning bus 1 for devices... 2 USB Device(s) found

USB device 0: unknown device
BOOTP broadcast 1
BOOTP broadcast 2
BOOTP broadcast 3
BOOTP broadcast 4

I think it is highly unlikely that the error is in the SD card as I am not able to mount the .img file on my laptop running the newest Ubuntu build. Also when examining the partitions of the SD card in gparted afterwards the partition that I made using the generated .img file has a File System that is "unknown".

[ThomasKaiser]

Out of curiousity: is there a specific reason to combine BRANCH=dev with BUILD_DESKTOP=yes?

[multivac61]

Well funny you should ask, but yes. In my current project I am using a TFT screen with a NanoPi NEO. Unfortunately that setup crashes pretty hard when I am running the default kernel. I feel that I might as well try the dev kernel to see if the fbtft driver has been updated. That is the sole reason why I needed to have a desktop, so that the image came included with a window manager (because I am lazy and don't want to have to install it myself).

This is not a major issue, I am currently compiling the dev branch without a desktop. Let's see how that goes..

[ThomasKaiser]

Please keep us updated. In case it doesn't work it would be great if you provide a bit more output (build log). BTW: to quickly check an image you could do on the build host:

apt-get install testdisk
testdisk /path/to/image

and then examine it. BTW: SPI with dev kernel might require DT overlays

[multivac61]

Compilation using the command ./compile.sh BRANCH=dev BOARD=nanopineo KERNEL_ONLY=no PROGRESS_DISPLAY=plain RELEASE=xenial BUILD_DESKTOP=no NO_APT_CACHER=yes finished. The .img file I get also seams to be corrupt, that is I am not able to mount it on my build host or boot using it from my SD card.

I am using docker:

root@2b0742ed1922:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.1 LTS
Release:    16.04
Codename:   xenial
root@2b0742ed1922:~# uname -a
Linux 2b0742ed1922 4.4.0-38-generic #57-Ubuntu SMP Tue Sep 6 15:42:33 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

so I am running software that as far as I can tell is supported. I will compile once again and save the build log to file.

Btw I have never used testdisk before, which commands would you expect to generate useful results in this scenario?

[ThomasKaiser]

testdisk works interactively and when you specify the path to the image in question nothing can go wrong (didn't test it but you might then also be able to call if from a non root account). You can tell testdisk to log everything it does so regardless what you do we have a log later :)

[multivac61]

@ThomasKaiser Alright so I run testdisk and supply it with the images as arguments. Then I select Intel/PC partition as my partition table type, choose to Analyze and get the following view

Disk Armbian_5.21_Nanopineo_Ubuntu_xenial_4.7.6.img - 1488 MB / 1420 MiB - CHS 22720 4 32
Current partition structure:
     Partition                  Start        End    Size in sectors

No ext2, JFS, Reiser, cramfs or XFS marker
 1 P Linux                   16   0  1 22719   3 32    2906112
 1 P Linux                   16   0  1 22719   3 32    2906112
No partition is bootable

A "Quick Search" doesn't reveal anything and neither does "Deep Search".

[multivac61]

The compilation I mentioned just finished and the log files are here

[ThomasKaiser]

Yes, this looks pretty wrong :)

@neomanic did you ever encounter something like this from within a Docker container? To be honest: never used Docker before and not that keen on starting with it...

[zador-blood-stained]

@multivac61
Can you also provide build script console output? There may be some error messages that don't go into log files.

[multivac61]

Yes, I just added as much console output as I had access to, to the gist. If you need more output than that I can pipe it to another file. I do however think that it is logical that the console output is saved if the log flag is set to yes.

[zador-blood-stained]

[ o.k. ] Creating partitions [ root: ext4 ]
The file /dev/loop2p1 does not exist and no size was specified.
tune2fs: No such file or directory while trying to open /dev/loop2p1
Couldn't find valid filesystem superblock.
mount: special device /dev/loop2p1 does not exist

Yes, this is what I expected to see. For some reason this was reported as working before.

[zador-blood-stained]

@multivac61
Also are you running Docker in privileged mode?

[neomanic]

I encounter this problem all the time and am currently working around it by recreating the loopXpY nodes every time I restart the Docker container.

mknod /dev/loop0p1 b 259 0
mknod /dev/loop0p2 b 259 1
mknod /dev/loop0p3 b 259 2
mknod /dev/loop0p4 b 259 3
mknod /dev/loop1p1 b 259 0
mknod /dev/loop1p2 b 259 1
mknod /dev/loop1p3 b 259 2
mknod /dev/loop1p4 b 259 3

You will probably need to add /dev/loop2p1 etc, but I've never got as far as needing 3 loopback devices.

I'm sure there's a way to tell Docker/the kernel to do this, but I've been too lazy to figure it out.

[ThomasKaiser]

I wonder whether we shouldn't add an article 'Docker best practices' to lib.docs? Can't contribute myself since no Docker user but IMO it would save both us and our users some time when this stuff gets properly documented. Maybe a thread started in development forum to discuss the issue amongst different Docker/Vagrant users might be an idea?

[neomanic]

Great idea. I'd be happy to put together a tutorial/walkthrough and FAQ, but would require input from people using it on Linux since I'm on Docker/Mac myself.

If everyone else is happy, I can kick off the thread on the forum. Is a good name for the docs file Developer-Guide_Docker-build.md, in line with the other docs?

[zador-blood-stained]

I'm sure there's a way to tell Docker/the kernel to do this, but I've been too lazy to figure it out.

This just needs to be added to debootstrap-ng.sh where partitions are mounted.

[ThomasKaiser]

If everyone else is happy, I can kick off the thread on the forum

@neomanic please start with it over there. And yes, the name looks good to me. Stuff like below could also discussed there.

This just needs to be added to debootstrap-ng.sh where partitions are mounted.

...or as part of a more Docker specific script as suggested here to separate Docker stuff from 'usual' execution (really no opinion on that since Docker noobie)

[multivac61]

@zador-blood-stained Yes, the docker command I run is docker run -t -i --privileged <image id> /bin/bash.

@neomanic Now that you mention it I remember having to restart loopXpY nodes before I compiled .raw files the last time I was working with Docker. Also I am running docker on Linux and would be happy to help with testing and writing docs. Once set up, using Docker is nice and easy and it's adoption for new users should also be that way..

Restarting loopXpY prior to running compile.sh seems to have done the trick. Here is the build log file.

[ThomasKaiser]

I'm closing that now since the culprit has been found. But I really hope you guys start discussing Docker issues over at the forums and come up with Developer-Guide_Docker-build.md later.

Discussion whether mknod calls and so on should happen in a separate 'Docker wrapper' script that calls compile.sh or somewhere else in our sources can also be discussed there. And as a personal result of discussion and documentation improvements I hope to join the Docker party afterwards (being adventurous and building Armbian on OS X then :) )

[neomanic]

Igor has added me as docs contributor now. I am away this week on family holiday, so I haven't disappeared. Will do the things I've said I will as soon as I'm back.