Pmbootstrap: ERROR: postmarketos-splash-1-r4: Permission denied (binary package mirror out of sync)

[joshas]

Tried to build an existing device, chosen all default options for init command. Build stopped on step 2, while creating device rootfs. Excerpts from log:

(002332) [21:32:19] (rootfs_samsung-i9100) % apk --no-progress add -u postmarketos-base [...]
[...]
(61/191) Installing postmarketos-splash (1-r4)
ERROR: postmarketos-splash-1-r4: Permission denied
[...]
(80/191) Installing postmarketos-mkinitfs (0.5.4-r0)
ERROR: postmarketos-mkinitfs-0.5.4-r0: Permission denied
[...]
(85/191) Installing device-samsung-i9100 (1-r15)
ERROR: device-samsung-i9100-1-r15: Permission denied
[...]
(140/191) Installing postmarketos-base (3-r11)
ERROR: postmarketos-base-3-r11: Permission denied
[...]
(191/191) Installing postmarketos-ui-weston (4-r3)
ERROR: postmarketos-ui-weston-4-r3: Permission denied

Full build log here: https://pastebin.com/9Umzb5Ag

Environment: Ubuntu 16.04 LTS (with latest updates installed) guest in VirtualBox.

[ollieparanoid]

Thanks for trying out pmbootstrap!
Could it be, that you have your "work" folder (as asked for in pmbootstrap init) in a VirtualBox shared folder? That could lead to such errors and is not supported.

If that doesn't help, my next guess would be binfmt_misc problems (we use binfmt to be able to execute foreign arch binaries with Qemu, and your Ubuntu installation might have a conflicting handler set up, that doesn't work as expected). Try to run pmbootstrap shutdown, so it unregisters the binfmt_misc handler, and try again (it will properly register it this time).

[joshas]

No, I don't have shared folders on this virtual machine. Tried using custom work directory ~/pm-work/ - got same errors during build.

[ollieparanoid]

Thanks for trying it out. I've just edited my post, does the other solution help?

[joshas]

Just ran ./pmbootstrap shutdown , followed by ./pmbootstrap install - same error.

[ollieparanoid]

Okay, that is odd. One more idea to debug this (the error is coming from apk, Alpine package keeper, and this will show verbose output):

./pmbootstrap.py chroot
apk add -vvvv postmarketos-mkinitfs

What do you get when running that?

Another idea would be installing without the binary repository, because we have just merged that. Although osk-sdl gets installed fine from the binary repository, but just to rule it out:

./pmbootstrap.py zap # delete all chroots
./pmbootstrap.py --mirror-pmOS="" build postmarketos-mkinitfs
./pmbootstrap.py chroot
apk add -vvvv postmarketos-mkinitfs

This will build one of the failing packages from source and install it.

The output of both would be very helpful. I will try to help more with fixing this tomorrow, and you might have more luck with help chat in case you did not ask there yet.

[joshas]

First command completed without errors:
OK: 136 packages, 910 dirs, 14829 files, 476 MiB

Second set of commands also completed without errors:
OK: 123 packages, 832 dirs, 13423 files, 278 MiB

Tried to install other failing packages manually in chroot:
postmarketos-splash - success
device-samsung-i9100 - error: unsatisfiable constraints: so:libcrypto.so.41 (missing)

postmarketos-base and postmarketos-ui-weston complain about missing /etc/device info, which I suppose should've been installed from device-samsung-i9100 package?

Executing postmarketos-ui-weston-4-r3.post-install
var/cache/misc/postmarketos-ui-weston-4-r3.post-install: .: line 3: can't open '/etc/deviceinfo'
ERROR: postmarketos-ui-weston-4-r3.post-install: script exited with error 2

[joshas]

One more thing I've noticed, that errors actually start showing up in log much earlier, there's an error about busybox and other packages, not sure if that's related to my build issues:

Executing busybox-1.27.2-r3.post-install
ERROR: busybox-1.27.2-r3.post-install: script exited with error 127

[ollieparanoid]

The "exited with error 127" messags are normal, see:
https://wiki.postmarketos.org/wiki/Troubleshooting#ERROR:_busybox-1.27.0-r1.post-install:_script_exited_with_error_127

postmarketos-base and postmarketos-ui-weston complain about missing /etc/device info, which I suppose should've been installed from device-samsung-i9100 package?

Exactly.

The results from what you've tested show, that the error only appears in the armhf chroot. Could you try this and send us the full output? (-r means: use the rootfs_samsung-i9100 chroot):

./pmbootstrap.py -r chroot
apk add -vvvv postmarketos-mkinitfs

Could you post the output of that?

[joshas]

When should I run this command? After zapping failed install, or after running install command?
Now I get pmbootstrap: error: unrecognized arguments -r

[joshas]

After manually chroot'ing into chroot_rootfs_samsung-i9100 directory and a bit of fiddling with apk, I found that permission error actually comes from download server.
Tried wget http://postmarketos.brixit.nl/armhf/postmarketos-splash-1-r4.apk and got HTTP/1.1 403 Forbidden error. Same url works from main OS. Something strange about this particular chroot, because there's no issues on "other" chroot (one that is accessible through ./pmbootstrap.py chroot).

[ollieparanoid]

Oh sorry, I meant to type pmbootstrap chroot -r.
(There are multiple chroots in your work folder, and they all start with chroot_.)

It's really strange that you get a 403 - does this also happen with pmbootstrap chroot -r?

[joshas]

Just finished build with --mirror-pmOS="" option (had to do a fix for musl-* 404 error), everything went OK.
Here's what I got from pmbootstrap chroot -r after unsuccessful install command:

/ # apk add -vvvv postmarketos-mkinitfs
The following NEW packages will be installed:
  postmarketos-splash postmarketos-mkinitfs device-samsung-i9100
  postmarketos-base postmarketos-ui-weston
After this operation, 220 KiB of additional disk space will be used.
(1/5) Installing postmarketos-splash (1-r4)
ERROR: postmarketos-splash-1-r4: Permission denied
(2/5) Installing postmarketos-mkinitfs (0.5.4-r0)
ERROR: postmarketos-mkinitfs-0.5.4-r0: Permission denied
(3/5) Installing device-samsung-i9100 (1-r15)
ERROR: device-samsung-i9100-1-r15: Permission denied
(4/5) Installing postmarketos-base (3-r11)
ERROR: postmarketos-base-3-r11: Permission denied
(5/5) Installing postmarketos-ui-weston (4-r3)
ERROR: postmarketos-ui-weston-4-r3: Permission denied
5 errors; 202 packages, 937 dirs, 8739 files, 208 MiB

wget http://postmarketos.brixit.nl/armhf/APKINDEX.tar.gz works from chroot, file is downloaded. I noticed, that files that are failing are missing from http://postmarketos.brixit.nl/armhf/ directory index. Maybe permissions issue on server side?

Just for the sake of interest tried downloading same packages from x86_64 directory (in samsung chroot, using wget) - no errors, files downloaded successfully. And all files are listed in directory index there.

[ollieparanoid]

Thanks for all the feedback, looks like we can solve the mystery now 🎉

http://postmarketos.brixit.nl randomly redirects to one of the following:

• http://postmarketos1.brixit.nl
• http://postmarketos2.brixit.nl

The directory index of the 2nd server has postmarketos-splash for example, while the other one has not. I'll look into this, maybe I didn't upload the packages properly or the server is out of space or something.

Workaround

In the meantime, you can force the 2nd server with:
./pmbootstrap.py --mirror-pmOS="http://postmarketos2.brixit.nl"

[ollieparanoid]

Okay, here's the resolution:

• One server's config did not allow symlinks, but the other one did
• noarch packages get symlinked to foreign architectures, since I'm building on x86_64, that means armhf is foreign
• That means, the server refused to download armhf links broken for the noarch packages

Special thanks to @MartijnBraam for the quick reaction and fix (via chat)!