Pmbootstrap: linux-xiaomi-ido doesn't compile

[ollieparanoid]

It fails for linux-xiaomi-ido with:

...
  OBJCOPY arch/arm64/boot/Image
  GZIP    arch/arm64/boot/Image.gz
No valid dtbs found

@lawl: do you know how to fix this directly / could you make a PR that fixes it?

[lawl]

Can confirm it fails. Jumping one commit behind 19e06cf7f280c220bc6851a6af648a4ca0f134f2 and it compiles again, I haven't fully bisected it but I strongly suspect this is the commit to blame. Since the "No valid dtbs found" comes from dtbTool.

Have we compiled all other kernels that rely on dtbTool since that commit? Is this the only one that breaks?

[z3ntu]

I just tried compiling linux-fairphone-fp2 and the image still boots (= I can access the debug shell).

[ollieparanoid]

So much for drop-in replacement 😕

@drebrez compiled linux-huawei-y530 with it and compared the resulting dt.img file, and it was the same one as with the old dtbTool.

The following devices depend on dtbtool, I will check if they still compile:

• [x] linux-amazon-thor (OK, tested by @MoreRobustThanYou)
• [x] linux-fairphone-fp2 (OK, tested by @z3ntu)
• [x] linux-huawei-y530 (OK, tested by @drebrez)
• [x] linux-oneplus-bacon (OK, tested by @z3ntu and me)
• [x] linux-oneplus-onyx (OK, tested by @z3ntu and me)
• [x] linux-t2m-flame (NOK, tested by @z3ntu, see below)
• [x] linux-xiaomi-ido (NOK)

@lawl: thanks for investigating this and for the PR!
@z3ntu: thanks for confirming that the linux-fairphone-fp2 still works!

[z3ntu]

 /m/h/p/pmbootstrap   master $…  ./pmbootstrap.py build linux-oneplus-bacon --arch=armhf --force   1148ms  Sam 30 Dez 2017 13:55:02 CET
[13:55:05] (native) install dtbtool python2 xz
[sudo] password for luca: 
[13:55:07] (native) install
[13:55:07] (native) build armhf/linux-oneplus-bacon-3.4.113-r7.apk
[14:02:55] Done

 /m/h/p/pmbootstrap   master $…  ./pmbootstrap.py build linux-oneplus-onyx --arch=armhf --force      7.8m  Sam 30 Dez 2017 14:02:55 CET
[14:03:21] (native) install
[sudo] password for luca: 
[14:04:29] (native) build armhf/linux-oneplus-onyx-3.4.0-r0.apk
[14:12:54] Done



md5-c620638e1846bd119a597ababbc04f97



 /m/h/p/pmbootstrap   master $…  ./pmbootstrap.py build linux-t2m-flame --arch=armhf --force         9.6m  Sam 30 Dez 2017 14:12:55 CET
[14:18:25] (native) install
[sudo] password for luca: 
[14:18:33] (native) build armhf/linux-t2m-flame-3.4.0-r8.apk
[14:23:56] NOTE: The failed command's output is above the ^^^ line in the logfile: /home/luca/.local/var/pmbootstrap/log.txt
[14:23:56] ERROR: Command failed: (native) % cd /home/pmos/build && su pmos -c 'CARCH=armhf CROSS_COMPILE=armv6-alpine-linux-muslgnueabihf- CC=armv6-alpine-linux-muslgnueabihf-gcc abuild -d -f'
[14:23:56] Run 'pmbootstrap log' for details.
[14:23:56] See also: <https://postmarketos.org/troubleshooting>



md5-5036e3a05880bce39c5d3e6b6bc6c7fd



  Kernel: arch/arm/boot/zImage-dtb is ready
Traceback (most recent call last):
  File "/usr/bin/dtbTool", line 444, in <module>
    records += generate_records(f, pagesize)
  File "/usr/bin/dtbTool", line 359, in generate_records
    x = iter(msm_id)
TypeError: 'NoneType' object is not iterable
>>> ERROR: linux-t2m-flame: all failed

[ollieparanoid]

@z3ntu: thanks for checking the other kernels! As I'm writing this, the flame kernel built crashed on my end as well, so I can confirm the results.

From what I understand, @MoreRobustThanYou needs skales' dtbTool. But it's obvious that other devices break with it, so we can't ship it as default.

TODO:

• submit PR that ships both versions
• report the errors we found with skales' tool upstream so they can fix it.

I'll be working on the PR.

@z3ntu: if you have time, would you like to report the errors upstream? They probably need the input files for the flame and for the ido to reproduce the error.

[drebrez]

looking at the dtbtool code seems like the DTS if it has qcom,board-id it also need to have qcom,msm-id

[drebrez]

@ollieparanoid I just tried a simple fix for the dtbtool and tested the linux-t2m-flame, it builds correctly and the generated dt.img is exactly the same as the one generated with the old dtbtool

[ollieparanoid]

Oh, I did not see your comment before I opened the PR.

I would prefer if we kept both versions anyway, so we can fall back to the lineageos version when we hit another bug with the skales one, what do you think?

[drebrez]

@ollieparanoid I've just finished compiling the linux-xiaomi-ido and it doesn't even generate any dtb file, and looking at the kernel source code seems that there aren't for the CONFIG_ARCH_MSM8916
https://github.com/postmarketOS/pmbootstrap/blob/master/aports/device/linux-xiaomi-ido/config-xiaomi-ido.aarch64#L267
and https://github.com/LineageOS/android_kernel_xiaomi_msm8916/blob/8f4c7c59fb3a29344af202150030ae712f801d6b/arch/arm/boot/dts/Makefile

[drebrez]

Isn't it possible that the previous dtbTool version was just ignoring if there aren't any dtb file generated?

[lawl]

so the old tool reported "6 valid files found" or something and then generates a .dt file. if your question is that it might work completely without it, then i must admit i haven't tried that, i've only read on XDA to use dtbtool for this kernel. I can try it later if that's what you meant.

[ollieparanoid]

So @drebrez found out, that skales' dtbTool doesn't look in subfolders. If we change

dtbTool -s 2048 -p "scripts/dtc/" -o "arch/arm/boot/dt.img" "arch/arm/boot/"

to

dtbTool -s 2048 -p "scripts/dtc/" -o "arch/arm/boot/dt.img" "arch/arm/boot/dts"

...then it produces the file for the xiaomi-ido as with the LineageOS version. I have followed these steps and diffed the files (lineage vs. skales version) and they are identical.

I recommend these steps:

• [x] build all kernels depending on dtbtool with #1057
• [x] merge the PR
• [x] explain in the wiki how to use the version from LineageOS again (to debug possibly new issues with skales' version/to have a fallback) -> https://wiki.postmarketos.org/wiki/Dtbtool
• [x] report upstream that skales' dtbtool needs (to be compatible with the other dtbTools, what it claims to do):
  
  
  
  • the fix from drebrez
  
  
  • look into subfolders
  
  
• [x] close this issue

[drebrez]

@ollieparanoid no need to change the linux-xiaomi-ido because I just patched the skales dtbTool to look into subfolders also.

[z3ntu]

I've posted a message into the ##linux-msm channel on freenode and notified sboyd (the author of this dtbtool) of our problems / conversations.