Ironos: Cannot build with GCC 6.3.1 or higher

[agatti]

• I'm submitting a ...

  • [x] Bug report
  • [ ] Feature request
  • [ ] Translation

• Do you want to request a feature or report a bug?

Bug

• What is the current behavior?

The firmware, as per revision bf7f1f50625bb85f4505519a8b2455a69b86b25a, fails to build either via plain make or via ./build.sh when using GCC for Arm that's installed via Homebrew Cask on macOS.

• What is the expected behavior?

The firmware should build successfully.

• If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

Steps to reproduce:

1. Install the gcc-arm-embedded toolchain via Homebrew Cask on macOS. The GCC version it will install will be as such:

Using built-in specs.
COLLECT_GCC=arm-none-eabi-gcc
COLLECT_LTO_WRAPPER=/usr/local/Caskroom/gcc-arm-embedded/6-2017-q2-update/gcc-arm-none-eabi-6-2017-q2-update/bin/../lib/gcc/arm-none-eabi/6.3.1/lto-wrapper
Target: arm-none-eabi
Configured with: /tmp/jenkins-GCC-6-build_build-toolchain-mac-262_20170621_1498074986/src/gcc/configure --target=arm-none-eabi --prefix=/tmp/jenkins-GCC-6-build_build-toolchain-mac-262_20170621_1498074986/install-native --libexecdir=/tmp/jenkins-GCC-6-build_build-toolchain-mac-262_20170621_1498074986/install-native/lib --infodir=/tmp/jenkins-GCC-6-build_build-toolchain-mac-262_20170621_1498074986/install-native/share/doc/gcc-arm-none-eabi/info --mandir=/tmp/jenkins-GCC-6-build_build-toolchain-mac-262_20170621_1498074986/install-native/share/doc/gcc-arm-none-eabi/man --htmldir=/tmp/jenkins-GCC-6-build_build-toolchain-mac-262_20170621_1498074986/install-native/share/doc/gcc-arm-none-eabi/html --pdfdir=/tmp/jenkins-GCC-6-build_build-toolchain-mac-262_20170621_1498074986/install-native/share/doc/gcc-arm-none-eabi/pdf --enable-languages=c,c++ --enable-plugins --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-newlib --with-headers=yes --with-python-dir=share/gcc-arm-none-eabi --with-sysroot=/tmp/jenkins-GCC-6-build_build-toolchain-mac-262_20170621_1498074986/install-native/arm-none-eabi --build=x86_64-apple-darwin10 --host=x86_64-apple-darwin10 --with-gmp=/tmp/jenkins-GCC-6-build_build-toolchain-mac-262_20170621_1498074986/build-native/host-libs/usr --with-mpfr=/tmp/jenkins-GCC-6-build_build-toolchain-mac-262_20170621_1498074986/build-native/host-libs/usr --with-mpc=/tmp/jenkins-GCC-6-build_build-toolchain-mac-262_20170621_1498074986/build-native/host-libs/usr --with-isl=/tmp/jenkins-GCC-6-build_build-toolchain-mac-262_20170621_1498074986/build-native/host-libs/usr --with-libelf=/tmp/jenkins-GCC-6-build_build-toolchain-mac-262_20170621_1498074986/build-native/host-libs/usr --with-host-libstdcxx='-static-libgcc -Wl,-lstdc++ -lm' --with-pkgversion='GNU Tools for ARM Embedded Processors 6-2017-q2-update' --with-multilib-list=rmprofile
Thread model: single
gcc version 6.3.1 20170620 (release) [ARM/embedded-6-branch revision 249437] (GNU Tools for ARM Embedded Processors 6-2017-q2-update)

1. Run either make or ./build.sh in workspace/TS100/.

The output will be littered with errors like:

...

Linking TS100_EN.elf
/usr/local/Caskroom/gcc-arm-embedded/6-2017-q2-update/gcc-arm-none-eabi-6-2017-q2-update/bin/../lib/gcc/arm-none-eabi/6.3.1/../../../../arm-none-eabi/bin/ld: Hexfile/TS100_EN.elf section `.rodata' will not fit in region `ROM'
/usr/local/Caskroom/gcc-arm-embedded/6-2017-q2-update/gcc-arm-none-eabi-6-2017-q2-update/bin/../lib/gcc/arm-none-eabi/6.3.1/../../../../arm-none-eabi/bin/ld: region `ROM' overflowed by 2236 bytes
collect2: error: ld returned 1 exit status
make: *** [Hexfile/TS100_EN.elf] Error 1

...

Linking TS100_CS_CZ.elf
/usr/local/Caskroom/gcc-arm-embedded/6-2017-q2-update/gcc-arm-none-eabi-6-2017-q2-update/bin/../lib/gcc/arm-none-eabi/6.3.1/../../../../arm-none-eabi/bin/ld: Hexfile/TS100_CS_CZ.elf section `.rodata' will not fit in region `ROM'
/usr/local/Caskroom/gcc-arm-embedded/6-2017-q2-update/gcc-arm-none-eabi-6-2017-q2-update/bin/../lib/gcc/arm-none-eabi/6.3.1/../../../../arm-none-eabi/bin/ld: region `ROM' overflowed by 2424 bytes
collect2: error: ld returned 1 exit status
make: *** [Hexfile/TS100_CS_CZ.elf] Error 1

and so on, for each localisation region. With plain make, the error will be as such:

Linking TS100_EN.elf
/usr/local/Caskroom/gcc-arm-embedded/6-2017-q2-update/gcc-arm-none-eabi-6-2017-q2-update/bin/../lib/gcc/arm-none-eabi/6.3.1/../../../../arm-none-eabi/bin/ld: Hexfile/TS100_EN.elf section `.rodata' will not fit in region `ROM'
/usr/local/Caskroom/gcc-arm-embedded/6-2017-q2-update/gcc-arm-none-eabi-6-2017-q2-update/bin/../lib/gcc/arm-none-eabi/6.3.1/../../../../arm-none-eabi/bin/ld: region `ROM' overflowed by 2236 bytes
collect2: error: ld returned 1 exit status
make: *** [Hexfile/TS100_EN.elf] Error 1

[Ralim]

Hi,
Looks like that compiler version does not optimise the code to a sufficient degree for it to fit on the iron.

Not really a lot I can do as I don't own any macs :/

[agatti]

Thank you for your quick answer!

I spun up a VM and installed GCC 7 (I need 6.3.1 for another project) and unfortunately the issue is still there:

Linking TS100_EN.elf
/usr/local/Caskroom/gcc-arm-embedded/7-2017-q4-major/gcc-arm-none-eabi-7-2017-q4-major/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/bin/ld: Hexfile/TS100_EN.elf section `.rodata' will not fit in region `ROM'
/usr/local/Caskroom/gcc-arm-embedded/7-2017-q4-major/gcc-arm-none-eabi-7-2017-q4-major/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/bin/ld: region `ROM' overflowed by 1824 bytes
collect2: error: ld returned 1 exit status
make: *** [Hexfile/TS100_EN.elf] Error 1

Now the overflow amount is smaller but still there... For the record, the GCC version I tried is:

Using built-in specs.
COLLECT_GCC=arm-none-eabi-gcc
COLLECT_LTO_WRAPPER=/usr/local/Caskroom/gcc-arm-embedded/7-2017-q4-major/gcc-arm-none-eabi-7-2017-q4-major/bin/../lib/gcc/arm-none-eabi/7.2.1/lto-wrapper
Target: arm-none-eabi
Configured with: /Users/build/work/GCC-7-build/src/gcc/configure --target=arm-none-eabi --prefix=/Users/build/work/GCC-7-build/install-native --libexecdir=/Users/build/work/GCC-7-build/install-native/lib --infodir=/Users/build/work/GCC-7-build/install-native/share/doc/gcc-arm-none-eabi/info --mandir=/Users/build/work/GCC-7-build/install-native/share/doc/gcc-arm-none-eabi/man --htmldir=/Users/build/work/GCC-7-build/install-native/share/doc/gcc-arm-none-eabi/html --pdfdir=/Users/build/work/GCC-7-build/install-native/share/doc/gcc-arm-none-eabi/pdf --enable-languages=c,c++ --enable-plugins --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-newlib --with-headers=yes --with-python-dir=share/gcc-arm-none-eabi --with-sysroot=/Users/build/work/GCC-7-build/install-native/arm-none-eabi --build=x86_64-apple-darwin10 --host=x86_64-apple-darwin10 --with-gmp=/Users/build/work/GCC-7-build/build-native/host-libs/usr --with-mpfr=/Users/build/work/GCC-7-build/build-native/host-libs/usr --with-mpc=/Users/build/work/GCC-7-build/build-native/host-libs/usr --with-isl=/Users/build/work/GCC-7-build/build-native/host-libs/usr --with-libelf=/Users/build/work/GCC-7-build/build-native/host-libs/usr --with-host-libstdcxx='-static-libgcc -Wl,-lstdc++ -lm' --with-pkgversion='GNU Tools for Arm Embedded Processors 7-2017-q4-major' --with-multilib-list=rmprofile
Thread model: single
gcc version 7.2.1 20170904 (release) [ARM/embedded-7-branch revision 255204] (GNU Tools for Arm Embedded Processors 7-2017-q4-major)

I assume you may be able to try with the toolchains from https://developer.arm.com/open-source/gnu-toolchain/gnu-rm - which is where Homebrew fetches the macOS versions, and said toolchains are available also for Windows and Linux.

[Ralim]

Hi,
Yeah this is most likely changes in the new versions, I'm on 4.9.3 on all of my systems at the moment.

Annoying if they have gotten worse, hopefully it just needs more flags to optimise better.

[agatti]

When using -Os on GCC 7, everything fits except for localised versions that use Cyrillic (UA and BG). Luckily it's only ~300 bytes so I have some chances to shave code down a bit - however I wonder whether all of this is moot if the firmware cannot be effectively compiled on anything newer than 4.9.3.

[Ralim]

This is a larger problem since if we wanted to start to combine languages (#222) there is no chance they will fit with the newer compilers.

Do you know if there are extra flags that could be passed to improve the newer compilers ?

[jonnieZG]

Guys, I am working on in-memory compression of fonts and texts, so hopefully we should be able to squeeze in 6-8 languages per build. We should just have about 6-7 kB of free working RAM, because once loaded from FLASH, the fonts should be kept decompressed in the RAM. @Ralim, do you know how much RAM we can spare for that purpose? (I am talking about RAM, not FLASH)

[agatti]

@Ralim I've had a quick read of the available options but it seems like you've gone through all the useful ones already except for -Os - which is still a problem for Bulgarian and Ukrainian localisations. I'll take a more in-depth look later on today and see if there's anything worth exploiting.

[agatti]

@jonnieZG with readelf on a binary I built with GCC 7 using -Os I got this:

  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
[...]
  [ 9] .data             PROGBITS        20000000 010000 0006d4 00  WA  0   0  8
  [10] .bss              NOBITS          200006d8 0106d8 002e54 00  WA  0   0  8
  [11] ._user_heap_stack NOBITS          2000352c 01352c 000604 00  WA  0   0  1
[...]

Which summed up makes it up to around 14k. The linkerscript claims that there are 20k total on the board so you might be able to squeeze that in, but at the expense of more features in the future.

[jonnieZG]

Well, once we run out of memory, we could do some of the following:

• Shortening description texts to bare minimum. Current descriptions in some languages are overly verbose!
• Optimizing the code. I was raised in times of 8-bit computers, where entire universes were getting squeezed into 48 kB, so I am pretty good with optimizations. I could take a look deeper into the code to see how to optimize it, if Ralim is OK with that.
• Lowering number of languages per build.

[agatti]

For what it's worth, my personal opinion would be to shorten the strings first and then start optimising things for size (I already found a couple of spots that might be modified for that), otherwise you're painting yourself into a corner when it comes to future features.

[agatti]

Something like this (totally untested): https://gist.github.com/agatti/1f6682002c8f18ff65e66ffc5ba7024f should shave off 72 bytes already, for instance.

[agatti]

And https://gist.github.com/agatti/aa1106efc81a0a7c590f8b10227c3ea3 shaves off 52 bytes more...

[agatti]

Anyway, I pushed (untested on real hw) https://github.com/agatti/ts100/commit/9206b7b04a43af04601680688f68a636bc307b76 https://github.com/agatti/ts100/commit/223151b075d216f9b3b7d7696d03dfb6449e37e1 https://github.com/agatti/ts100/commit/a1164315449e970a98c529b7191eff6f666f2fb6 and https://github.com/agatti/ts100/commit/a1164315449e970a98c529b7191eff6f666f2fb6

to save around 140 bytes - right now my main target is to be able to fit all localisation with no other changes in features, and I'm more or less halfway there.

[agatti]

Ok, the tree at https://github.com/agatti/ts100 does compile successfully for all languages with Arm's own GCC 7-based toolchain. Changes are from https://github.com/agatti/ts100/commit/9206b7b04a43af04601680688f68a636bc307b76 and onwards, but it is: 1) totally untested and 2) filled with dirty tricks and whatnot to let it compile. I can make a PR if needed, but the lack of firmware space is a problem that may need to be addressed in a better way than what I've done, I guess...

[JohnEdwa]

@jonnieZG
I quickly checked the 'SettingsDescriptions' translations and there are only two that are shorter than the original: CS_CZ and BR. Almost all others are between 5-15% longer while the longest are IT at +27%, and HR at +20%. Taking into account that written English is one of the denser languages compared to others, this is a pretty good result - I don't think there is much you could shave to still convey the neccessary information. Well, maybe IT could be trimmed a little.

[jonnieZG]

@JohnEdwa, I am the one responsible for the verbose Croatian translation, but I could easily strip it down by removing obvious descriptions like "Advanced Options" >> "Setting advanced options" ;)

[Ralim]

I'm actually thinking of removing the descriptions for the main menu options? I don't think we really need them...

[Mrkvozrout]

Please take this as my opinion that triggers discussion, don't take it bad or personally.

I just need to ask this question (elephant in room?):

Is it still a good idea to strive for multilingual FW when there is not enough memory to have all in 1 (or at least whole language familly in 1) comfortably?

It would be a nice feature, BUT:

• it is not that useful (at least in my opinion) - feature is used only one time an the rest of device life it just occupies most of memory without real usage
• it limits the space for translators
• it limits space for new features (e.g. features with menu options, animations...) - features that add more value to every time usage
• user must go to settings and change the language to desired one - not comfortable
• user still needs to flash this FW so why not to just flash the correct one (with target language) - saves tons of space, no need to dig in the menu to set correct language before first use

222 Multilingual feature would be nice with enough space, but not with all these downsides. It is a nice excercise to optimize the code and memory usage but when it goes that far so description lengths are limited or removed (effectivelly meaning new features with menu options are forbidden) - I don't like the idea any more.

[JohnEdwa]

@Ralim
For us the descriptions are useless as we know what they do, but the way the UI limits the length means most of the feature names don't make any sense on their own for a new user. What does "Auto Start T/F/S" mean? What is "Boost Mode" and how do I use it? Which is less sensitive, "Motion Sensitivity" 1 or 9? And if you add something new, you have no way of explaining how it works in the iron itself.

I mean, if someone wants to start maintaining a neat manual explaining everything in all the translated languages then it would be fine, but for now, I feel that we should keep the descriptions in.

@Mrkvozrout
I agree with you, having multilingual FW would very nice, but it's clear there simply isn't enough space to properly implement it.
There are some combinations I can think of that could make sense though, like making all the translated builds have English as an option as well and combining very similar translations together like Swedish/Danish/Norwegian and Russian/Bulgarian/Ukranian as the Huffman encoding should make the sizes rather small, but most of the space should be reserved for new stuff just in case.

[Ralim]

Clearing space

The only descriptions I think could be dropped are those on the submenu's (i.e. "Soldering Settings" or "Advanced Settings"). I think all the rest should be kept for new users.

I dont think we are really going to see many new features being pushed to the firmware. And really, the entire 2.x version has mostly been fixing features and tweaks, rather than entirely new code being written and merged in. I'm willing to lose out on flash space for now to be able to provide a better user experience at this point, and should size become too limiting the language choices could be reduced.

I currently have about 6k of free ram, but there is also around 1.5k unused in the FreeRTos that can be freed up as needed if we get to a pinch :smile:

But Ill leave that over to #222

Compiler

@agatti I'm curious to test your changes and would appreciate a pull request.

To further the discussion I have been looking into the random lockups that users are reporting.
I can replicate these issues when built using the Makefile & gcc 4.9.3.
I can't replicate any of these issues when using the Atollic toolchain (based on 6.3.1).

I would like to move onto a newer compiler either way, so would be happy to take your modifications and test them and verify them on both sets of hardware.

I would like to hear any ideas you have on this too :)
I have been moving most of my other projects to Atollic just because its now free and performs significantly better than System Workbench for me. Will probably change this projects project folder to Atollic at some point eventually.

Here is a current tester as built by Atollic that hasnt (yet) crashed for me.
TS100A.hex.zip

Edit

Of course, as soon as I post this that firmware does eventually lock up. Just took hours longer than the other. _Grumble..._ I'll keep trying to find it...

Edit 2

Newer version :
TS100A.hex.zip

[agatti]

@Ralim I'll try to split the changes up into something more manageable and send those PRs your way.

Incidentally, I can give a shot at shortening the Italian language strings as I happen to be a native Italian speaker, if that is still needed, that is.

[agatti]

@Ralim I wonder whether switching the C++ parts to pure C can actually help.

Right now the accelerometer drivers are in C++ but do not make use of any benefit C++ can give in this situation - namely, class inheritance and whatnot. Those drivers do not derive from a common base class and no new allocations are possible, thus the final code must carry the RAM baggage for all accelerometer drivers even if there's none on board.

Plus, I also wonder whether the extra C++ support code/data that the compiler introduces (f.ex., vtables) are indeed bloating the final binary for little or no real benefit.

[Ralim]

@agatti Thanks for looking into this :)

It could help, but I'm a big fan of keeping items as separate objects so that they can be used in other projects (Since both accelerometer libraries are from previous projects of mine). It does help keep them portable enough to be easily re-used. (Same reason as to why I went for the HAL rather than other library options).

Off memory the newer arm compilers are fairly good at removing the unused C++ features, though I cant find any good reference for that, so I may be wrong here.

I have been meaning to see if moving the two accelerometers to be derived from a base class and then only instantiate the relevant one would help much but haven’t had time to test as of current.

I was looking at the sizing breakdown and the I2C drivers in the STM HAL are a nice big chunk that I'll try and attack, mostly because the STM32 I2C peripheral isnt great.

[agatti]

@Ralim Ever thought of using https://github.com/libopencm3/libopencm3 rather than STM HAL? I haven't used it that much often in the past, but as far as I could see it should produce smaller binaries after all. Granted, this is akin to the nuclear option, but if space gets that tight, maybe switching HALs could be a possibility...

Regarding instantiating just one driver at runtime, if you cannot perform allocations I'm not sure you can pull that off in C++. I tried earlier on, but bringing std::new in made the binary too large for any localisation, if I remember correctly. The equivalent code in C would just take enough RAM to fit a pointer to a driver function table and to the I2C handler, after all.

[Ralim]

@agatti I did indeed consider libopencm3 when I started the 2.x branch of the code (which moved off the old stm stdperiph). However, as its api wasn’t stable didn’t want to feel forced into updating for it as well :smile:. Its certainly an option, but the HAL also provided a nicer experience for other developers. I will concider this in the future should space get that tight, but there are easier options first (such as squashing settings and bootup logo onto one page) that would be significantly less work.

Thanks for testing the idea, I hadn't yet tried bringing in new and seeing the results. Might leave that alone for now then :)

[agatti]

@Ralim regarding the switch to C, I went on and came up with a (very) untested proof of concept at https://github.com/agatti/ts100/tree/cminusminus - which doesn't even try to optimise OLED or GUI code and yet is around 3 kilobytes smaller when compared with a normal C++ build. I suspect that with a bit more work things can be shrunk down by another half a kilobyte or so, possibly more if libopencm3 is used :)

Except for a few ugly bits (the #defines in Translation.h, for example) to avoid linkage and compilation errors, it seems to be somewhat manageable. I spotted a few more possibilities for size reduction but that'd mean generate the font and translation data upon building - as in, strip away unused glyphs in the current locale and then re-index the characters inside the string to reflect the new compacted font, but this can be done even with the current codebase if needed.

[Ralim]

The font optimization you are talking about is already in progress in #222. Which will also address the translation hacky-ness.

Ditching the C++ is certainly possible, it isnt really required for something this low in complexity, its just nicer to have.

I'll be testing your optimizations of the init code for the accelerometers shortly too. Seems to work ok so far.

[Eldenroot]

I would like to see only one language instead of multilanguage support. Would be better to add animations amd other improvements instead to have 8 languages packed together.

[Ralim]

@Eldenroot Even if the only outcome of the multiple language support is a few firmwares that have multiple languages, it will still result in more room in the firmware due to the compression. Which at this point if you want to have animations in the firmware the multiple language work will come into play either way.

If @jonnieZG implements some of the ideas of allowing customized builds then you will be able to pick between animations or multiple languages yourself etc.

[agatti]

@Ralim I believe that #305 fixed this issue, although there are still some potential savings in main.cpp (in showBootLogoIfavailable(), namely pre-swapping the logo data before compilation and compare the signature as an appropriately crafted uint32_t rather than as four separate bytes).

Feel free to close this if that's ok with you.

[Ralim]

Going to close this as we have enough room for now, I'm sure that someone will come along and criticize my coding for future things anyway to fight for more flash space. 😁