Gentoolto: Personal Shared Info About Follow InBetweenNames Profile On The Wiki.

[javashin]

[javashin]

1 . Disable Pie + SSP https://github.com/InBetweenNames/gentooLTO/issues/261 done When Complete emerge -e world DONE.......
2 . Use GentooLTO default profile -O3 Graphite LTO SEMIINTERPOS. Already Done DONE
3 . Add IPA And No PLT Complete When emerge -e world DONE
-fno-plt break xorg-server and drivers , ffmpeg too pcsx2
4 . Add -fdevirtualize-at-ltrans to The Mix DONE , DEFAULT NOW .
5 . Using ld.gold as Default LD Done DONE
6 . Use Prelink TODO https://wiki.gentoo.org/wiki/Prelink DONE
[ebuild N *] sys-devel/prelink-99999999::gentoo patched DONE

Not Added And Never Will dialect=gnu2 dont know what its do and if really helps on performance and like shane says dont work with prelink . no-common ????

Extras By Added By ME ::

FASTMATH="-fno-math-errno -fno-trapping-math"

-fno-stack-protector

-mfpmath=both -funroll-loops -falign-functions=32

LDFLAGS="-Wl,-O2 -Wl,--as-needed,-z,now -Wl,--hash-style=gnu"

Added -z,now Because Arch Uses It For Speed When Using No PLT By Default

EDIT : system is fast and stable no hot cpu , and startup times are ok , system boot fast than before
this new build is going to stay ..... after emerge -e world i had no sound fixed with https://github.com/InBetweenNames/gentooLTO/issues/327

system is fast without using prelink , feeling scary because i borked archlinux before using prelink , is very outdated software but i will prelink everything and emerge preload too ..... prelink+preload the perfect match

I think IPA can Be Enabled by default on GentoolTO Default Profile [DONE DEFAULT NOW by Shane]
i had compiled like 3 times world since gcc 9.1 with other flags and IPA never gave me problems even glibc with ipa is fine

EDIT 22/05/2019 18:43 AST.
This New System ROCKS!

[javashin]

What Does And Is Good For Performance ?
"-ftree-vectorize"

[nivedita76]

-ftree-vectorize is turned on by default at -O3.

-ftree-vectorize

Perform vectorization on trees. This flag enables -ftree-loop-vectorize and -ftree-slp-vectorize if not explicitly specified.

-ftree-loop-vectorize

Perform loop vectorization on trees. This flag is enabled by default at -O3 and by -ftree-vectorize, -fprofile-use, and -fauto-profile.

-ftree-slp-vectorize

Perform basic block vectorization on trees. This flag is enabled by default at -O3 and by -ftree-vectorize, -fprofile-use, and -fauto-profile.

[javashin]

Thanks @nivedita76

[javashin]

lazy binding ? is this the same as lazy loading ? the lazy loading that is going to be default on chrome 75?

[nivedita76]

@javashin no. It's the stuff related to no-plt -- i.e. with that you don't get lazy binding, possibly increasing startup times.

[javashin]

I See Thanks For All Your Help

[javashin]

@InBetweenNames What Do You Think About This Im Following Your Profile Good Enough ?

[javashin]

prelink always ends on prelink: arch-x86_64.c:421: x86_64_arch_prelink: Assertion `i < dso->ehdr.e_shnum' failed.

[javashin]

@InBetweenNames how do you make prelink to work ? this is all i get
prelink: arch-x86_64.c:421: x86_64_arch_prelink: Assertion `i < dso->ehdr.e_shnum' failed

[InBetweenNames]

Add exceptions for the .so files that break to your /etc/prelink.conf. I only needed to add a couple.

[InBetweenNames]

@javashin

I just wrote an experimental patch to get around the assertion. Not pushing this out for many reasons. I've reached out to the prelink-cross team to see what the right way to go is, but I'm not sure when I'll hear back from them.

~~~
diff --git a/src/arch-x86_64.c b/src/arch-x86_64.c
index 2f6c551..f0e9ce1 100644
--- a/src/arch-x86_64.c
+++ b/src/arch-x86_64.c
@@ -418,9 +418,18 @@ x86_64_arch_prelink (struct prelink_info *info)
".plt"))
break;

• assert (i < dso->ehdr.e_shnum);
• //GentooLTO workaround: -fno-plt means that executables will not contain a .plt section
• //So, the previous logic here won't work. Leave got[1] untouched and bail out.
• if (i < dso->ehdr.e_shnum)
• {
  data = dso->shdr[i].sh_addr + 0x16;
  write_le64 (dso, dso->info[DT_PLTGOT] + 8, data);
• }
• else
• {
• //error (0, 0, "%s: .plt entry does not exist", dso->filename);
• //return 1;

• }
  }

  return 0;
  ~~~

The reason for the assertion failure is that the .plt section itself does not exist in -fno-plt executables. Now, prelink's master branch ignores all PIE executables. The staging branch notably does not. Now that I have properly disabled PIE, I have noticed this assertion firing much more often, and in fact it seems to happen on any non-PIE executable. If I had to guess, shared objects still include a .plt even if it isn't used.

Now, I'm not sure if it should be a return 1 or not. Is the rest of the prelink process unaffected by the fact that the PLT is missing in the executable, and this can be safely ignored? Pinging @nivedita76 @ionenwks @pchome for advice.

[javashin]

@InBetweenNames ok i will test the patch too

[InBetweenNames]

Note: I can't find many references to .plt + 0x16. Only one in glibc, where it is used to "un-prelink" a shared object when it must be relocated for some reason. However, for position dependent executables, there should be no need for any relocation of the executable code itself, and therefore that condition shouldn't ever be hit -- am I mistaken?

[pchome]

For https://sourceware.org/ml/binutils/2016-05/msg00322.html example I still have .plt.
objdump output just different for gold/bfd, and for stripped/unstripped file.

bfd

Disassembly of section .plt:

0000000000001020 <.plt>:
    1020:       ff 35 e2 2f 00 00       pushq  0x2fe2(%rip)        # 4008 <_GLOBAL_OFFSET_TABLE_+0x8>
    1026:       ff 25 e4 2f 00 00       jmpq   *0x2fe4(%rip)        # 4010 <_GLOBAL_OFFSET_TABLE_+0x10>
    102c:       0f 1f 40 00             nopl   0x0(%rax)

0000000000001030 <__stack_chk_fail@plt>:
    1030:       ff 25 e2 2f 00 00       jmpq   *0x2fe2(%rip)        # 4018 <__stack_chk_fail@GLIBC_2.4>
    1036:       68 00 00 00 00          pushq  $0x0
    103b:       e9 e0 ff ff ff          jmpq   1020 <.plt>

Disassembly of section .plt.got:

0000000000001040 <__cxa_finalize@plt>:
    1040:       ff 25 b2 2f 00 00       jmpq   *0x2fb2(%rip)        # 3ff8 <__cxa_finalize@GLIBC_2.2.5>
    1046:       66 90                   xchg   %ax,%ax

bfd, stripped

Disassembly of section .plt:

0000000000001020 <__stack_chk_fail@plt-0x10>:
    1020:       ff 35 e2 2f 00 00       pushq  0x2fe2(%rip)        # 4008 <__cxa_finalize@plt+0x2fc8>
    1026:       ff 25 e4 2f 00 00       jmpq   *0x2fe4(%rip)        # 4010 <__cxa_finalize@plt+0x2fd0>
    102c:       0f 1f 40 00             nopl   0x0(%rax)

0000000000001030 <__stack_chk_fail@plt>:
    1030:       ff 25 e2 2f 00 00       jmpq   *0x2fe2(%rip)        # 4018 <__cxa_finalize@plt+0x2fd8>
    1036:       68 00 00 00 00          pushq  $0x0
    103b:       e9 e0 ff ff ff          jmpq   1020 <__stack_chk_fail@plt-0x10>

Disassembly of section .plt.got:

0000000000001040 <__cxa_finalize@plt>:
    1040:       ff 25 b2 2f 00 00       jmpq   *0x2fb2(%rip)        # 3ff8 <__cxa_finalize@plt+0x2fb8>
    1046:       66 90                   xchg   %ax,%ax

gold

Disassembly of section .plt:

0000000000000520 <__cxa_finalize@plt-0x10>:
 520:   ff 35 ca 1a 00 00       pushq  0x1aca(%rip)        # 1ff0 <_GLOBAL_OFFSET_TABLE_+0x8>
 526:   ff 25 cc 1a 00 00       jmpq   *0x1acc(%rip)        # 1ff8 <_GLOBAL_OFFSET_TABLE_+0x10>
 52c:   90                      nop
 52d:   90                      nop
 52e:   90                      nop
 52f:   90                      nop

0000000000000530 <__cxa_finalize@plt>:
 530:   ff 25 ca 1a 00 00       jmpq   *0x1aca(%rip)        # 2000 <__cxa_finalize@GLIBC_2.2.5>
 536:   68 00 00 00 00          pushq  $0x0
 53b:   e9 e0 ff ff ff          jmpq   520 <_init+0x18>

0000000000000540 <__stack_chk_fail@plt>:
 540:   ff 25 c2 1a 00 00       jmpq   *0x1ac2(%rip)        # 2008 <__stack_chk_fail@GLIBC_2.4>
 546:   68 01 00 00 00          pushq  $0x1
 54b:   e9 d0 ff ff ff          jmpq   520 <_init+0x18>

gold, stripped

Disassembly of section .plt:

0000000000000520 <__cxa_finalize@plt-0x10>:
 520:   ff 35 ca 1a 00 00       pushq  0x1aca(%rip)        # 1ff0 <__stack_chk_fail@plt+0x1ab0>
 526:   ff 25 cc 1a 00 00       jmpq   *0x1acc(%rip)        # 1ff8 <__stack_chk_fail@plt+0x1ab8>
 52c:   90                      nop
 52d:   90                      nop
 52e:   90                      nop
 52f:   90                      nop

0000000000000530 <__cxa_finalize@plt>:
 530:   ff 25 ca 1a 00 00       jmpq   *0x1aca(%rip)        # 2000 <__stack_chk_fail@plt+0x1ac0>
 536:   68 00 00 00 00          pushq  $0x0
 53b:   e9 e0 ff ff ff          jmpq   520 <__cxa_finalize@plt-0x10>

0000000000000540 <__stack_chk_fail@plt>:
 540:   ff 25 c2 1a 00 00       jmpq   *0x1ac2(%rip)        # 2008 <__stack_chk_fail@plt+0x1ac8>
 546:   68 01 00 00 00          pushq  $0x1
 54b:   e9 d0 ff ff ff          jmpq   520 <__cxa_finalize@plt-0x10>

Profile 17.1, default ssp and pie.

[InBetweenNames]

I think .plt is completely omitted if you have PIE disabled and you run -fno-plt.

[nivedita76]

The plt is also used for calling shared library functions. It will only be omitted if you have no-plt as well. Even then I’m not sure you won’t have plt calls from the startup code.

[nivedita76]

I wonder if prelink doesn’t work with no-plt. It might be hacking the plt to achieve the prelink goals since normally any shared library call will go through that.

[InBetweenNames]

I wonder as well. It seems that's the only part of prelink that doesn't play nice with .plt missing. The rest of it seems to work fine. I had assumed prelink would pre-fill the GOT entries and set the base address of the shared libraries and wouldn't necessarily need the PLT. That assert sure has me wondering.

[pchome]

Also, I trying to remember why I disabled prelinking during migration to the 17.1 profile.
Was it due to:

• just paused
• required additional configuration
• don't work with pie/ssp
• performance boost is negligible
• KDE also using some optimizations to load faster

? Still not clear for me.

There is warning in Gentoo wiki about hardened configurations, and Hardened part somewhat explains and says something like "you don't need this":

There are some technologies on the way which reduce the need for prelink. These include:

    Symbol visibility support, which when used properly, reduces dramatically the number of symbols to resolve and hence the amount of time taken to resolve them
    Hash tables, which will be generated by the linker and included as a extra section in the ELF file, which make looking up symbols to resolve them nearly free.
    Direct binding, which simplifies the search that the loader has to perform by incorporating information in each library detailing exactly where the symbol to be resolved is located.

So, do Prelink still a thing for default desktop 17.1 profile?

[InBetweenNames]

It looks like direct binding is dead: https://bugs.gentoo.org/show_bug.cgi?id=114008

I'm curious about the hash tables. We already have --hash-style=gnu being passed to the linker, so we're definitely using DT_GNU_HASH. According to https://utcc.utoronto.ca/~cks/space/blog/linux/PrelinkingIntro prelink and this method are complementary. I have noticed my system loads KDE faster with prelinking enabled. The rest of the system feels pretty much the same with prelinking however.

[javashin]

Nice Patch prelink is happy now no assertions prelink finished just fine let me reboot

[javashin]

prelink caused some problems with polkit.service of systemd and others with mate-settings-daemon but systemd did boot faster , i know what i need to do i need to blacklist all the files prelinks shows with :
"Prelink" not the ones that show "Prelinking"

[InBetweenNames]

Run prelink with -v to see potential errors. Some packages are known to cause problems, such as X. I'm on OpenRC so I can't test out any systemd related stuff.

[javashin]

here my list https://gist.github.com/javashin/fc629245cea14235d45b8e944062c584

and this are the libs that prelink dont like https://gist.github.com/javashin/319844ed8141c3928e6295f941ed1975

[javashin]

Fixed ! some apps start faster and systemd is very happy booting faster also im using preload which makes google chrome start faster @InBetweenNames thanks for the patch

[javashin]

zfs-kmod dont like my flags https://gist.github.com/javashin/db83b9c2dab62b85edf3d6b264c27215

[javashin]

i like this dmesg message i think is related to prelink or not ?

[ 2296.026633] 15701 (ostree-system-g): Uhuuh, elf segment at 0000000000401000 requested but the memory is mapped already

[javashin]

flatpak apps start faster

[nivedita76]

@InBetweenNames hey can you remove the default-gold requirement in ltoize? You can add it as a recommendation but there's no need to compile binutils with that flag even to switch to gold.

You should depend on binutils[gold] I think.

[javashin]

omg kernel compiled super faster faster than ever before
time make -j3 all
real 15m20.477s

[InBetweenNames]

Yeah I can change it to just gold and not default-gold. Won't be able to do it until later tonight though. If you make a PR I can merge it through my phone.

[InBetweenNames]

Should be fixed now. @javashin , how stable are things looking for you? So far no issues on my system, but we'll see how it goes after my world rebuild. I expect that message in dmesg about the relocation means that a prelinked library had an overlapping address region with another library somehow. It just means that a relocation had to take place. This is probably due to an exception in /etc/prelink.conf.

[javashin]

@InBetweenNames very stable everything works fine so far bin apps works fine like firefox-bin thunderbird-bin spotify google-chrome virtualbox vmware , also im using a custom patched kernel based on pf-kernel https://gitlab.com/post-factum/pf-kernel/wikis/README everything is fast and smooth using zfs filesystem systemd 17.1 profile with mate-desktop nothing broken , kernel compiled faster than before and im using only gcc 9.1 , fedora already have gcc 9.1.1 on repos , i only need gcc 8.3.0 for compile pcsx2 emulator. almost all my pkgs are compiled with LTO IPA NOPLT FASTMATH GRAPHITE SEMIINTERPOS -fdevirtualize-at-ltrans and my extras , Packages installed: 1329
so lets say Very Stable . Im Going To Make A Liveusb of this system and keep it around . As Backup and maybe add other flags to this current install and emerge -e world again maybe NO=COMMON :)

[InBetweenNames]

Good to hear. I'm also running the patch, and I haven't heard back from the prelink-cross team yet. In the middle of a world rebuild to really get rid of PIE this time, except for on the few packages that need it. Regarding GCC 9.1.1, it looks like there is no official release for that yet, but I will include one in the overlay as soon as it's out.

[Kokokokoka]

I hope you'll mention this patch in wiki too

[InBetweenNames]

I certainly will once I'm 100% sure it won't do something awful to your systems!

[InBetweenNames]

Did a bit more source browsing today. It seems what my patch does is similar to how some other architectures handle it. From arch_i386.c:

~
if (i == dso->ehdr.e_shnum)
return 0;
data = dso->shdr[i].sh_addr + 0x16;
write_le32 (dso, dso->info[DT_PLTGOT] + 4, data);
~

arch_arm.c:

~
if (i == dso->ehdr.e_shnum)
return 0;
data = dso->shdr[i].sh_addr;
write_ne32 (dso, dso->info[DT_PLTGOT] + 4, data);
~

arch_sh.c:

~
if (i == dso->ehdr.e_shnum)
return 0;
data = dso->shdr[i].sh_addr + 36;
write_ne32 (dso, dso->info[DT_PLTGOT] + 4, data);
~

arch-mips.c:

~
if (i == dso->ehdr.e_shnum)
return 0;
data = dso->shdr[i].sh_addr;
write_ne32 (dso, dso->info_DT_MIPS_PLTGOT + 4, data);
~

arch-s390.c:

~
if (i == dso->ehdr.e_shnum)
return 0;
data = dso->shdr[i].sh_addr + 0x2c;
write_be32 (dso, dso->info[DT_PLTGOT] + 4, data);
~

arch-s390x.c:

~
if (i == dso->ehdr.e_shnum)
return 0;
data = dso->shdr[i].sh_addr + 0x2e;
write_be64 (dso, dso->info[DT_PLTGOT] + 8, data);
~

arch-ia64.c:

~
if (plt == -1)
return 0;
~

notably it has checks for more than one .plt or .got, but they are not asserts and just return 1 as an error code, skipping over the binary.

arch-cris.c does have the assert.

This does give me a bit more confidence about the patch. I'd still like to hear from the upstream developers first though.

[javashin]

hello i have to share this , startup times i shared before on a closed issue in https://github.com/InBetweenNames/gentooLTO/issues/304 everything was wrong because i was using ZFS , Zfs Is Really SLow , compared to btrfs i changed my FS and now my system boots in 40 seconds compared to 1 minute 40 seconds in ZFS and , the apps start a lot faster almost instantly , zfs on a laptop with a hdd is crap .

[InBetweenNames]

Thanks for the followup @javashin ! Very glad to hear you've resolved the issue. By the way, I've started hearing back from upstream about prelink-cross. It seems my patch is on the right track so far.

[InBetweenNames]

Testing out the cross_prelink_staging branch. It seems it does not fully support reading /etc/ld.so.conf. In particular, it can't interpret include lines. Depending how my interactions go with the upstream Yocto devs, I may write a patch to add that functionality in.

[javashin]

Nice :)

[InBetweenNames]

More investigating: It looks like the branches master and master_staging use your system runtime linker to do the prelinking, and cross_prelink and cross_prelink_staging use prelink-rtld to do it, which simulates your system's runtime linker. It's intended for cross-compiled environments. It looks like master_staging is kept in sync with the cross_prelink_staging branch.

[InBetweenNames]

I just got the all-clear from upstream for the patch. I'll be submitting it there and then updating the wiki here.

[javashin]

So the patch is going to be in upstream ? i dont need to add the patch to /etc/portage/patches ? if there a updated patch paste here ::) nice work @InBetweenNames

[InBetweenNames]

It's been submitted but it hasn't been accepted yet. It's pretty much the
same patch. No need to rebuild. I'll post a wiki entry about it when I
get a chance

On Sat, May 25, 2019, 10:37 AM javashin, [e-mail] wrote:

So the patch is going to be in upstream ? i dont need to add the patch to
/etc/portage/patches ? if there a updated patch paste here ::) nice work
@InBetweenNames https://github.com/InBetweenNames

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/InBetweenNames/gentooLTO/issues/325?email_source=notifications&email_token=AB3VLDFDML43MWBT7UCEQNDPXFFI7A5CNFSM4HN5KZC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWHS7TA#issuecomment-495923148,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB3VLDHBT4BMLN52IW6XZL3PXFFI7ANCNFSM4HN5KZCQ
.

[javashin]

@InBetweenNames hi wine-vanilla have a prelink useflag i should use this on my prelinked system ?
euse -i prelink says : prelink
app-emulation/wine-vanilla: Run prelink on DLLs during build; For Gentoo hardened, do not disable if you do not know what this means as it
can break things at runtime

[pchome]

@javashin

wine-vanilla have a prelink useflag i should use this on my prelinked system ?

It just adds prelink to dependencies, since you already have prelink installed it would change nothing for you.

prelink used by Wine to set relocations, if linker not support them. I guess on Gentoo hardened many things are restricted, so Wine may not be able to do it's work, than it will try to do something like
prelink --reloc-only ....

[javashin]

Ok Thanks @pchome

[InBetweenNames]

Still in discussion with the upstream prelink devs. Ran some tests for them and things look promising. I'd link it here, but it's a mailing list and I haven't been able to find any archives of it.

[Kokokokoka]

Which ML are you talking about? And how to use this prelink?

[javashin]

@Kokokokoka We Use Prelink-99999999 patched with user patches using the patch here but our systems have no pie and no ssp and compiled with -fno-plt i dont know if is a requirement but after patching you can use prelink just like any other prelink , like : prelink -amvR to prelink and prelink -au to unprelink

patch >>> https://github.com/InBetweenNames/gentooLTO/issues/325#issuecomment-494910320

[javashin]

@InBetweenNames hi any news on prelink still working fine here

[InBetweenNames]

Still not merged upstream. Hopefully Mark Hatle will merge it tomorrow.

[javashin]

@InBetweenNames Cool

[InBetweenNames]

Well, upstream has been very busy, so I suppose I'll just add the patch to the overlay.

[javashin]

thanks

[javashin]

@InBetweenNames your prelink patch its upstream !!

This Saved me a lot of time rebuilding everything , i copy / paste some flags from other users and ended with a totally borked system , never trust others silly stuff ...... just follow @InBetweenNames Defaults / recommendations / Profile.

[InBetweenNames]

Awesome, very glad to see that!

[javashin]

:)