Graal: [native-image] Running into "PosixDirectives.c:2223:101: error: ‘SIOCGSTAMP’" after libs upgrade

[ricardozanini]

Hi!

This could be related to my own environment, but since I haven't found any issues reporting this same error, I guess that might help someone else in the future in case we found the root cause.

I ran a full update on my workstation (Fedora 30) and the native-image compilation stopped working since then (might be related, idk yet):

/usr/lib/jvm/graalvm/bin/native-image -H:+ReportExceptionStackTraces  -jar target/graalvm-playground-0.0.1-SNAPSHOT.jar 
Build on Server(pid: 2420, port: 46197)
[graalvm-playground-0.0.1-SNAPSHOT:2420]    classlist:     181.54 ms
[graalvm-playground-0.0.1-SNAPSHOT:2420]        (cap):     353.87 ms
[graalvm-playground-0.0.1-SNAPSHOT:2420]        setup:     455.22 ms
Error: Error compiling query code (in /tmp/SVM-6422748959591789691/PosixDirectives.c). Compiler command  gcc /tmp/SVM-6422748959591789691/PosixDirectives.c -o /tmp/SVM-6422748959591789691/PosixDirectives output included error: /tmp/SVM-6422748959591789691/PosixDirectives.c:2223:101: error: ‘SIOCGSTAMP’ undeclared (first use in this function); did you mean ‘SIOCGRARP’?
    method com.oracle.svm.core.posix.headers.Unistd._SC_LEVEL3_CACHE_ASSOC()
    C file contents around line 2223:
    /tmp/SVM-6422748959591789691/PosixDirectives.c:2222:     printf("NativeCodeInfo:PosixDirectives:ConstantInfo:_SC_LEVEL3_CACHE_ASSOC:PropertyInfo:value=%lX\n", _SC_LEVEL3_CACHE_ASSOC);
    /tmp/SVM-6422748959591789691/PosixDirectives.c:2223:     printf("NativeCodeInfo:PosixDirectives:ConstantInfo:SIOCGSTAMP:PropertyInfo:size=%lu\n", sizeof(SIOCGSTAMP));
    /tmp/SVM-6422748959591789691/PosixDirectives.c:2224:     printf("NativeCodeInfo:PosixDirectives:ConstantInfo:SIOCGSTAMP:PropertyInfo:signedness=$%s$\n", (IS_CONST_UNSIGNED(SIOCGSTAMP)) ? "UNSIGNED" : "SIGNED");
com.oracle.svm.core.util.UserError$UserException: Error compiling query code (in /tmp/SVM-6422748959591789691/PosixDirectives.c). Compiler command  gcc /tmp/SVM-6422748959591789691/PosixDirectives.c -o /tmp/SVM-6422748959591789691/PosixDirectives output included error: /tmp/SVM-6422748959591789691/PosixDirectives.c:2223:101: error: ‘SIOCGSTAMP’ undeclared (first use in this function); did you mean ‘SIOCGRARP’?
    method com.oracle.svm.core.posix.headers.Unistd._SC_LEVEL3_CACHE_ASSOC()
    C file contents around line 2223:
    /tmp/SVM-6422748959591789691/PosixDirectives.c:2222:     printf("NativeCodeInfo:PosixDirectives:ConstantInfo:_SC_LEVEL3_CACHE_ASSOC:PropertyInfo:value=%lX\n", _SC_LEVEL3_CACHE_ASSOC);
    /tmp/SVM-6422748959591789691/PosixDirectives.c:2223:     printf("NativeCodeInfo:PosixDirectives:ConstantInfo:SIOCGSTAMP:PropertyInfo:size=%lu\n", sizeof(SIOCGSTAMP));
    /tmp/SVM-6422748959591789691/PosixDirectives.c:2224:     printf("NativeCodeInfo:PosixDirectives:ConstantInfo:SIOCGSTAMP:PropertyInfo:signedness=$%s$\n", (IS_CONST_UNSIGNED(SIOCGSTAMP)) ? "UNSIGNED" : "SIGNED");
    at com.oracle.svm.core.util.UserError.abort(UserError.java:114)
    at com.oracle.svm.hosted.c.NativeLibraries.reportErrors(NativeLibraries.java:196)
    at com.oracle.svm.hosted.NativeImageGenerator.processNativeLibraryImports(NativeImageGenerator.java:1512)
    at com.oracle.svm.hosted.NativeImageGenerator.setupNativeLibraries(NativeImageGenerator.java:993)
    at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:827)
    at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:520)
    at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:440)
    at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Error: Image build request failed with exit status 1

glibc-devel version is glibc-devel-2.29-15.fc30.x86_64
zlib-devel version is zlib-devel-1.2.11-15.fc30.x86_64

GraalVM:

openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-20190711120915.buildslave.jdk8u-src-tar--b08)
OpenJDK 64-Bit GraalVM CE 19.1.1 (build 25.222-b08-jvmci-19.1-b01, mixed mode)

Have anyone faced this error before?

[bobmcwhirter]

Seems on my F29 machine that #define lives in

./asm-generic/sockios.h:#define SIOCGSTAMP 0x8906 /* Get stamp (timeval) */

The PosixDirectives file is the root headers that are processed to bring in the various constants/defines.

The route to that seems to be:

sys/socket.h -> bits/socket.h -> asm/socket.h -> asm-generic/socket.h -> asm/sockios.h -> asm-generic/sockios.h

Can you verify on your system the location of the #define SIOCGSTAMP and if the same series of #includes can reach it from sys/socket.h which is included in the PosixDirectives?

[ricardozanini]

Hi Bob! Thanks for your help! :)

Here's the output for my system:

echo "#include <sys/socket.h>" | gcc -E -dM - | grep SIOCGSTAMP
#define SIOCGSTAMP_OLD 0x8906
#define SIOCGSTAMPNS_OLD 0x8907

I guess someone updated the variable and now might be somewhere else. I'm not gcc specialist, so I could be doing something really wrong.

[ricardozanini]

Found this:
https://lists.sr.ht/~philmd/qemu/patches/6945
https://lkml.org/lkml/2019/7/20/108

The SIOCGSTAMP symbol was previously defined in the
asm-generic/sockios.h header file. QEMU sees that header
indirectly via sys/socket.h

In linux kernel commit 0768e17073dc527ccd18ed5f96ce85f9985e9115
the asm-generic/sockios.h header no longer defines SIOCGSTAMP.
Instead it provides only SIOCGSTAMP_OLD, which only uses a
32-bit time_t on 32-bit architectures.

The linux/sockios.h header then defines SIOCGSTAMP using
either SIOCGSTAMP_OLD or SIOCGSTAMP_NEW as appropriate. If
SIOCGSTAMP_NEW is used, then the tv_sec field is 64-bit even
on 32-bit architectures

:(

[bobmcwhirter]

Are you comfortable building Substrate from source?

If so, try adding <linux/sockios.h> to this stanza in PosixDirectives maybe.

    private static final String[] linuxLibs = new String[]{
                    "<arpa/inet.h>",
                    "<sys/epoll.h>",
                    "<sys/sendfile.h>",
                    "<mntent.h>",
                    "<link.h>",
    };

[ricardozanini]

Yeah, makes sense recompiling the Substrate with the updated headers. Looking at the docs, I'll have to set some tools I don't have yet. I'm going to give it a try and let you know. Many thanks for your help!

[peter-hofer]

Can you please try whether simply removing SIOCGSTAMP() in com/oracle/svm/core/posix/headers/Socket.java works? There don't seem to be any usages to begin with.

[ricardozanini]

I'll give it a try tomorrow morning. Sorry to not replying until now, I got a lot of other things to work with :cry:

[adinn]

Can you please try whether simply removing SIOCGSTAMP() in com/oracle/svm/core/posix/headers/Socket.java works? There don't seem to be any usages to begin with.

I just updated and ran into this problem. I can confirm that your suggestion bypasses the problem.

[arashi01]

Also just ran into this on Fedora 30. Is the workaround confirmed to work?

[omeuerodrigofreitas]

Hi, I'm having the same problem in fedora 29

erro.txt

[peter-hofer]

Thanks @adinn, I'll merge a fix to master.

[adinn]

Hi, I'm having the same problem in fedora 29

Yes, the same OS upgrade has been posted for both fedora 29 and 30. SO, my home and work machine are both broken.

This is a real nuisance as it affects current published releases of GraalVM . I can no longer use either 19.0 or 19.2 to build native apps. If there is a new release on the way then it would be good to get this patch in.

[ricardozanini]

+1 for this patch.

Many thanks @adinn for testing @peter-hofer suggestion. I can confirm that his suggestion worked fine. :heart:

[peter-hofer]

Fixed in b09b1f6babaaf16d3eaa7888b7826df63964d490. I will also soon remove many more declarations like this to reduce the chances of this happening again.
Unfortunately, the fix did not make it into 19.2.0. Is there a suitable workaround on the system level?

[adinn]

Unfortunately, the fix did not make it into 19.2.0. Is there a suitable workaround on the system level?

Well, I think the only options are to build your own GraalVM release with the patch included or to run the native image build in an OS that has not been updated. I suspect that only the latter will be feasible for most users -- probably best achieved by building in a container or VM.

[omeuerodrigofreitas]

Hi, the problem still persists :/ (GraalVM-19.02.0)
error.txt

[peter-hofer]

@RFigueira , unfortunately, the fix did not make it into 19.2.0.

[omeuerodrigofreitas]

hi, @peter-hofer, how are you?
some prevision about?