Esp-idf: OSX High Sierra (10.31.1), Xcode 9.1, tools/kconfig build issues (IDFGH-115)

[markwj]

Since upgrading to High Sierra, and doing a 'make clean', I can't get tools/kconfig to build any more.

I am using the release/v2.1 version of ESP-IDF.

Two faults I can find (so far):

1] flex on zconf.l:

$ flex -V
flex 2.5.35 Apple(flex-31)
$ make
cc  -I/opt/local/include  -DCURSES_LOC="<ncurses.h>" -DNCURSES_WIDECHAR=1 -DKBUILD_NO_NLS -Wno-format-security  -DLOCALE   -c -o mconf.o mconf.c
flex -L -P zconf -o zconf.lex.c zconf.l
zconf.l:255: warning, -s option given but default rule can be matched
make: *** [zconf.lex.c] Error 1

2] zconf.tab MAX_PATH

$ make
bison -t -l -p zconf -o zconf.tab.c zconf.y
sed -E "s/\\x0D$//" zconf.gperf | gperf -t --output-file zconf.hash.c -a -C -E -g -k '1,3,$' -p -t
cc  -I/opt/local/include  -DCURSES_LOC="<ncurses.h>" -DNCURSES_WIDECHAR=1 -DKBUILD_NO_NLS -Wno-format-security  -DLOCALE   -c -o zconf.tab.o zconf.tab.c
In file included from zconf.tab.c:237:
./zconf.hash.c:244:44: warning: static variable 'kconf_id_strings_contents' is used in an inline function with external linkage [-Wstatic-in-inline]
              register const char *s = o + kconf_id_strings;
                                           ^
./zconf.hash.c:160:43: note: expanded from macro 'kconf_id_strings'
#define kconf_id_strings ((const char *) &kconf_id_strings_contents)
                                          ^
./zconf.hash.c:162:1: note: use 'static' to give inline function 'kconf_id_lookup' internal linkage
__inline
^
static
./zconf.hash.c:123:40: note: 'kconf_id_strings_contents' declared here
static const struct kconf_id_strings_t kconf_id_strings_contents =
                                       ^
In file included from zconf.tab.c:2560:
./confdata.c:113:23: error: use of undeclared identifier 'PATH_MAX'
        static char fullname[PATH_MAX+1];

The second (zconf.tab) one is relatively simple. While limits.h is included in mconf.c, nconf.h, zconf.l; it is not included in zconf.y. Adding it there, we now get some warnings:

$ make
bison -t -l -p zconf -o zconf.tab.c zconf.y
cc  -I/opt/local/include  -DCURSES_LOC="<ncurses.h>" -DNCURSES_WIDECHAR=1 -DKBUILD_NO_NLS -Wno-format-security  -DLOCALE   -c -o zconf.tab.o zconf.tab.c
In file included from zconf.tab.c:238:
./zconf.hash.c:244:44: warning: static variable 'kconf_id_strings_contents' is used in an inline function with external linkage [-Wstatic-in-inline]
              register const char *s = o + kconf_id_strings;
                                           ^
./zconf.hash.c:160:43: note: expanded from macro 'kconf_id_strings'
#define kconf_id_strings ((const char *) &kconf_id_strings_contents)
                                          ^
./zconf.hash.c:162:1: note: use 'static' to give inline function 'kconf_id_lookup' internal linkage
__inline
^
static
./zconf.hash.c:123:40: note: 'kconf_id_strings_contents' declared here
static const struct kconf_id_strings_t kconf_id_strings_contents =

but at least it compiles.

I'm still looking at the first issue (zconf.l). That is resulting in a 0 byte generated zconf.lex.c, while messes up the compilation.

[mringwal]

looks like a bug in flex but I have no idea where it comes from.

temporary fix: place zconf.lex.c into tools/kconfig
zconf.lex.c.zip

[igrr]

Couldn't reproduce this with OS X High Sierra 10.13.2, Xcode 9.2, homebrew installed. Are you using MacPorts or Homebrew? Is MacPorts, have you updated it after upgrading to 10.13?

[mringwal]

@igrr I usually use Fink (I'm old-school). Could you manually delete zconf.lex.c and try to compile a fresh esp-idf project? I think make clean in a project also did remove zconf.lex.c file.

And/or: what flex version do you have? $ flex -V

[igrr]

Did make -C $IDF_PATH/tools/kconfig clean, did make all again — same result, still builds:

($IDF_PATH/tools/kconfig) $ make all
cc  -DCURSES_LOC="<ncurses.h>" -DKBUILD_NO_NLS -Wno-format-security  -DLOCALE -MD   -c -o mconf.o mconf.c
flex -L -P zconf -o zconf.lex.c zconf.l
bison -t -l -p zconf -o zconf.tab.c zconf.y
sed -E "s/\\x0D$//" zconf.gperf | gperf -t --output-file zconf.hash.c -a -C -E -g -k '1,3,$' -p -t
cc  -DCURSES_LOC="<ncurses.h>" -DKBUILD_NO_NLS -Wno-format-security  -DLOCALE -MD   -c -o zconf.tab.o zconf.tab.c
lxdialog/check-lxdialog.sh -check cc  -DCURSES_LOC="<ncurses.h>" -DKBUILD_NO_NLS -Wno-format-security  -DLOCALE -MD -lncurses 
cc  -DCURSES_LOC="<ncurses.h>" -DKBUILD_NO_NLS -Wno-format-security  -DLOCALE -MD   -c -o lxdialog/checklist.o lxdialog/checklist.c
cc  -DCURSES_LOC="<ncurses.h>" -DKBUILD_NO_NLS -Wno-format-security  -DLOCALE -MD   -c -o lxdialog/util.o lxdialog/util.c
cc  -DCURSES_LOC="<ncurses.h>" -DKBUILD_NO_NLS -Wno-format-security  -DLOCALE -MD   -c -o lxdialog/inputbox.o lxdialog/inputbox.c
cc  -DCURSES_LOC="<ncurses.h>" -DKBUILD_NO_NLS -Wno-format-security  -DLOCALE -MD   -c -o lxdialog/textbox.o lxdialog/textbox.c
cc  -DCURSES_LOC="<ncurses.h>" -DKBUILD_NO_NLS -Wno-format-security  -DLOCALE -MD   -c -o lxdialog/yesno.o lxdialog/yesno.c
cc  -DCURSES_LOC="<ncurses.h>" -DKBUILD_NO_NLS -Wno-format-security  -DLOCALE -MD   -c -o lxdialog/menubox.o lxdialog/menubox.c
cc -o mconf mconf.o zconf.tab.o lxdialog/checklist.o lxdialog/util.o lxdialog/inputbox.o lxdialog/textbox.o lxdialog/yesno.o lxdialog/menubox.o -lncurses 
cc  -DCURSES_LOC="<ncurses.h>" -DKBUILD_NO_NLS -Wno-format-security  -DLOCALE -MD   -c -o conf.o conf.c
cc -o conf conf.o  zconf.tab.o -lncurses 

$ flex -V
flex 2.5.35 Apple(flex-31)

Fwiw, i think that the "warning, -s option given but default rule can be matched" issue has been resolved recently in https://github.com/espressif/esp-idf/commit/78aa1ec704ad79bf15cd52fa75db2c1eb8767d8d.

For the compilation warnings, since they are all from zconf.hash.c, gperf would be my suspect. Here's the version i have:

$ gperf -v
GNU gperf 3.0.3
...
$ which gperf
/usr/bin/gperf

[jannik-kramer]

I'm having (as far as i can tell) the same error messages. Managed to get bison running after i updated to high sierra but now i get a lot "undeclared identifier"-errors.
zconf.lex.c is always empty and overwriting it will only result in it being overwritten again as soon as i run make menuconfig.

$ gperf -v
GNU gperf 3.0.4
...
$ which gperf
/opt/local/bin/gperf
$ flex -V
flex 2.5.35 Apple(flex-31)

Full error on gist

[theproxy]

I get the exact same errors as @jannik-kramer but when I do gperf -V I have 3.1. My flex version is identical to above (flex 2.5.35 Apple(flex-31)).

EDIT: fix from @mringwal worked for me. Copied his file into $IDF_PATH/tools/kconfig

[avrahamshukron]

Adding the include <limits.h> in zconf.y, and copying the zconf.lex.c fixed the problem for me...

[toxuin]

Updating flex with brew seems to solve the issue.

$ flex -V
flex 2.6.4

[ideaoven]

This is still an issue in 3.1.1 using MacOS Mojave (10.14.1 (18B75)). During the build the flex command returns a blank zconf.lex.c file. For me this occurred with both the Apple flex (2.5.35 Apple(flex-31)) and the latest flex installed with homebrew (2.6.4). For anyone trying to get this to compile using the getting started tutorial, you need to copy the file from @mringwal into tools/kconfig and comment out the line in Makefile where flex is used to generate zconf.lex.c. I banged my head on this not realizing the zconf.lex.c was getting overwritten with blank file each time I ran 'make menuconfig'

[avrahamshukron]

This is so fucked up... flex just produces an empty file and fails silently. No error whatsoever and no way to understand what went wrong.

Subsequent compilation will further hide the problem since zconf.lex.c - although empty - will exist, so every time you compile make will just skip the generation of this file.

I have no idea what is the underlying problem here but this is a really big deal...

[dobairoland]

I'm closing this old issue reported for (now) unsupported v2.1 IDF. If the problem still exists then please open a new issue for it. Note that starting with IDF v4.1 the old kconfig tools have been replaced and issue similar to this will not happen.