Esp-idf: Build Fails in Python 3.9 (IDFGH-4129)

[Leonardobat]

Environment

• Module or chip used: ESP32-S2-WROOM
• IDF version: master
• Build System: idf.py
• Compiler version : 8.4.0
• Operating System: Linux Fedora 33 Beta

Problem Description

My build stops on ´[984/987] Generating ld/esp32s2.project.ld´ when it crashes.

Expected Behavior

Finish the build.

Actual Behavior

It crashes showing this traceback:

Traceback (most recent call last):
  File "/home/leo/.esp-idf/tools/ldgen/generation.py", line 591, in add_sections_info
    results = parser.parseString(first_line)
  File "/home/leo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages/pyparsing.py", line 1814, in parseString
    raise exc
  File "/home/leo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages/pyparsing.py", line 1804, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/home/leo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages/pyparsing.py", line 1548, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/leo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages/pyparsing.py", line 3722, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/home/leo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages/pyparsing.py", line 1552, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/leo/.espressif/python_env/idf4.3_py3.9_env/lib/python3.9/site-packages/pyparsing.py", line 3472, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pyparsing.ParseException: Expected end of line (at char 33), (line:1, col:34)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/leo/.esp-idf/tools/ldgen/ldgen.py", line 158, in <module>
    main()
  File "/home/leo/.esp-idf/tools/ldgen/ldgen.py", line 116, in main
    sections_infos.add_sections_info(dump)
  File "/home/leo/.esp-idf/tools/ldgen/generation.py", line 593, in add_sections_info
    raise ParseException("Parsing sections info for library " + sections_info_dump.name + " failed. " + p.msg)
pyparsing.ParseException: Parsing sections info for library /home/leo/Documentos/Códigos/Esp/Lophostrix/build/esp-idf/xtensa/libxtensa.a failed. Expected end of line (at char 0), (line:1, col:1)

Steps to reproduce

1. Install ESP-IDF in a system with Python 3.9
2. Try build any code

[Leonardobat]

Found the issue, it seems to be a lack of Unicode support. I don't know exactly what this piece of code does, but it seems the parser cannot receive any non-ASCII char. I'm Brazillian and in Portuguese, we use chars like ã, ó, or ç these appear to break your code.

[dobairoland]

Hi @Leonardobat. Thank you for reporting this issue.

Can you please check if the following simple change fixes your issue?

Line 117 of tools/ldgen/ldgen.py is the following:

dump = StringIO(subprocess.check_output([objdump, "-h", library]).decode())

Could you please try to add utf-8 into .decode()? The line should look like:

dump = StringIO(subprocess.check_output([objdump, "-h", library]).decode('utf-8'))

[Leonardobat]

Still the same issue

[dobairoland]

Hi @Leonardobat. I'm sorry I didn't get it right in the first place. Here is another fix which should work if there is no other missed Unicode handling.

Line 582 of tools/ldgen/generation.py is:

        archive_path = (Literal("In archive").suppress() +
                        # trim the last character from archive_path, :
                        Word(printables + " ").setResultsName("archive_path").setParseAction(lambda t: t[0][:-1]) +
                        LineEnd())

Please replace it with these two lines (keeping the same indentation):

        from pyparsing import restOfLine
        archive_path = (Literal("In archive").suppress() + White().suppress() + restOfLine.setResultsName("archive_path").setParseAction(lambda t: t[0][:-1]))

Is your build successful with this patch?

[Leonardobat]

Yes, It worked, the build was successful

[dobairoland]

Thank you for the confirmation!