Robotframework: Update standalone jar distribution to use Jython 2.7.2

[tpokki]

Encountered this issue when trying to use RESTinstance module with robotframework-maven-plugin: https://bugs.jython.org/issue2476

Importing test library 'REST' failed: AttributeError: 'reflectedfunction' object has no attribute '__module__'
Traceback (most recent call last):
  File "/work/src/test/robotframework/libraries.zip/REST/__init__.py", line 37, in <module>
    from .keywords import Keywords
  File "/work/src/test/robotframework/libraries.zip/REST/keywords.py", line 35, in <module>
    from jsonschema import validate, FormatChecker
  File "/work/src/test/robotframework/libraries.zip/jsonschema/__init__.py", line 33, in <module>
    import importlib_metadata as metadata
  File "/work/src/test/robotframework/libraries.zip/importlib_metadata/__init__.py", line 15, in <module>
    from ._compat import (
  File "/work/src/test/robotframework/libraries.zip/importlib_metadata/_compat.py", line 30, in <module>
    import pathlib2 as pathlib
  File "/work/src/test/robotframework/libraries.zip/pathlib2/__init__.py", line 580, in <module>
    class _NormalAccessor(_Accessor):
  File "/work/src/test/robotframework/libraries.zip/pathlib2/__init__.py", line 598, in _NormalAccessor
    open = _wrap_strfunc(os.open)
  File "/work/src/test/robotframework/libraries.zip/pathlib2/__init__.py", line 583, in _wrap_strfunc
    @functools.wraps(strfunc)
  File "/home/user/.m2/repository/org/robotframework/robotframework/3.1.1/robotframework-3.1.1.jar/Lib/functools.py", line 33, in update_wrapper
    setattr(wrapper, attr, getattr(wrapped, attr))
PYTHONPATH:
  /work/src/test/robotframework/libraries.zip
  /home/user/.m2/repository/org/robotframework/robotframework/3.1.1/Lib
  /home/user/.m2/repository/org/robotframework/robotframework/3.1.1/robotframework-3.1.1.jar/Lib
  __classpath__
  __pyclasspath__/
CLASSPATH:
  /work/apache-maven-3.5.4/boot/plexus-classworlds-2.5.2.jar

Uprading to jython 2.7.1 might solve that, but apparently it is not as trivial as just changing the version in tasks.py...

[pekkaklarck]

Yeah, changing the Jython version that's embedded to the Maven plugin requires creating a new robotframework.jar with that Jython version and then creating a new Maven plugin with that jar. Luckily RF 3.2 is currently under active development, hopefully released still this year, and we could change the Jython version at the same time. I recently tested Robot with Jython 2.7.2 beta 2 and fixed some issues that affected also Jython 2.7.1 (#3364). If Jython 2.7.2 is released before RF 3.2, we could use that instead.

It would also be good to verify that Jython 2.7.1 (or .2) actually fixes this issue. You can test that by installing Jython 2.7.1/2 separately and then testing RESTinstance with it. If the problem is fixed, we definitely should update the version.

[tpokki]

Both 2.7.1 and 2.7.2b seem to resolve the issue:

2.7.0 fails

$ java -jar /work/ext-lib/jython-standalone-2.7.0.jar /tmp/deps/robot /tmp/deps/REST rest.robot 
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/work/ext-lib/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
WARNING: Please consider reporting this to the maintainers of jnr.posix.JavaLibCHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Nov 22, 2019 11:22:08 AM org.python.netty.util.internal.PlatformDependent <clinit>
INFO: Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system unstability.
==============================================================================
REST & Rest                                                                   
==============================================================================
REST & Rest.REST                                                              
==============================================================================
REST & Rest.REST                                                      | PASS |
0 critical tests, 0 passed, 0 failed
0 tests total, 0 passed, 0 failed
==============================================================================
[ ERROR ] Error in file '/tmp/rest.robot': Importing test library 'REST' failed: AttributeError: 'reflectedfunction' object has no attribute '__module__'
Traceback (most recent call last):
  File "/tmp/deps/REST/__init__.py", line 37, in <module>
    from .keywords import Keywords
  File "/tmp/deps/REST/keywords.py", line 35, in <module>
    from jsonschema import validate, FormatChecker
  File "/tmp/deps/jsonschema/__init__.py", line 33, in <module>
    import importlib_metadata as metadata
  File "/tmp/deps/importlib_metadata/__init__.py", line 15, in <module>
    from ._compat import (
  File "/tmp/deps/importlib_metadata/_compat.py", line 30, in <module>
    import pathlib2 as pathlib
  File "/tmp/deps/pathlib2/__init__.py", line 580, in <module>
    class _NormalAccessor(_Accessor):
  File "/tmp/deps/pathlib2/__init__.py", line 598, in _NormalAccessor
    open = _wrap_strfunc(os.open)
  File "/tmp/deps/pathlib2/__init__.py", line 583, in _wrap_strfunc
    @functools.wraps(strfunc)
  File "/work/ext-lib/jython-standalone-2.7.0.jar/Lib/functools.py", line 33, in update_wrapper
    setattr(wrapper, attr, getattr(wrapped, attr))
PYTHONPATH:
  /tmp/d2
  /work/ext-lib/Lib
  /work/ext-lib/jython-standalone-2.7.0.jar/Lib
  __classpath__
  __pyclasspath__/
CLASSPATH:
  /work/ext-lib/jython-standalone-2.7.0.jar
REST & Rest.Rest                                                              
==============================================================================
Successful Rest API Call                                              | FAIL |
No keyword with name 'GET' found.
------------------------------------------------------------------------------
REST & Rest.Rest                                                      | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================
REST & Rest                                                           | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================
Output:  /tmp/output.xml
Log:     /tmp/log.html
Report:  /tmp/report.html

2.7.1 works, although it has some obscure exception printed

$ java -jar /work/ext-lib/jython-standalone-2.7.1.jar /tmp/deps/robot /tmp/deps/REST rest.robot 
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/work/ext-lib/jython-standalone-2.7.1.jar) to method sun.nio.ch.SelChImpl.getFD()
WARNING: Please consider reporting this to the maintainers of jnr.posix.JavaLibCHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[ERROR] Failed to construct terminal; falling back to unsupported
java.lang.NumberFormatException: For input string: "0x100"
    at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.base/java.lang.Integer.parseInt(Integer.java:652)
    at java.base/java.lang.Integer.valueOf(Integer.java:983)
    at org.python.jline.internal.InfoCmp.parseInfoCmp(InfoCmp.java:59)
    at org.python.jline.UnixTerminal.parseInfoCmp(UnixTerminal.java:242)
    at org.python.jline.UnixTerminal.<init>(UnixTerminal.java:65)
    at org.python.jline.UnixTerminal.<init>(UnixTerminal.java:50)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at java.base/java.lang.Class.newInstance(Class.java:584)
    at org.python.jline.TerminalFactory.getFlavor(TerminalFactory.java:211)
    at org.python.jline.TerminalFactory.create(TerminalFactory.java:102)
    at org.python.jline.TerminalFactory.get(TerminalFactory.java:186)
    at org.python.jline.TerminalFactory.get(TerminalFactory.java:192)
    at org.python.jline.console.ConsoleReader.<init>(ConsoleReader.java:243)
    at org.python.util.JLineConsole.install(JLineConsole.java:107)
    at org.python.core.Py.installConsole(Py.java:1744)
    at org.python.core.PySystemState.initConsole(PySystemState.java:1269)
    at org.python.core.PySystemState.doInitialize(PySystemState.java:1119)
    at org.python.core.PySystemState.initialize(PySystemState.java:1033)
    at org.python.core.PySystemState.initialize(PySystemState.java:989)
    at org.python.core.PySystemState.initialize(PySystemState.java:984)
    at org.python.util.jython.run(jython.java:263)
    at org.python.util.jython.main(jython.java:142)

==============================================================================
REST & Rest                                                                   
==============================================================================
REST & Rest.REST                                                              
==============================================================================
REST & Rest.REST                                                      | PASS |
0 critical tests, 0 passed, 0 failed
0 tests total, 0 passed, 0 failed
==============================================================================
REST & Rest.Rest                                                              
==============================================================================
Successful Rest API Call                                              | PASS |
------------------------------------------------------------------------------
REST & Rest.Rest                                                      | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
REST & Rest                                                           | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output:  /tmp/output.xml
Log:     /tmp/log.html
Report:  /tmp/report.html

2.7.2b works

$ java -jar /work/ext-lib/jython-standalone-2.7.2b2.jar /tmp/deps/robot /tmp/deps/REST rest.robot 
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.python.core.io.StreamIO (file:/work/ext-lib/jython-standalone-2.7.2b2.jar) to field java.io.FilterOutputStream.out
WARNING: Please consider reporting this to the maintainers of org.python.core.io.StreamIO
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
==============================================================================
REST & Rest                                                                   
==============================================================================
REST & Rest.REST                                                              
==============================================================================
REST & Rest.REST                                                      | PASS |
0 critical tests, 0 passed, 0 failed
0 tests total, 0 passed, 0 failed
==============================================================================
REST & Rest.Rest                                                              
==============================================================================
Successful Rest API Call                                              | PASS |
------------------------------------------------------------------------------
REST & Rest.Rest                                                      | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
REST & Rest                                                           | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output:  /tmp/output.xml
Log:     /tmp/log.html
Report:  /tmp/report.html

[pekkaklarck]

OK, thanks for testing! This is definitely a good enough reason to upgrade Jython embedded to the jar distribution to 2.7.1 or 2.7.2 (if it is released before RF 3.2).

[pekkaklarck]

We've typically only created the standalone jar only with the final release, but I guess this time we could do that already with release candidates or possibly even with betas. I don't want to upload those the Maven central, though, but it is possible to attach distributions to GitHub releases as well.

[tysonite]

As per Jython web site, 2.7.2rc1 has been release in March 2020.

[pekkaklarck]

Yeah, looks like we can upgrade to Jython 2.7.2. Need to test how that rc1 works. Would it be hard to get it running on CI?

[tysonite]

I can give it a shot. Let me try on my fork.

[tysonite]

The run for Jython 2.7.2rc1 on Java 8 finished. The results are pretty good, no unit test failures, acceptance tests pass on Linux, but there were few failures on Windows:

FAIL: Robot.Running.Non Ascii Bytes.In Message
Wrong log message: ?iti != �iti
----------------------------------------------------------------------------------------------------
FAIL: Robot.Running.Non Ascii Bytes.In Multiline Message
Wrong log message: ?iti
is? != �iti
is�
----------------------------------------------------------------------------------------------------
FAIL: Robot.Standard Libraries.Operating System.Create File.Create File With Console Encoding
Test 'Create File With Console Encoding' was expected to PASS but it FAILED.

Error message:
UnicodeEncodeError: 'charmap' codec can't encode character u'\xcf' in position 7: character maps to <undefined>
====================================================================================================
Run suite 'Robot' with 4840 tests in 3 hours 10 minutes 9 seconds 874 milliseconds.

FAILED
4840 critical tests, 4837 passed, 3 failed
4840 tests total, 4837 passed, 3 failed

Full log can be downloaded here.

Results for Jython 2.7.2rc1 on Java 11 will be available tomorrow. However, I expect many failures (e.g. 4 tests fail as I noted in #3495).

[pekkaklarck]

I tested Jython 2.7.2rc1 on Java 8 locally and these are the current results:

• On Linux all tests passed.
• On my Windows VM I just started execution and it'll take time before all tests are run.
• I tested the three tests above that had failed on Windows separately and they all passed. That's pretty strange and my guess is that it somehow depends on the environment. What is the chcp there?

[tysonite]

We set chcp 850 on Windows before running acceptance tests.

[pekkaklarck]

That's what I have as well. As you probably remember, we earlier had a bug related to getting the _current_ code page but that was fixed by 856afa6ed2a16e39194b14bce138aa2044e0b0b6. One possibility is that there's still code somewhere, possibly in Jython or Java, that gets the system wide code page, not the current one. Do tests succeed on Windows with earlier Jython versions?

[tysonite]

Yes, 2.7.1 passes tests on the same RF revision. One thing I noted is about number of tests:

• on Jython 2.7.1 (Win) there were 4831 tests in total
• on Jython 2.7.2rc2 (Win) - 4840 tests

Command lines used to run tests:

C:\hostedtoolcache\windows\Python\3.6.8\x64\python.exe D:\a\robotframework\robotframework\src\robot\run.py --doc Robot Framework acceptance tests --metadata interpreter:Jython 2.7.2 on Windows --variablefile D:\a\robotframework\robotframework\atest\interpreter.py;d:\a\robotframework\robotframework/jython/bin/jython;Jython;2.7.2 --pythonpath D:\a\robotframework\robotframework\atest\resources --outputdir D:\a\robotframework\robotframework\atest\results\Jython-2.7.2-Windows --splitlog --console dotted --consolewidth 100 --SuiteStatLevel 3 --TagStatExclude no-* --exclude no-jython --exclude require-lxml --exclude require-ipy --exclude require-py3 --exclude require-py3.5 --exclude require-py3.7 --exclude no-windows --exclude no-windows-jython --exclude no-ci atest/robot

C:\hostedtoolcache\windows\Python\3.6.8\x64\python.exe D:\a\robotframework\robotframework\src\robot\run.py --doc Robot Framework acceptance tests --metadata interpreter:Jython 2.7.1 on Windows --variablefile D:\a\robotframework\robotframework\atest\interpreter.py;d:\a\robotframework\robotframework/jython/bin/jython;Jython;2.7.1 --pythonpath D:\a\robotframework\robotframework\atest\resources --outputdir D:\a\robotframework\robotframework\atest\results\Jython-2.7.1-Windows --splitlog --console dotted --consolewidth 100 --SuiteStatLevel 3 --TagStatExclude no-* --exclude no-jython --exclude require-lxml --exclude no-jython-2.7.1 --exclude require-ipy --exclude require-py3 --exclude require-py3.5 --exclude require-py3.7 --exclude no-windows --exclude no-windows-jython --exclude no-ci atest/robot

I see difference only in --exclude no-jython-2.7.1 option.

[tysonite]

I tried to run Robot.Running.Non Ascii Bytes on my local Windows box using Jython 2.7.2rc1 / Java 8. The results are:

• chcp 850: The same failures as above
• chcp 437: No failures.

[tysonite]

I also run Robot.Running.Non Ascii Bytes on my local Windows box using Jython 2.7.1 / Java 8, no failures with both codepages. Thus there is definitely something different within Robot/tests.

[pekkaklarck]

I was able to reproduce Non Ascii Bytes failure with Jython 2.7.2rc1 by running first chcp 437 with my native code page being 850. I then noticed that the result of this command differs between Jython 2.7.0 and 2.7.2:

jython -c "import sys; print(sys.stdout.encoding)" > out.txt

With Jython 2.7.0 I get cp437 but with 2.7.2 I got cp850. If output isn't redirected, both print cp437. This problem is easy to avoid by not using the above way for encoding detection with Jython 2.7.2+ on Windows. I also submitted an issue about the change to Jython's tracker: https://bugs.jython.org/issue2868

I wasn't able to reproduce the problem with Robot.Standard Libraries.Operating System.Create File.

[tysonite]

Awesome finding! Acceptance tests pass now on Jython 2.7.2rc1 with Java 8. However, unit tests started to fail:

======================================================================
FAIL: test_invalid_encoding (test_encodingsniffer.TestGetConsoleEncodingFromStandardStreams)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\a\robotframework\robotframework\utest\utils\test_encodingsniffer.py", line 34, in test_invalid_encoding
    assert_equal(get_console_encoding(), 'ascII')
  File "D:\a\robotframework\robotframework\utest\..\src\robot\utils\asserts.py", line 182, in assert_equal
    _report_inequality(first, second, '!=', msg, values, formatter)
  File "D:\a\robotframework\robotframework\utest\..\src\robot\utils\asserts.py", line 231, in _report_inequality
    raise AssertionError(msg)
AssertionError: cp850 != ascII

[pekkaklarck]

Probably utest needs to be fixed. My bad.

[pekkaklarck]

I changed the code to read encoding from streams to better. No Jython special cases needed and thus no utest fixes needed either.

[tysonite]

Unit and acceptance tests pass now. We could add Jython 2.7.2rc1 (on Java 8) into CI scheduled runs if necessary.

[pekkaklarck]

Based on the discussion on Jython mailing list, Jython 2.7.2 will be available next week.

[tysonite]

It looks like that final version of Jython 2.7.2 has been made available: https://search.maven.org/artifact/org.python/jython-installer/2.7.2/jar

[pekkaklarck]

Updated the version in our tasks.py to Jython 2.7.2. Now it's possible to create a jar distribution with that version by running

invoke jar

in the project root directory. Based on quick testing it seems to work fine, but I want to run all acceptance tests with it before closing this issue. That needs to wait for tomorrow.

[tysonite]

Do you want it to be on CI as well? If yes, how to run acceptance tests with JAR (I don't see it is described in atest/README.md)?

[pekkaklarck]

1. I doubt it's worth the effort running it on CI. There shouldn't be any real differences compared to running with normal Jython but, if I remember correctly, the execution time is even longer. If you @tysonite are interested to try it out, I'm not really against it either.

2. It should be possible to run tests with the jar using atest/run.py so that the jar is the interpreter to use. For example, this should run all tests:

    $ atest/run.py dist/robotframework-3.2b3.dev1.jar -e no-ci atest/robot
   

3. Unfortunately the above seems to be broken due to the recent enhancement to check the Java version because that "interpreter" cannot be used to run any Python code. There's a separate StandaloneInterpreter that could check the Java version differently and one way would be running --version as its output contains also this information:

    $ java -jar dist/robotframework-3.2b3.dev1.jar --version
    Robot Framework 3.2b3.dev1 (Jython 2.7.2 on java1.8.0_201)
   

   This requires starting Robot, not only Jython, so it's slower than what we use normally. Thus it's not a good idea to change the normal approach to use --version. Do you @tysonite have time and interest to look at it? If not, I can take a look a bit later today.

4. It should also be possible to generate jar then run tests with it by using just jar as the interpreter. That seems to be totally broken and I seriously doubt fixing that is worth the effort. Probably best to remove that support altogether and require using invoke jar before running tests.

5. Using the jar as an interpreter is documented at least in atest/run.py docstring which is shown in atest/run.py --help. If there's nothing about it in atest/README.rst, something should be added.

[pekkaklarck]

The above commit fixes running atests with the standalone jar and I just started the test run locally.

[tysonite]

I am not feeling well for a couple of days already thus can't help much. Would it be possible to add an argument to invoke jar that specifies output JAR file name? It would be easier to use this approach on CI like below (of course, if we decide to run tests with JAR):

invoke jar --jar_name=robotframework.jar
atest/run.py dist/robotframework.jar --exclude no-ci atest/robot

[pekkaklarck]

If we decide to run atests w/ the jar, we certainly can make the jar path configurable. Let's first see how much work there's to make all tests pass locally.

Hope you get well soon @tysonite!

[pekkaklarck]

Tests with the standalone JAR revealed some problems both in tests and in code, some issues in code could have caused also problems in other contexts.

At the moment the only unresolved issues is that for some reason the new standalone JAR doesn't obey JYTHONPATH set externally before execution. I tested that if I build the JAR with Jython 2.7.0 using the current sources, JYTHONPATH works just fine. I don't know is this caused by some deliberate change in Jython or or we just embedding it wrong. I asked about the issue on Jython mailing list. If there's not other solution, we can process JYTHONPATH ourselves before starting execution.

[pekkaklarck]

The above commit adds explicit JYTHONPATH processing to standalone JAR. I'm not certain is that the best way to do it, but based on quick testing it seems to work. I'm now running all tests locally with a newly created jar both on Linux and on Windows. I hope there aren't major issues and we finally get this issue closed soon.

[pekkaklarck]

Tests passed on Linux and Windows. I guess we can consider this issue done.

[pekkaklarck]

RF 3.2 rc 1 is not out. You can find the standalone jar here:
https://github.com/robotframework/robotframework/releases/tag/v3.2rc1