Sbt: regression: letters that worked in 1.3.13 does not work in 1.4.x for example ÅÄÖ åäö, maybe jline 3 problem with Unicode or Utf-8 encoding?

[bjornregnell]

steps

Running sbt 1.4.x (x in any of {0,1,2,3}) on Ubuntu 18.04 (@SethTisue reported same error on MacOS)

$ sbt
[info] welcome to sbt 1.4.3 (Ubuntu Java 11.0.9.1)
sbt:bonus> ååäö  // does not echo åäö although these keys were pressed
sbt:bonus> console
[info] Starting scala interpreter...
Welcome to Scala 2.13.3 (OpenJDK 64-Bit Server VM, Java 11.0.9.1).
Type in expressions for evaluation. Or try :help.

scala> val åäö = "ÃÃÃ"  // wrong echoed char also in console when typing `val åäö = "ÅÄÖ"`
            ^
       error: illegal character '\u00a5'
              ^
       error: illegal character '\u00a4'
                ^
       error: illegal character '\u00b6'

problem

Special letters, e.g. the swedish letters ÅÄÖ åäö does not work anymore neither in sbt shell or in REPL started by console, althoug the worked fine in 1.3.13 and the work in vanilla scala 2.13.3 REPL.

expectation

Should work as it used to in 1.3.13:

[info] welcome to sbt 1.3.13 (Ubuntu Java 11.0.9.1)
sbt:bonus> åäö  // does echo as it should
sbt:bonus> console
[info] Starting scala interpreter...
Welcome to Scala 2.13.3 (OpenJDK 64-Bit Server VM, Java 11.0.9.1).
Type in expressions for evaluation. Or try :help.

scala> val åäö = "ÅÄÖ"
val åäö: String = ÅÄÖ

And as in vanilla REPL:

 $ scala
Welcome to Scala 2.13.3 (OpenJDK 64-Bit Server VM, Java 11.0.9.1).
Type in expressions for evaluation. Or try :help.

scala> val åäö = "ÅÄÖ"    // works just fine
val åäö: String = ÅÄÖ

notes

Maybe this has to do with jline

[eed3si9n]

@bjornregnell Thanks for the report.

Could you try starting sbt with sbt -v -Dfile.encoding=UTF-8 and see if the situation changes?

[bjornregnell]

@eed3si9n same problem with sbt -v -Dfile.encoding=UTF-8:

$ sbt -v -Dfile.encoding=UTF-8
[sbt_options] declare -a sbt_options=()
[process_args] java_version = '11'
[copyRt] java9_rt = '/home/bjornr/.sbt/0.13/java9-rt-ext-ubuntu_11_0_9_1/rt.jar'
# Executing command line:
java
-Dfile.encoding=UTF-8
-Dfile.encoding=UTF-8
-Xms1024m
-Xmx1024m
-Xss4M
-XX:ReservedCodeCacheSize=128m
-Dscala.ext.dirs=/home/bjornr/.sbt/0.13/java9-rt-ext-ubuntu_11_0_9_1
-jar
/usr/share/sbt/bin/sbt-launch.jar

[info] welcome to sbt 1.4.3 (Ubuntu Java 11.0.9.1)
[info] loading settings for project global-plugins from gpg.sbt ...
[info] loading global plugins from /home/bjornr/.sbt/1.0/plugins
[info] loading project definition from /home/bjornr/git/cs/pgk-admin/2020HT/bonus/project
[info] loading settings for project bonus from build.sbt ...
[info] set current project to bonus (in build file:/home/bjornr/git/cs/pgk-admin/2020HT/bonus/)
[info] sbt server started at local:///home/bjornr/.sbt/1.0/server/fc04812400c5fdf05c15/sock
[info] started sbt server
sbt:bonus> åäöÃÃÃ
[error] Expected ID character
[error] Not a valid command: Ã
[error] Expected project ID
[error] Expected configuration
[error] Expected ':'
[error] Expected configuration ident
[error] Expected '/'
[error] Expected key
[error] Not a valid key: Ã
[error] åäöÃ
Ã
 Ã
[error]  ^
sbt:bonus> console
[info] compiling 1 Scala source to /home/bjornr/git/cs/pgk-admin/2020HT/bonus/target/scala-2.13/classes ...
[info] Starting scala interpreter...
Welcome to Scala 2.13.3 (OpenJDK 64-Bit Server VM, Java 11.0.9.1).
Type in expressions for evaluation. Or try :help.

scala> val åäö = "ÃÃÃ"
            ^
       error: illegal character '\u00a5'
              ^
       error: illegal character '\u00a4'
                ^
       error: illegal character '\u00b6'

[eatkins]

Thanks for the report! Should be fixed by #6106.