Scala-native: Improve performance of line count with .getLines

Created on 29 May 2017  路  3Comments  路  Source: scala-native/scala-native

The following code is much slower on scala-native than the JVM for large files:

import scala.io.Source

object Count {
  def main(args: Array[String]) = {
    val data = Source.fromFile(args(0))
    println(data.getLines.length)
  }
}

See the following timings for various file sizes:
(Where data_1k contains 1,000 md5 hashes, data_10k has 10,000, and so on.)

   data_1k  native  0m0.033s
  data_10k  native  0m0.227s
 data_100k  native  0m2.047s
data_1000k  native  0m20.671s

   data_1k  jvm 0m0.791s
  data_10k  jvm 0m0.755s
 data_100k  jvm 0m0.788s
data_1000k  jvm 0m0.902s

Additional instructions to reproduce results available here: https://github.com/valencik/scala-native-linecount

bug performance

Most helpful comment

0.3.0 is going to be around 15x faster at this https://github.com/scala-native/scala-native/pull/799

All 3 comments

Thanks for the report, this is most definitely a performance bug.

0.3.0 is going to be around 15x faster at this https://github.com/scala-native/scala-native/pull/799

Confirmed this has been greatly improved upon. Thanks!

Was this page helpful?
0 / 5 - 0 ratings

Related issues

frgomes picture frgomes  路  4Comments

LeeTibbert picture LeeTibbert  路  4Comments

langley picture langley  路  6Comments

densh picture densh  路  3Comments

Andrei-Pozolotin picture Andrei-Pozolotin  路  6Comments