Scalapb: TypeError: a bytes-like object is required, not 'str'

[moglideveloper]

Got a question? Please use https://groups.google.com/forum/#!forum/scalapb instead.

Getting below error, while generating scala class for search.proto file :-

TypeError: a bytes-like object is required, not 'str'
--scala_out: protoc-gen-scala: Plugin failed with status code 1.

Python version used is 3.6 and scalapbc version is (libprotoc 3.1.0)

search.proto

syntax = "proto3";

message SearchRequest {
  string query = 1;
    int32 page_number = 2;
      int32 result_per_page = 3;
      }

Please find below the executed console commands with error :-

C:\Users\vermari\Desktop\sandbox\GpEx\src\main\protobuf>cat search.proto
syntax = "proto3";

message SearchRequest {
  string query = 1;
    int32 page_number = 2;
      int32 result_per_page = 3;
      }

C:\Users\vermari\Desktop\sandbox\GpEx\src\main\protobuf>python --version
Python 3.6.0

C:\Users\vermari\Desktop\sandbox\GpEx\src\main\protobuf>scalapbc --version
protoc-jar: protoc version: 310, detected platform: windows 7/amd64
protoc-jar: executing: [C:\Users\vermari\AppData\Local\Temp\protocjar4183113281495272352\bin\protoc.exe, --plugin=protoc-gen-scala=C:\Users\vermari\AppData\Local\Temp\protocbridge4600966888313403794.bat, --version]
libprotoc 3.1.0

C:\Users\vermari\Desktop\sandbox\GpEx\src\main\protobuf>scalapbc search.proto --scala_out=.
protoc-jar: protoc version: 310, detected platform: windows 7/amd64
protoc-jar: executing: [C:\Users\vermari\AppData\Local\Temp\protocjar3585376835034754213\bin\protoc.exe, --plugin=protoc-gen-scala=C:\Users\vermari\AppData\Local\Temp\protocbridge4548162299190212618.bat, search.proto, --scala_out, .]
Traceback (most recent call last):
  File "C:\Users\vermari\AppData\Local\Temp\protocbridge2129919967984048872.py", line 6, in <module>
    s.sendall(content)
TypeError: a bytes-like object is required, not 'str'
--scala_out: protoc-gen-scala: Plugin failed with status code 1.

[xuwei-k]

Maybe scalapb require python 2.x

[thesamet]

Right, that looks like the issue. Please have Python 2.x installed. When using SBT, set the full path to the Python 2.x executable like this:

PB.pythonExe := "C:\\...\Python.exe"

[thesamet]

Closing due to inactivity - please let me know if your issue is not resolved.

[DieBauer]

I can confirm, Python 2.7 works, while Python 3.6 doesn't.

[thesamet]

Re-opening. I'd like to make it work with both Py2 and Py3.

[thesamet]

The code generation phase no longer relies on Python, so this issue can be resolved by updating sbt-protoc to 0.99.15.

Thanks @augi for removing the dependency on Python.

[ajanaliz]

nope, still gave me the following error:

Error: Could not find or load main class protocbridge.frontend.BridgeApp
--scala_out: protoc-gen-scala: Plugin failed with status code 1.
[trace] Stack trace suppressed: run last compile:protocGenerate for the full output.
[error] (compile:protocGenerate) protoc returned exit code: 1
[error] Total time: 6 s, completed Feb 20, 2018 4:01:45 PM

here is a copy of my build.sbt:

val namespace = settingKey[String]("namespace")

val export = """^([^=]+)="?(.*?)"?$""".r

lazy val root = project
  .in(file("."))
  .settings(
    name := "APP-Backend",
    version :=
      scala.io.Source.fromFile(file(".env"))
        .getLines()
        .map(_.trim)
        .flatMap {
          case export(key, value) =>
            Some((key, value))
          case _ => None
        }
        .toMap.get("VERSION").get.toString,
    scalaVersion := "2.12.4",
    organization := "com.app",
    namespace := sys.props.getOrElse("namespace", default = "alij"),
    mainClass in Compile := Some("com.app.AkkaHttpService"),
    libraryDependencies ++= {
      val akkaV = "2.5.8"
      val akkaHttpV = "10.0.11"
      val scalaTestV = "3.0.4"

      Seq(
        "com.typesafe.akka" %% "akka-actor" % akkaV,
        "com.typesafe.akka" %% "akka-stream" % akkaV,
        "com.typesafe.akka" %% "akka-testkit" % akkaV,
        "com.typesafe.akka" %% "akka-http" % akkaHttpV,
        "com.typesafe.akka" %% "akka-http-spray-json" % akkaHttpV,
        "com.typesafe.akka" %% "akka-http-testkit" % akkaHttpV,
        "com.typesafe" % "config" % "1.3.1",
        "org.scalatest" %% "scalatest" % scalaTestV % "test",
        "com.typesafe.scala-logging" %% "scala-logging" % "3.7.2",
        "ch.qos.logback" % "logback-classic" % "1.2.3",
        "com.trueaccord.scalapb" %% "scalapb-runtime" % com.trueaccord.scalapb.compiler.Version.scalapbVersion % "protobuf",
        "io.grpc" % "grpc-netty" % com.trueaccord.scalapb.compiler.Version.grpcJavaVersion,
        "com.trueaccord.scalapb" %% "scalapb-runtime-grpc" % com.trueaccord.scalapb.compiler.Version.scalapbVersion,
        "org.elasticsearch.client" % "transport" % "6.2.1",
        "joda-time" % "joda-time" % "2.9.9",
        "ch.megard" %% "akka-http-cors" % "0.2.2"
      )
    },
    dockerRepository := Some(s"registry.gitlab.com/${namespace.value}/app-backend/image"),
    PB.targets in Compile := Seq(
      scalapb.gen() -> (sourceManaged in Compile).value
    )
  )
  .enablePlugins(JavaAppPackaging)
//.enablePlugins(DockerPlugin)

[thesamet]

@ajanaliz , if you are on ScalaPB 0.6.x please try the following (it's not necessary with 0.7.x):

addSbtPlugin("com.thesamet" % "sbt-protoc" % "0.99.15") 

libraryDependencies += "com.trueaccord.scalapb" %% "compilerplugin" % "0.6.7" exclude ("com.trueaccord.scalapb", "protoc-bridge_2.10"))

If you are on SBT>=1.0, the last line should have protoc-bridge_2.12 instead of 2.10.

If that doesn't work, can you give me everything you have under project/*.sbt, or maybe build a minimal example and post it here?

[augi]

@thesamet Is this issue caused by my PR ? Can I help with it?

If the generated bat file contains something fishy, then it would be nice to see content of that file (it's something like C:\Users\user_name\AppData\Local\Temp\protocbridge7875021896390530245.bat).

[thesamet]

My guess is that @ajanaliz is getting two copies of protoc-bridge, one from com.thesamet.scalapb and one from com.trueaccord because ScalaPB 0.6.7 is involved. Waiting for @ajanaliz to report.

[augi]

OK. I've tried the latest version of plugin in another project on different Windows computer and it was working as well.

[thesamet]

We have appveyor testing on Windows every commit so it is most likely a combination of versions that is incompatible. https://ci.appveyor.com/project/thesamet/sbt-protoc/branch/master

[filfreire]

Hi there, I can confirm this is happening for

Maven 3 + scala 2.11.8 + Windows 10
protoc version: 320, detected platform: windows 10/amd64 with Python 3.6.4

I don't have all the data, since this wasn't on my machine, but on a colleague's machine (I'm not using Windows)

Here is a full error log:

protoc-jar: executing: [C:\Users\username\AppData\Local\Temp\protocjar8731229890450376134\bin\protoc.exe, --plugin=protoc-gen-scala=C:\Users\username\AppData\Local\Temp\protocbridge6372432427916353559.bat, --proto_path=C:\IntelliJ_Projects\project\project-model\target/proto, --scala_out=single_line_to_string:C:\IntelliJ_Projects\project\project-model\target/generated-sources/protobuf, C:\IntelliJ_Projects\project\project-model\target/proto/src/main/proto/context2.proto, C:\IntelliJ_Projects\project\project-model\target/proto/src/main/proto/instruction2.proto]
Traceback (most recent call last):
  File "C:\Users\username\AppData\Local\Temp\protocbridge6409608682069464369.py", line 6, in <module>
    s.sendall(content)
TypeError: a bytes-like object is required, not 'str'
--scala_out: protoc-gen-scala: Plugin failed with status code 1.
[WARNING] 
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:294)
    at java.lang.Thread.run(Thread.java:748)
Caused by: com.trueaccord.scalapb.ScalaPbcException: Exit with code 1
    at com.trueaccord.scalapb.ScalaPBC$.main(ScalaPBC.scala:41)
    at com.trueaccord.scalapb.ScalaPBC.main(ScalaPBC.scala)
    ... 6 more
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE

/cc @thesamet

[thesamet]

@filfreire this shouldn't happen once you upgrade to latest version of sbt-protoc, the dependency on Python has been removed.

[ajanaliz]

apparently, it does @thesamet, I'm having the same issue on my own system which has Scala version 2.12.4 and protoc version 340 and Windows 10.

protoc-jar: protoc version: 340, detected platform: windows 10/amd64
protoc-jar: executing: [C:\Users\ALIJ~1\AppData\Local\Temp\protocjar8861891845646122596\bin\protoc.exe, --plugin=protoc-gen-scala=C:\Users\ALIJ~1\AppData\Local\Temp\protocbridge5305452030468080702.bat, --scala_out=grpc:C:\Users\Ali J\Desktop\Miras\29\imm-backend\target\scala-2.12\src_managed\main, -IC:\Users\Ali J\Desktop\Miras\29\imm-backend\src\main\protobuf, -IC:\Users\Ali J\Desktop\Miras\29\imm-backend\target\protobuf_external, C:\Users\Ali J\Desktop\Miras\29\imm-backend\src\main\protobuf\crawler.proto, C:\Users\Ali J\Desktop\Miras\29\imm-backend\src\main\protobuf\zaal.proto]
Traceback (most recent call last):
  File "C:\Users\ALIJ~1\AppData\Local\Temp\protocbridge3431838731623202643.py", line 6, in <module>
    s.sendall(content)
TypeError: a bytes-like object is required, not 'str'
--scala_out: protoc-gen-scala: Plugin failed with status code 1.
[trace] Stack trace suppressed: run last compile:protocGenerate for the full output.
[error] (compile:protocGenerate) protoc returned exit code: 1
[error] Total time: 5 s, completed Mar 29, 2018 5:25:45 PM

[thesamet]

@ajanaliz looks like you're on an old version of sbt-protoc and/or ScalaPB, can you update sbt-protoc to 0.99.18 and ScalaPB to 0.7.0? If this doesn't help, can you put together a minimal example project that demonstrates this problem and post it here (zip file or a repo)?

[ajanaliz]

I get the following error while trying to update my scalaPB from 0.6.6 to 0.7.0:

sbt.ResolveException: unresolved dependency: com.trueaccord.scalapb#compilerplugin_2.10;0.7.0: not found
        at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:313)
        at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:191)
        at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:168)
        at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156)
        at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156)
        at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:133)
        at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57)
        at sbt.IvySbt$$anon$4.call(Ivy.scala:65)
        at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:95)
        at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:80)
        at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:99)
        at xsbt.boot.Using$.withResource(Using.scala:10)
        at xsbt.boot.Using$.apply(Using.scala:9)
        at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:60)
        at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:50)
        at xsbt.boot.Locks$.apply0(Locks.scala:31)
        at xsbt.boot.Locks$.apply(Locks.scala:28)
        at sbt.IvySbt.withDefaultLogger(Ivy.scala:65)
        at sbt.IvySbt.withIvy(Ivy.scala:128)
        at sbt.IvySbt.withIvy(Ivy.scala:125)
        at sbt.IvySbt$Module.withModule(Ivy.scala:156)
        at sbt.IvyActions$.updateEither(IvyActions.scala:168)
        at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1488)
        at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1484)
        at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$121.apply(Defaults.scala:1519)
        at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$121.apply(Defaults.scala:1517)
        at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:37)
        at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1522)
        at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1516)
        at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:60)
        at sbt.Classpaths$.cachedUpdate(Defaults.scala:1539)
        at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1466)
        at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1418)
        at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
        at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
        at sbt.std.Transform$$anon$4.work(System.scala:63)
        at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
        at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
        at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
        at sbt.Execute.work(Execute.scala:237)
        at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
        at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
        at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
        at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
[error] (*:update) sbt.ResolveException: unresolved dependency: com.trueaccord.scalapb#compilerplugin_2.10;0.7.0: not found
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore?

[thesamet]

@ajanaliz package names have changed. please check out the migration instructions here https://scalapb.github.io/migrating.html

This seems to be also the case for @filfreire since com.trueaccord is mentioned in the exception log.

[ajanaliz]

thanks for the help, you were right. my issue is fixed after migrating to scalapb 0.7.1