Jsonnet: C++ version is much slower than the Go version

[braskey]

I evaluate the same snippet on my laptop, the C++ version cost 13.3 ms, while the Go version cost only 0.18 ms, Is this result reasonable? and is there any way to speed up the C++ process?

The test code shows as below

#include <iostream>
#include <chrono>
#include <libjsonnet++.h>

using namespace std;

#define TIMES 100

int main (int argc, char *argv[])
{
  struct JsonnetVm* vm = jsonnet_make();
  int error = 0;
  char *output = NULL;

  std::chrono::steady_clock::time_point begin = std::chrono::steady_clock::now();

  for (int i = 0; i < TIMES; i++) {
    output = jsonnet_evaluate_snippet(vm, "snippet", "{\"object\":[{\"attributes\":[{\"desc\":\"ocr\",\"key\":\"ocr\",\"ocr\":\"abcdefg\"}]}]}{'data':{'object':$.object}}", &error);
    if (error) {
      jsonnet_realloc(vm, output, 0);
      jsonnet_destroy(vm);
      return EXIT_FAILURE;
    }

    jsonnet_realloc(vm, output, 0);
  }

  std::chrono::steady_clock::time_point end= std::chrono::steady_clock::now();

  std::cout << "time spent: " << std::chrono::duration_cast<std::chrono::nanoseconds>(end - begin).count() / TIMES <<std::endl;

  jsonnet_destroy(vm);

  return 0;
}

package main

import (
    "fmt"
    "github.com/google/go-jsonnet"
    "time"
)

func main() {
    begin := time.Now().UnixNano()
  vm := jsonnet.MakeVM()
    for i := 0; i < 1e3; i++ {
        _, err := vm.EvaluateSnippet("~/jsonnet.error", "{\"object\":[{\"attributes\":[{\"desc\":\"ocr\",\"key\":\"ocr\",\"ocr\":\"abcdefg\"}]}]}{'data':{'object':$.object}}")
        if err != nil {
            fmt.Printf("error\n", err)
        }
    }
    end := time.Now().UnixNano()
    fmt.Printf("time spent: %v\n", (end-begin)/1e3)

}

[sbarzowski]

The Go implementation is newer and it is expected to be faster, way faster in some cases. The efforts are focused on Go (for C++ we keep 100% compatibility as far as the language changes go, but it may not be as fast sometimes - https://github.com/google/go-jsonnet/pull/146#issuecomment-350943812).

In your case my guess would be that it has something to do with loading of stdlib. In cpp-jsonnet I think it's parsed every time jsonnet_evaluate_snippet is ran (see desugarFile function in desugarer.cpp) and in go-jsonnet stdlib AST is built once at build time. It's a constant overhead for every jsonnet file in cpp-jsonnet.

[braskey]

@sbarzowski Thank you for your reply. Is there any way to load stdlib in cpp only for one time just as in go-jsonnet? or if we have any plan to do so ?

[sbarzowski]

First thing is that C bindings for Go version could be prepared (stubs are already there). This also brings performance improvements in some other, IMO more significant places. If such bindings were available would you be interested in being an early adopter?

Another option would be to cache the ast for stdlib here: https://github.com/google/jsonnet/blob/master/core/desugarer.cpp#L913. It's going to be a bit tricky, because there's one field that is actually different for each file in which it is loaded https://github.com/google/jsonnet/blob/master/core/desugarer.cpp#L939. So it would need to be cloned. The api in libjsonnet.h cannot change, so the cache must use a global variable, but it's ok because stdlib AST is a well-defined thing which won't change. @sparkprime Do you think that's a good idea? Is there something else that could break (modification to the AST at runtime)? @braskey
It's not a priority from the project perspective currently, but a PR would be welcome.

@braskey BTW do you run Jsonnet on thousands of files at a time? Or do you just import a lot of stuff? The overhead <0.13ms per file, so I wonder what's the setup in which it matters that much. This may give us a hint what other optimizations may be important in real world conditions.

[sparkprime]

I think the best option is to do a C wrapper for the Go library so we can move the Python etc users over.

[braskey]

@sbarzowski
What I am doing is processing videos and getting a result which is represented in json, but the result format that I get is different from what the users want, so we use jsonnet as a template to generate the final result. The frame rate of the videos is 25 frame per second, so 13ms per file is quite high cost.

If the binding were available, I'd like to be an early adopter.

[abourget]

I'm also interested in high throughput jsonnet processing, as I found jsonnet to be the best tool out there to do JSON reshaping of streaming objects. Heavy Go user here.

[prune998]

maybe try with the sJsonnet implementation in... Scala :) They say they boosted some part of the Jsonnet std lib to make huge speed improvements :

https://github.com/databricks/sjsonnet#sjsonnet

[sparkprime]

We're not going to implement optimizations in the C++ version, in fact we're trying to get to a place where the Go implementation can replace it.