Rapidjson: Token-by-token parsing is unavailable

[vzaluckis]

The feature documented here: http://rapidjson.org/md_doc_sax.html#TokenByTokenParsing is not accessible, at least in the latest stable release.

I would really love to have push parsing and pull generation in order to combine RapidJSON with HTTP chunked transfer encoding. Thanks!

[miloyip]

This feature is now at the HEAD.

[mtrempoltsev]

Can you create release with this feature? It will be great!

[vzaluckis]

Hi @miloyip,

First of all, thanks for a great library! I really appreciate the efforts you and your contributors have put into it.

Now, I am trying to make use of iterative parsing in master. So what was the idea behind it anyway? It looks pretty unusable at the moment unless I add a separate tokenizer in front of the reader because whenever it spots an incomplete token it aborts the parsing altogether.

So, imagine I am loading a long JSON text from a chunked HTTP request. I can never know where the chunk boundary happens, so in order to avoid iterative parsing failure, I need to tokenize the input myself and stop feeding the reader before it gets the incomplete token. This is a burden I wouldn't like to take.

In fact, I would expect from a push parser to accept the incomplete token and store it internally, so that when the input continuation is received, it could pick up from where it left. What do you think about this idea? Would you be able to develop that, or at least would you agree to merge that if I develop it?

Thanks!

[JonMcDevitt]

Hi @miloyip,

When is this planned to be released?

~Jon