Flatbuffers: Support JSON cross-references with IEFT Draft from 2012

[phr34k]

Flatbuffers seems have the option to use the schema definition to convert a json file and encode/convert it to the binary format. The json based representation does have a problem in representing a hierarchical schema. I would propose implementing the $REF to allow for this sort of encoding.

Consider for example this:

namespace MyGame.Sample;

struct Vec3 {
  x:float;
  y:float;
  z:float;
}

table Node {
  pos:Vec3;
  parent:Node;
  name:string;  
}

table Mesh {
  node:Node;
  strides:[Strides];
}

table Strides {
  offset:int32;
  stride:int32;
  semantic:int32;
}

table Header {
  nodes:[Node];
  meshes:[Mesh];  
}

root_type Header;

This example intends on describing a tree-like structure (as evidently) a very common case around games, 3d visualisation, but due to the shortcominigs in the json data format. It would be impossible to author a valid example json or to represent valid data into it's textual format.

To address this shortcoming I would propose implementing an IEFT draft that was issued in 2012 which describes the addition of $ref to be able to declare references. The IEFT draft can be found here https://tools.ietf.org/html/draft-pbryan-zyp-json-ref-03

If implemented an example json could be representated with the example below. The data structures when converted back and forth between binary and textual representation can maintain their relationships.

{
    nodes: [
        { 
            name: "hello world",
            pos: {x: 0, y: 0, z: 0},            
        },
        { 
            name: "lorum ipsum",
            pos: {x: 0, y: 0, z: 0},            
                parent: {"$ref": "nodes[0]" },
        },
    ],

    meshes: [
        nodes: {"$ref": "nodes[0]" },
        strides: [
            { offset: 0, stride: 0, }
        ]
    ],
}

[aardappel]

First, FlatBuffers can at best represent a DAG, never a general graph, so the concept of a "parent" is not representable if there are also "child" pointers. If there are no child pointers, then parent is not really a parent? :)

I am not sure why you say that JSON has a problem in representing hierarchical data. It does just fine in the "natural" way to represent trees:

table Node { left:Node; right:Node; value:int; }

{ left: { left: { value: 1 }, right: { value: 2 } }, right: { value: 3 } }

What you're referring to is that there is no way to have 2 references to the same tree node in JSON, which is something we could fix. Your link is a draft and still not a standard.. but are you saying this is the closest we have in JSON to a standard for references?

[phr34k]

@aardappel I'm not entirely familiar what the limitations of FlatBuffers of in terms of graph representation. I'm evaluating the technology if it can be used as a replacement library for a handcrafted serialisation/deserialisation logic which involves a graph like structure.

Please don't get me wrong, I never intended to say JSON has a problem representing hierarchical data in general. But in the binary format, you can use offsets to make cross-table references. In order to that in JSON you should share a tree node- but JSON has never standardized that and so that cannot be done. So this is what I would say is a feature disparity.

The closest we have to a JSON standard for references is that draft that I posted. It's been used in a few technologies like JSONSchema out of necessity.

[aardappel]

Well, if someone wants to try implementing such references, I think it would be a nice feature addition. @mikkelfj @vglavnyy opinions?

[mikkelfj]

I'd need to think about some more. JSON / FB does indeed have a problem with exploding references.

If we were to add such I format, I think JSON pointer would be the more obvious choice: https://tools.ietf.org/html/rfc6901

But JSON pointer is an external lookup format, not an internal one. I think JSON schema has a JSON pointer notation within the schema JSON, but I am not familiar with it. So using something like { "$ref": "<JSON-pointer>" } could make sense, but the "$ref" part probably has a lot of pitfalls. The right hand side of the earlier pbryan proposal is probably obsoleted although very similar in design.

RapidJSON supports JSON pointer towards the client API, but not in the JSON itself, as I understand. There is also relative JSON pointers: https://tools.ietf.org/id/draft-handrews-relative-json-pointer-00.html

But one also has to remember that FB is an excellent format for a general data schema, including JSON data, so every time we add something very flexible, it can cause problems, and slow down performance in various areas. For example parse JSON, or verification. It also adds security concerns.

To some extend I would sometimes go the other way and prohibit non-tree structures for some use cases. A verifier could easily detect this if the tree branches are not allowed to criss-cross. Today a verifier risks denial of service attacks if it doesn't maintain an external data structure.

I think an integer based offset is often a good solution for graphs. There is no need to necessarily use pointers in the format. They don't have vulnerabilities, and they can be stored much more space efficient. Clarification: data in one vector, indices in another integer vector, and/or edge pairs in one integer vector.

That said, I can see a use case for cycles and for DAG, of which DAG is the only one supported today.

[mikkelfj]

BTW: I know that @mzaks really wants cycles.

[aardappel]

Agree that DAGs are tricky, certainly one issue that introducing these refs would have that turning a buffer with them in it back to JSON would not automatically re-create the refs (but would produce duplicate objects) unless we did a bunch of extra work. That would have to be an option since the base case should indeed stay fast.

[mikkelfj]

I have this refmap that is an optional runtime thing that can be used when cloning Flatbuffer tables. Cloning has the problem that you need to track object reuse, but you also need to control by how much. For example you might want to clone an object and reuse shared objects. But the next clone you make should only reuse in the cloned context and not from previously clone objects.

The refmap keeps track of this and can be reset by the user to manage these concepts:

https://github.com/dvidelabs/flatcc/blob/master/src/runtime/refmap.c

Yeah, it's just a minimal hash table. It's more about the interface, but details are not important here.

https://github.com/dvidelabs/flatcc/blob/master/test/monster_test/monster_test.c#L918

[mzaks]

Given a schema the compiler can evaluate if definitions have cycles. In FlatBuffersSwift I do just that. Keeping the overhead of managing cycles only if schema has cycles.
It is much harder to do with Tree vs. DAG, but potentially if a schema is super simple and you can prove that table instance reuse is not possible than you can generate the most simple and efficient code skipping all kind of checks lookup tables etc. That sad, a schema provable to be a tree is rather rare. Most schema definitions imply potential DAG. It would be possible to introduce a flag in schema or compiler parameter which forces a tree (prohibiting pointer reuse) though.

Now to the proposal. I think it would be nice to have a human writable notation for DAG representation, however it means that the JSON parser needs to be extended with reference expression parsing. Second problem is the fact that currently (even though it is technically not necessary) the pointers need to be positive numbers, meaning that in following example:

{
    nodes: [
        { 
            name: "hello world",
            pos: {x: 0, y: 0, z: 0},            
        },
        { 
            name: "lorum ipsum",
            pos: {x: 0, y: 0, z: 0},            
                parent: {"$ref": "nodes[0]" },
        },
    ],

    meshes: {
        nodes: {"$ref": "nodes[0]" },
        strides: [
            { offset: 0, stride: 0, }
        ]
    },
}

nodes[0] needs to be added to the buffer before nodes[1] and meshes.nodes (given that the buffer is built up backwards as it is the case in flatc). Which is in current examples quite naturally, but could be much more complicated.
So internally after the JSON is parsed and a reference-expression is found there needs to be a "table append order resolution algorithm", which also could figure out if the order is non determined, because the represented graph has a cycle.

All this can be simplified if a table ref could be a negative number as well. It would be a forwards compatible change btw. In this case we would still need a lookup for table/node instances, but we would not need the "table append order resolution algorithm" and could support graphs with cycles.

[mzaks]

One more remark from my side. The proposal is targeting the use case where a JSON needs to be translated to a binary representation. Squeezing the last bit of execution performance is neglectable in this use case IMHO. It should be rather a "debugging" tool where convenience and developer experience is key.

[mikkelfj]

The problem is that a heavily optimised parser as the one generated by flatcc cannot trivially support a slow path, and users wouldn't care that some other user only needs it for a slow debugging purpose.

But, I' not sure it necessarily needs to slow things down in the parser step. The problem is different: during parse you do not have access to the generated flatbuffer so you need to record all references and resolve them later. This is doable, but it adds some complexity and even more if error reporting has to be good, which is a priority for first error seen. I already have some problems with the union type field being allowed to appear after the union value which causes the parser to back track. The performance hit is not an issue since it only happens if the type is late, but the amount of complexity it adds is non-trivial. If unions had the form: "type: { value }" or "[type, value]", that wouldn't have been a problem.

Of course that shouldn't prevent other parsers from adopting this, but then it comes at an interop cost. I'm not all against this, I just don't want to jump in head first.

[mikkelfj]

BTW: you also need a cycle checker.

[mzaks]

BTW: you also need a cycle checker.

Yes this is what I mean with __"table append order resolution algorithm"__.

A -> B
C -> B
C -> A

Is a DAG where we need to add B first than A and than C

A -> B
A -> C
C -> A

Is a cycle which you would also catch if you would need to figure out which instance needs to be encoded first. It's a chicken egg problem. Which can be solved with second pass linking.
I tried to explain how it works in FlatBuffersSwift in this article:
https://medium.com/@icex33/performance-is-not-the-only-thing-you-lose-while-using-json-d7fc788c3056

And this is how the object API looks like in Swift:
https://github.com/mzaks/FriendsPlaygroundFB/blob/master/FriendsPlayground.playground/Contents.swift

Speaking of object API, @aardappel how do other languages handle cycles in object API? Do they crash?

[mikkelfj]

I'm currently writing a response - I missed your first post. Stand by.

[mikkelfj]

@mzaks Sorry, I completely missed your first comment when writing my response, but it doesn't really change anything (and I haven't read you linked articles yet).

Yes, a static cycle checker is a clever thing, but you can easily have cycles in the schema without having them in a buffer. For example a classic binary tree: {left: tree-node, right: tree-node}, so I'm not sure how much you gain.

On the adding node[0] first: If you mean technically and not a semantic load on the user, then lets discuss that. But forward references in the schema must be allowed or the usability would suffer badly - but I think you would agree here. Unfortunately forward references has the same problem as my union type example.

Now, on technically writing the buffer front to back (StreamBuffers). I think we should adopt them at some point, but we are not quite there yet, and it doesn't solve forward references. Also, flatcc doesn't allow access to buffers before they are complete because the backend emitter is allowed to do its own thing. Therefore it wouldn't help much. I would say that a stream buffers should disallow late union types in JSON - this would allow parsed JSON to be streamed on the network.

Now, references would prevent such streaming, but you don't now that you have them before you see them. So you need to spec this up front, possibly in the schema.

So, what can be done? You can use the refmap I listed earlier and extend it with a source pointer. Then parser everything with a null reference stored in the flatbuffers. Once you are done, you visit all entries in the source map and look up the JSON pointer. If found you fill in th offset. If not found you raise an error with the exact line number and column that you find via the source pointer.

This still requires the full JSON source in memory. Currently flatcc requires that as well, but that might change in the future. If so, it suffice to buffer the JSON pointers and source row/col.

But overall, it wouldn't be very difficult to support since "$" is easy to detect. Especially if the schema requires an attribute that says "json_ptr", or something.

The verifier is already capable of detecting cycles because offsets can only be positive for the time being. This of course won't work, and we are back to what you said about stream buffers (signed offsets). I do have an algorithm that can check stream buffers if you require that a subtree must not reference memory into sibling or parent space. That is a reference can change direction but children must then be contained in a strict sub-section of the buffer.

With this algorithm and stream buffers, it can be done. You could then rewrite the buffer without negative offsets if you need to. The clone operation I already have could do this with minor modifications.

But the sum of all this indicates a major effort. So I'd say we should add this to possible future features in:

https://github.com/dvidelabs/flatcc/blob/master/doc/binary-format.md#possible-future-features

See also StreamBuffers:
https://github.com/dvidelabs/flatcc/blob/master/doc/binary-format.md#streambuffers

[mikkelfj]

@mzaks on your question about other API's being able to handle cycles: i think they are safe if they use a verifier, but most languages do not have access to one. But even without cycles you can create an exponentially large DAG, so in praxis it might not matter.

[mikkelfj]

On the title (you do have a way with titles don't you :) ) https://medium.com/@icex33/performance-is-not-the-only-thing-you-lose-while-using-json-d7fc788c3056

Keep in mind that flatcc parses JSON only 4 times slower than you can build a FlatBuffer directly in C and that number probably goes down significantly if you leave out floating point numbers. JSON also often uses less space than FlatBuffers depending on the situation and compression.

But of course you can read an existing FlatBuffer 40 times faster than you parse JSON into FlatBuffers (roughly).

[mzaks]

Will reply in-place

On the adding node[0] first: If you mean technically and not a semantic load on the user, then lets discuss that. But forward references in the schema must be allowed or the usability would suffer badly - but I think you would agree here. Unfortunately forward references has the same problem as my union type example.

Yeah I mean technically, in a human writable format the burden of internal structure should not be offloaded to the creator. The creators have enough points of failure for defining correct paths to objects 😉. Which as you mentioned before a good compiler should point out with a readable and easily actable error message.

Lets look at a more complex example:

f1.friends = [f3, f2,  f4]
f2.mother = f3
f2.father = f4
f4.friends = [f6, f5]
f6.mother = f5

f1 is our root node. Nodes can have vector of friends, and property mother and father
Now f1 has 3 friends f2, f3, f4 where f3 is mother of f2 and f4 is father of f2.
f4 - father of f2 and friend of f1 has also 2 friends f5 and f6. Where f5 is mother of f6.

It is a mess but it is actually a DAG. Given this mess we need to figure out how to add objects to the buffer so they are still pointing forwards. IMHO it is actually not important if you store data forwards or backwards. The example is a mess and needs "linearisation" anyways.

In this particular example the correct order for prepand would be:

f5 > f6 > f4 > f3 > f2 > f1

And it can be found only by traversing the graph.

Unions btw have the same complexity as table references and vectors of tables. As union can "wrap" a reference to a table already present in the buffer. In my playground I use the union type Lover to visualise the problem: https://github.com/mzaks/FriendsPlaygroundFB/blob/master/FriendsPlayground.playground/Contents.swift#L22

So, what can be done? You can use the refmap I listed earlier and extend it with a source pointer. Then parser everything with a null reference stored in the flatbuffers. Once you are done, you visit all entries in the source map and look up the JSON pointer. If found you fill in th offset. If not found you raise an error with the exact line number and column that you find via the source pointer.

This is exactly how it works in FlatBuffersSwift, the only clever thing I do is analysing the schema and if there are no recursive definitions there is no need for building up refmap and "visiting all entries in the source map"

This still requires the full JSON source in memory. Currently flatcc requires that as well, but that might change in the future. If so, it suffice to buffer the JSON pointers and source row/col.

In FlatBuffersSwift as the creation of the buffer is supported on object API, the objects are in memory anyways. The toByteArray() call just traverses the graph in depth first manner and sets table references to 0 if given table is already on the traversal path (e.g. A -> B -> A). This breaks the cycle, which is restored by what I call second linking pass. When a 0 is stored as table reference I store the table instance and position of the 0 reference . In the second pass I update the 0 reference with proper position of table instances.

But overall, it wouldn't be very difficult to support since "$" is easy to detect. Especially if the schema requires an attribute that says "json_ptr", or something.

Yeah the complexity comes rather with error handling of bad reference expressions and the linearisation of objects, because of forward pointing references.

The verifier is already capable of detecting cycles because offsets can only be positive for the time being. This of course won't work, and we are back to what you said about stream buffers (signed offsets). I do have an algorithm that can check stream buffers if you require that a subtree must not reference memory into sibling or parent space. That is a reference can change direction but children must then be contained in a strict sub-section of the buffer.

To be honest with you, I haven't checked the internals of the verifier, but I can imagine that __signed offsets__ will make its internal implementation much more complex.

[mikkelfj]

To be honest with you, I haven't checked the internals of the verifier, but I can imagine that signed offsets will make its internal implementation much more complex.

Within certain constraints it can be very efficient:

https://github.com/dvidelabs/flatcc/blob/master/doc/binary-format.md#bidirectional-buffers

Essentially treat a buffer like wooden ruler. Start at the end, or beginning depending on the direction of any offset you find in the root table. If the offset is negative, you get a saw and cut off the end of the ruler and proceed following that offset. If you find a new offset with a postive sign, you cut off the start of the rule, then follow that link. If you ever find a link outside the ruler you have left, something is wrong. Repeat until you get to a leaf node.

The chopped off ruler must be restored when processing other offsets in the same table unless you require them to all have the same offset sign, but the principle remains.

Now, you can have signed DAGs that would be invalid in the above, but by not permitting those, you get bidi buffers, and these are efficient to check for cycles.

[mikkelfj]

Something completely different:

The notation of "$ref" is not without pitfalls as I suggested earlier. What does "ref" mean. Is it just to indicate a pointer, if so, just call it "$". Is it to refer to a table field? If so, you need to handle references in vectors as well. This suggests that the pointer tag "$" should not be with the name, but with the value.

[vglavnyy]

If I right, the Flatbuffers already has almost all to describe a graph or a linked data structure.

table Vertex {
  id : ulong(key, hash:"fnv1a_64");
  refs : [ulong](hash:"fnv1a_64", cpp_type:"VertexT", cpp_ptr_type:"naked", cpp_ptr_type_get:"");
}
table VGraph {
  vertices : [Vertex](cpp_ptr_type:"std::shared_ptr");
}
"graph_in_json" : {
  "vertices" :[  {"id": "node_0", "refs" :["node_0", "node_1"]}, {"id": "node_1", "refs" :["node_0"]}  ]
}

One can use LookupByKey to traverse over a graph.
Also, I expect that @KageKirin PRs (#4643, #4615, #4613) introduced powerful tools in C++ (need time to research and write examples).

The current hash attribute has a little drawback for json with linked-data: it is impossible to get the original string ("node_0") without code or data duplication.
Maybe introduce hash_tag type as tuple?

table Vertex {
  // id()->first : ulong, id()->second : string
  id : hash_tag(key, hash:"fnv1a_64");
 // ulong values
  refs : [ulong](hash:"fnv1a_64", cpp_type:"VertexT", cpp_ptr_type:"naked", cpp_ptr_type_get:"");
}

Another way: introduce std::map type instead of vector to use strings directly as key.

table Vertex {
  id : string(key);
  refs : [string];
}
table VGraph {
 // use std::map<string, owning_pointer> for vertices
  vertices : [Vertex](map, cpp_ptr_type:"std::shared_ptr");
}

[mikkelfj]

The problem is not so much that you can do it, you want it to be a natural JSON representation of anything you might have in FlatBuffers more than having a FlatBuffer schema that can create graph.

[vglavnyy]

One more remark from my side. The proposal is targeting the use case where a JSON needs to be translated to a binary representation. Squeezing the last bit of execution performance is neglectable in this use case IMHO. It should be rather a "debugging" tool where convenience and developer experience is key.

For "debugging" better to use JS or Python.
You can dereference (expand json) with $ref into a Python dict and save it as a new json.
It is easy to write own two-pass JSON expander using Python (I have written).
Or you can use this: jsonref python package.

After that, you can load dereferenced JSON into flatc to generate binary FB file.

[phr34k]

For "debugging" better to use JS or Python.
You can dereference (expand json) with $ref into a Python dict and save it as a new json.
It is easy to write own two-pass JSON expander using Python (I have written).
Or you can use this: jsonref python package.

After that, you can load dereferenced JSON into flatc to generate binary FB file.

If you dereference the json and then give it to flatc you are not solving the fundamental issue @vglavnyy which is to preserve the graph by itself. For reasons, there's a use-case of shared objects, something that exists once on disk and in memory, and needs to be serialized/deserialized as such.

In most cases, I need the performance of flat buffers binary format, and more importantly the automatically generated verifiers but for debugging sometimes we need to be able to pull it back into JSON/text and verify our graph was produced correctly and to feature experiment with a graph.

[mzaks]

In most cases, I need the performance of flat buffers binary format, and more importantly the automatically generated verifiers but for debugging sometimes we need to be able to pull it back into JSON/text and verify our graph was produced correctly and to feature experiment with a graph.

What you mention is yet another feature. Transforming FlatBuffers binary to JSON without unfolding shared references. I think this would be actually easier to implement, as I can't see any hard edge cases (yet). I actually was thinking about FlatBuffers binary to DOT which could be a very useful tool to see all the reuse or lack of it in the binary (e.g. reuse of vTables per table instance, reuse of strings, etc..). DOT files can be visualised with GraphViz. From debugging perspective and figuring out if the size of the binary is the smallest it could be, it probably would be the best solution.
However as I sad it is a different use case: make binary human readable, not make binary human writable. With JSON ref it could be possible to achieve both, but with substantial cost.

Regarding using another language for debugging. I think FlatBuffersSwift is actually the most user friendly implementation, as you can build up buffers with a very ergonomic object API and you can also materialise a buffer into an object graph eagerly, so you can use an actual debugger to explore the values in the buffer.
However it means that you need to install Swift and an IDE which supports Swift and to be safe, run the binary through flatc validator. But I guess you will need it to debug / play around anyways so should not be that bigger problem. Also I haven't tried FlatBuffersSwift on Linux yet, potentially it should work 😉.

If you are interested how the object API works in FlatBuffersSwift have a look at the unit tests:
https://github.com/mzaks/FlatBuffersSwift/blob/master/FlatBuffersSwiftTests/FriendsTest.swift
https://github.com/mzaks/FlatBuffersSwift/blob/master/FlatBuffersSwiftTests/ContactsTest.swift#L327

[mikkelfj]

OK, I think that values in the form { "$ref": "<json-pointer>" } can be supported. It is a value, so it will work in vectors, and the RFC draft document does mention JSON-Pointer, although an earlier draft version.

There is an alternative option where you can add anchors, sort of what YAML does. I actually worked on a design around that at some point. However, adding anchors later does not preclude using "$ref" earlier, and preprocessor could translate one into the other, for example as a Python tool. Printing would happen without anchors.

The big question is the resulting signed offsets. This will require a buffer rewrite. The result without a rewrite will not be a bidi buffer that can be verified efficiently because forward references can break out of subtrees even if the buffer is logically a DAG, and of course cycles are easy to create which can't be fixed.

Concerning performance, a fast JSON parser will need more complexity in cases where it expects a '[' instead of '{' for a reference but this is doable. Rewriting the buffer subsequently requires additional generated code similar to the clone operation, and this is actually the largest cost and I'm not sure it is worthwhile. Hence I wonder if we should postpone this until we have a better grasp on bi-directional buffers.

In conclusion, I believe the idea is good and design is sound, but there are too many complexities to take on right now.

[phr34k]

What you mention is yet another feature. Transforming FlatBuffers binary to JSON without unfolding shared references.

From the start, I stated in the issue posting that the intention was to solve a feature disparity. To clarify I am not interested in "visualizing" the graph. I am interested in decoding a binary graph into a textual format, make some modifications and then encode it back into binary representations.

The data structures when converted back and forth between binary and textual representation can maintain their relationships.

[mikkelfj]

If you disallow forward references in the "$ref" syntax, you can print existing buffers and parse them back without problems with signs. That at least will achieve feature parity even if it is not maximally user-friendly. References can still be invalid if they reference a parent but that would never be allowed due to cycles.

[mikkelfj]

Of course, disallowing forward references would not be respected by arbitrary JSON processors that are free to sort fields as they like.

[mikkelfj]

What you mention is yet another feature. Transforming FlatBuffers binary to JSON without unfolding shared references. I think this would be actually easier to implement, as I can't see any hard edge cases (yet).

The problem is that you don't know that you have a shared reference when you print until you discover it, so you need to keep track of all offsets printed with the corresponding JSON-pointer value. This can of course be represented more efficiently than a text string, but it is a completely different operation from normal printing, and it will be orders of magnitudes slower.

You also don't know which among shared references represent that path that the user intended.

[phr34k]

The problem is that you don't know that you have a shared reference when you print until you discover it, so you need to keep track of all offsets printed with the corresponding JSON-pointer value. ...

You also don't know which among shared references represent that path that the user intended.

If my understanding of flat buffers is correct then in the binary format a shared object is represented with an offset. Wouldn't that mean when you detect the use of an offset you already know it can be substituted/simplified to a reference or pointer?

Of course, reconstructing the human-friendly path from an offset could be slow because having just an offset doesn't really say anything how it was 'lexically' scoped. But you do have the schema- and flatbuffers can already recreate the scoping of the data using that.

In the case where you already encountered the data already, resolving the friendly path could be a matter of a lookup. For other situations, you could have a slower path with an extra pass. But I don't think you need to switch to the slow path until the first occurrence that says so.

[mikkelfj]

Well, when you encounter an offset, you have a potential shared target, but you don't know. This is why you have to record all offsets in case one might be referenced later. This is not unsolvable, just slow and memory hungry.

An alternative solution is to do multiple passes. The first pass hashes all offsets and records which offsets are shared. A second pass keeps track of the current path to root and checks each offset seen. If an offset is a shared object, the path is materialised by adding the path to a buffer and storing a pointer in the hash table of shared objects. The next time the same offset is seen, the path is not materialised again, instead a reference is printed.

This can be done reasonably efficiently since the first pass is fast even if all offsets must be hashed.

[mikkelfj]

Since DAG's are a potential DoS attack vector on printing JSON, it can be argued that the mentioned first pass ought to be done in any case. The current flatcc mitigation is to print to a buffer with a maximum size.

[aardappel]

Note I would have no interest to introduce cycles as part of JSON reference parsing. Cycles is a much more invasive feature, that, if people care for it, should be discussed seperately.

[github-actions[bot]]

This issue is stale because it has been open 6 months with no activity. Please comment or this will be closed in 14 days.