Couchdb: Support sub-document operations

[wohali]

@janl:

Currently a doc update needs the whole doc body sent to the server. There are some obvious performance improvements possible. For the longest time, I wanted to see if we can model sub-document operations via JSON Pointers. These would roughly allow pointing to a JSON value via a URL.

For example in this doc:

{
"_id": "123abc",
"_rev": "zyx987",
"contact": {
  "name": "",
  "address": {
    "street": "Long Street",
    "nr": 123,
    "zip": "12345"
  }
}

An update to the zip code could look like this:

curl -X POST $SERVER/db/123abc/_jsonpointer/contact/address/zip?rev=zyx987 -d '54321'

GET/DELETE accordingly. We could shortcut the _jsonpointer to just _ if we like the short magic.

JSONPointer can deal with nested objects and lists and works fairly well for this type of stuff, and it is rather simple to implement (even I could do it: https://github.com/janl/erl-jsonpointer/blob/master/src/jsonpointer.erl — This idea is literally 5 years old, it looks like, no need to use my code if there is anything better).

This is just a raw idea, and I’m happy to solve this any other way, if somebody has a good approach.

HTTP PATCH / JSON Diff

Another stab at a similar problem are HTTP PATCH with JSON Diff, but with the inherent problems of JSON normalisation, I’m leaning towards the JSONPointer variant as simpler, but I’d be open for this as well, if someone comes up with a good approach.

@davisp:

I once played with writing a JSON diff/apply tool in Python. It worked ok-ish but it was fairly gnarly and I don't think it handles normalization correctly. I'd prefer to go with the JSON Pointer approach for something like this since it specifically ignores normalization.

Though I'd tweak the suggested API somewhat and not use a new URL end point.

For update operations I'd do something like:

POST /dbname/docid
Content-Type: application/json+jsonpointersomething

[
   ["path/to/field", "new value"],
   ["contract/address/zip", 54321]
]

Which gives us the obvious ability to bundle multiple operations into a single request.

I'm not entirely sure on GET though so maybe that's silly...

@wohali:

An idea left over from 2012: partial reads without a show would be nice, too.

[willholley]

if multiple operation support is desirable, it's worth considering JSON Patch rather than inventing our own format.

[wohali]

@willholley Care to discuss the differences between JSON Patch and the ones proposed by Paul and Jan in 2017? Sounded to me like JSON Pointer was something fairly well captured in RFC6901. Is it just multiple operations?

[willholley]

@wohali JSON Patch / RFC 6902 specifies an array of operations to apply to a document. Each operation consists of an operation type (add, remove, replace, delete, move, copy, test), a path (expressed as a JSON pointer) and operation-specific parameters. The operations are applied in sequence to the specified document, e.g.:

[
  { "op": "replace", "path": "/baz", "value": "boo" },
  { "op": "add", "path": "/hello", "value": ["world"] },
  { "op": "remove", "path": "/foo" }
]

It seems like an extension of Paul's proposal, with the benefit that there is existing tooling/support as it's an established format.

[janl]

@willholley I’m glad to see JSON Patch here, it looks like a format one would have badly NIH’d on the fly if it didn’t already exist. This sounds like a way forward.

[janl]

Superseded by https://github.com/apache/couchdb/issues/1559