Json-api: Standardized decent-sized examples / test cases

[beauby]

Having

1. a set of resource definitions for some (simplified) real-world service like Twitter, a Blog, or something;
2. a sample dataset (in a format to be defined);
3. a set of queries on those and their expected reply per the spec;
   could
4. serve as some common end-to-end testing of various server implementations;
5. show potential users the level of support/various opinions of each implementation;
6. show potential users how the definition of those resources could be done with each implementation (i.e. it could lead to some standardized docs - less hassle for potential users).

Thoughts?

Ref #733.

[dgeb]

This sounds very good to me. We could request that implementations use this data in test suites to verify compliance, and in return we could highlight "verified" implementations.

[beauby]

Since this seems to be getting _tremendous_ traction, let's move forward and chose an actual known service to model. It should:

• obviously, contain at the very least 3 resources;
• preferably, contain a self-referential resource;
• preferably, contain a resource with a possibly empty to-one relationship, and an other with an empty to-many relationship;
• some other stuff, let's add some ideas.

What comes to mind?

[brainwipe]

I would absolutely implement these in my .NET Core nuget package implementation. I've implemented Post/Article/Author but it's not quite enough for a good range of tests.

[beauby]

Here is a suggestion for the resources:

• User
  
  
  
  • attributes:
  
  
  • name
  
  
  • email
  
  
  • relationships:
  
  
  • tweets
  
  
  • friends
  
  
• Tweet
  
  
  
  • attributes:
  
  
  • content
  
  
  • date
  
  
  • relationships:
  
  
  • author
  
  
  • original_author // If this is a retweet, possibly null
  
  
• Message
  
  
  
  • attributes:
  
  
  • content
  
  
  • date
  
  
  • relationships:
  
  
  • sender
  
  
  • receiver
  
  

[richmolj]

I really like the premise of this issue. I took a whack at implementing @beauby's suggestion using a project I just open-sourced, jsonapi-suite, as I needed a sample application anyway. The sample rails app implementing users/tweets/messages is https://github.com/jsonapi-suite/jsonapi_twitter, the integration specs are here. There's read-only for everything, tweets get full CRUD.

One minor change @beauby already agreed with was to change original_author to parent.

[beauby]

Thanks a lot @richmolj - that is indeed pretty cool! I'm wondering whether it would make sense to also make a non-rails version where models are simple objects, and a set of tests to ensure behavior (probably the same test parameters as for the rails app). This would allow non-rails user to see which libraries they can use and how, as well as allowing more specific benchmarks without the rails overhead.

[richmolj]

I support everyone supplying their non-rails or differing implementations. The biggest thing that we can do to get there, I think, would to be publishing raw json examples on the jsonapi.org website (in a separate section from the current documentation). Even in my Rails example I made a bunch of assumptions about what links should be present, etc.

[beauby]

@richmolj Yeah, I think the next step would indeed be to provide:

1. the various resources in a generic format (list of attributes, relationships, links, meta, etc. – probably only attributes and relationships at first),
2. for every scenario (create/update/delete/index/show), the necessary information (http verb, url, initial data set, headers, payload, and expected response).

The format for 2. would ideally be easily parsed, so JSON seems like the best option. The format for 1. would probably be textual as it is mainly some meta information about resources (how meta/links/etc. are generated).

Proposal:
For 2.

{
  "id" : "1",
  "description" : "Create resource",
  "resources" : {
    "users" : [ ... ],
    "tweets" : [ ... ],
  },
  "request" : {
    "method" : "POST",
    "uri" : "/users",
    "headers" : {
      "Content-Type" : "application/vnd.api+json",
      "Accept" : "application/vnd.api+json"
    },
    "body" : {
      "data" : {
        "type" : "users",
        "attributes" : { ... },
        "relationships" : { ... }
      }
    }
  },
  "response" : {
    "status" : 200,
    "headers" : { ... },
    "body" : { ... }
  }
}

This format is cool because a trivial script could run those specs.

[beauby]

I went ahead and created a PR. Let's discuss the format first, and once we agree on it, let's start adding specs.

[beauby]

The idea would be that each implementation provides a way to

1. read the spec files, and for each spec, set up the resources,
2. execute the http query, and save headers, status and body (along with spec id) in a JSON file.

Then, we would make a script that takes the specs and the outputs and grades the implementation based on that (which would allow not to over-penalize implementations that do not handle links or some meta or top level jsonapi objects for instance). The results could then be displayed alongside each server implementation on the website.

[evolve2k]

I think there is a lot of benefit to having a scheme that includes a parent entity that has children and then those children have grandchildren. It covers many edge cases for more complex apps that need functionality beyond the simple parent/child relationship.

I'm proposing we use 'Author > Books > Reviews' as the basis of an example app, I think there is also a lot of benefit from having all resources in the domain based on as concrete concepts as possible to just make things easy to grok.

Would you be adverse to 'Author > Books > Reviews' as the basis of an example app @beauby as opposed to your twitter example? Thanks for moving this forward by the way.

[beauby]

@evolve2k Sure – the idea would be to create a few standardised scenario of request/response cycles.

[beauby]

Closing in favor of #1209.