Data: [Meta Issue] JSON-API support

[wecc]

Let's use this issue to keep track of status for the JSON-API adapter.

• [x] Basic working adapter - #2904

Improve metadata handling

• [ ] single records
• [x] hasMany (#3221)
• [ ] findAll
• [X] query

Features

• [ ] Pagination (collections, hasMany relationships)
• [ ] Filtering
• [x] Includes

Links

• [ ] Support self links for relationships (reloading, saving, deleting)
• [ ] Support self links for records (to override urlFor-methods)

Ping @kurko

[kurko]

:clap: :smile:

[btecu]

I thought the RESTAdapter was going to follow JSON AP, which currently does for the most part.
So we're going to have a new different adapter?

[Panman82]

Where is this at? I'm working on my backend API and wondering if I should follow the JSON-API spec or just follow the RESTAdapter for now. Thanks!

[IanVS]

+1, What is the status of JSON-API in Ember Data? According to jsonapi.org:

JSON API is at a third release candidate state. This means that it is not yet stable, however, libraries intended to work with 1.0 should implement this version of the specification. We may make very small tweaks, but everything is basically in place.

[seawatts]

May 21st 2015 is the release for json-api 1.0
https://twitter.com/dgeb/status/588179514817130497

[IanVS]

@seawatts That's great, but it's only about two weeks from now. Will Ember Data be ready to use with a json-api compliant API at that time? What's left to be done? This meta-issue doesn't seem to have been updated in quite some time. Does that mean no progress has been made, or the progress has just been silent?

[bmac]

There is work in progress on making Ember Data json-api compliant in this branch: https://github.com/emberjs/data/pull/2904

[kottenator]

Hi!

I'm new to Ember.js and I've found out in official documentation that ember-data supports JSON API.

I've created some mock data using JSON API v1.0 but RESTAdapter doesn't parse top-level "data" property into an array - it throws an error instead.

I use Ember.js v1.12.0

Here is my JSON API data sample:

{
    "data": [
        {
            "type": "profile",
            "id": "ffa13f788fd34751a218767689342d63",
            "attributes": {
                "first_name": "Irene",
                "last_name": "Dickson",
            },
            "links": {
                "self": "/assets/fixtures/individual-jsonapi.json"
            }
        }
    ],
    "links": {
        "self": "/assets/fixtures/individuals-jsonapi.json",
        "next": "/assets/fixtures/individuals-jsonapi-page-2.json",
        "last": "/assets/fixtures/individuals-jsonapi-page-10.json"
    }
}

Am I doing something wrong or RESTAdapter doesn't support this kind of data structure?

[WooDzu]

RESTAdapter is here to deal with simpler resources like:
http://guides.emberjs.com/v1.12.0/models/the-rest-adapter/

JSON-API adapter has just been merged (Yay!) into ember-data and should be publicly available on or after 26th June as far as I'm concerned. You might also try using the nightly version of ember-data to try it sooner.

[EmergentBehavior]

Is there any documentation on working with this? I've set it up and tweaked settings so there are no more deprecations, but none of the records loaded from my JSON-API compliant API are being loaded properly.

I was using a 1.0-spec compatible fork of @kurko's lib before and it was working fine (for loading, at least).

[EmergentBehavior]

Nevermind -- got it :)

[wwwdata]

I tried the jsonapi implementation now. Thanks a lot for the work so far! What I was noticing is that when you have a relationship with a related url but include the related entity inside the included object, the request for the related url will be made, although it is not necessary anymore because the referenced entity was already included in the first request.

So if there is a data field with id(s) for relations and the entities for these ids are included in the included field, we should avoid loading again from the url.

Is this still under work an the Improve link handling topic in your list?

Is there already a separate ticket for this? Can I maybe help with that? It should be straight forward to create a scenario for this now in the tests.

[wecc]

@wwwdata if there's both data and links:related in a relationship, data should take precedence. If it's not I'd consider it a bug. Would you mind opening a new issue for that?

[wwwdata]

I opened an issue #3380

[igorT]

there is an existing issue for that, should be easier to fix now. we just prioritized backwards compatible incompatible changes.

[SeyZ]

Any updates about "sugars"?

[davidlormor]

Per the JSON API spec: "An endpoint MAY also support an include request parameter to allow the client to customize which related resources should be returned." (i.e. /articles/1?include=comments). I've been putting the JSONAPIAdapter through its paces, and I can't seem to find a way to build this query using the built in findRecord method (i.e. this.store.findRecord('article', 1, {include: 'comments'}).

Note: I'm testing this against a backend built with Rails+jsonapi-resources. I'm able to hit the endpoints without issue via Postman, but can't seem to get an Ember client properly implemented without having to customize this adapter (which seems like it should support the full breadth of the spec...).

[wecc]

@SeyZ unfortunately not yet

@davidlormor that's what the "Sugar for includes" part is about

[SeyZ]

Ok thanks @wecc ! I'm watching this issue to be notified when updates are available :+1:

[JohnQuaresma]

Can we expect single record metadata in the next version of ember data? Having no recourse for the metadataFor deprecation message on my single record routes is frustrating.

Please let us know what to expect and when! :-) Thanks for the hard work!

[wecc]

@JohnQuaresma I understand your frustration :( per-record meta will probably be available in 2.1

[JohnQuaresma]

Ok. If it looks like it's going to slip beyond that, would be great to remove the deprecation message for now until we're at most one minor version away.

Sorry to be a pest and thanks again. Looking forward to upcoming improvements!

[kbjorklid]

Hi all,

I'm wondering about the 'sugar for includes'. It seems to me that it'd be very important to have an easy way of avoiding 1+n REST calls (for a list of things with dependencies) when just 1 would work.

[dbbk]

Has there been any progress on supporting record-level metadata?

[drewclauson]

On the Filtering Sugar - right now, parameters are sent back to a JSON API endpoint just as they are named in a Route. (i.e. customerIds is send back as a query parameter like http://api.mysite.com/?foo=bar) When I read the JSON API spec, it looks like filtering parameters are supposed to go on the filter query parameter (http://api.mysite.com?filter=...)

I had planned to do the filters like the JSON API spec for the page parameter (http://api.mysite.com?page[limit]=25&page[offset]=0), so http://api.mysite.com?filter[foo]=bar. However, I realized that I probably shouldn't assume this and then implement something that doesn't match where the adapter is going.

My question is what is this going to look like when implemented in the JSON API adapter or is that to be determined and I should just use the basic parameters (http://api.mysite.com/?foo=bar) for right now? I have full control over the API, so I can make it conform to how the adapter works.

[andrewobrien]

I'm in the same boat as @drewclauson is there any guidelines around how we should implement filtering in the interim until the full support is provided? It's also not possible to currently alias the properties to suit the JSONAPI format eg 'filter[activity]':'activity' see issue https://github.com/emberjs/ember.js/issues/11592

[drewclauson]

An interesting note - when using coalesceFindRequests: true in the JSONAPIAdapter, the callback uses the filter[id] syntax...

@andrewobrien I tried the same thing with aliasing and failed too.

[andrewobrien]

@drewclauson I hacked a workaround, I wrote a mixin to wrap the params in the model hook, before making the request, https://gist.github.com/andrewobrien/ab2cd218b1bf83ee0ffe I basically include the mixin on the route and set a const as a hash of the alias' I want eg const aliasQPs = { group_id: 'filter[group-id]'}, then I call params = this.wrapQPs(params, aliasQPs); in the model hook before making the request to the store. So far its working really well. Also means that I can easily remove it down the track when this kind of functionality is baked in or changes... There is probably a far better way to do it, but I'm still learning... and this seems to work :)

[bmac]

Hi @andrewobrien and @drewclauson. There are currently no plans to add any filtering sugar to the store.query with the JSONAPIAdapter.

In my app I use store.query('type', { filter: { activity: 'somevalue' } }) to get the JSONAPIAdapter to generate the correct URL. This may be a little extra verbose when you are always nesting properties under the filter parameter but it allows the API to compose filter and page query params plus any additional top level query params JSON API may reserve in the future.

[drewclauson]

@bmac Thanks for the example, I feel pretty stupid I didn't think of that and that is exactly what I needed.

[andrewobrien]

Hi @bmac thanks for clearing that up, a lot simpler :)

[Boubalou]

am I alone having issues trying to use the new syntax to obtain the metadata?

Previously, in 1.13, using this.get('store').metadataFor(MODEL_TYPE) would return the meta property associated, if any, returned by the JSONAPI server.

Now, in 2.1, they are telling us to use this.get('meta') where this would be a store object wrapped from either a query or a peek (see documentation for details: http://guides.emberjs.com/v2.1.0/models/handling-metadata/). Following the doc, I was unable to read my meta that was previously working flawlessly under 1.13 using metadataFor.

Is there anyone that can attest or point me in the right direction getting this to work? This is a blocker for us to get the pagination working again in our migration to 2.1.

[michaelrkn]

@bmac or @wecc see https://github.com/emberjs/data/issues/2905#issuecomment-152032438 - is this a problem with the Guides?

[wecc]

No, the guides seem correct to me :) You can access meta using .get('meta') on results from query and on relationships (belongsTo and hasMany). meta is not available on the results from peek.

Global metadata per model type is no longer available.

[Boubalou]

Totally right! Thanks for enlightening me, it now works greatly with a query() instead of a findAll() request.

Documentation is right, but the part about peekRecord misled me to think peekAll would work as well.

Thanks!

[sarus]

Tried following the guide for metadata (http://guides.emberjs.com/v2.1.0/models/handling-metadata/) but it doesn't appear to work when retrieving a model with this.store.findRecord()

Is there a workaround for this or plans to support it? (or is it supported and I'm just doing something wrong)

[vanadar]

I was looking for a way to read metadata on a single record (loaded with findRecord) and I ended up doing var meta = this.store.typeMapFor(model._internalModel.type).metadata

If someone has a better workaround to read the meta I would be interested to know about it.

[phcoliveira]

Hi there!

It is not clear to me if such feature - of extracting metada when saving a record - is going to be implemented. I would like to be able to do something like:

model() {
  return this.store.createRecord('ad').save().then(
    function(ad) {
      // use ad.get('meta');
    }
  )
}

If I cannot rely on using meta, I would have to include other resources along with the data. But would you consider a good practice to include other resources without explicitly asking for them in the URL from the request?

[henrymazza]

I want to conditionally include info on aggregated data on my models condicionally. Things like unread messages on User model, sales on last year on a Product model, and other non trivial database calculations. I want my admin to know sales figures for the product, but not my customer. Looking to JSONAPI documentation the best fit is Resources Meta. It's one way only, optional, doesn't need to be hardcoded on the model.

PS: I don't know what you guys are doing, I am fetching this info using $.getJSON() on a non-API controller and shoehorning it on each model. I also made a version with custom adapter/serializer and a SalesReport model, but then my true models starts to drown in so much report model that are read only and purpose specific. Hope you guys change your mind. Regards.

[oshchyhol]

No, the guides seem correct to me :) You can access meta using .get('meta') on results from query and on relationships (belongsTo and hasMany). meta is not available on the results from peek.

Global metadata per model type is no longer available.

@wecc
https://guides.emberjs.com/v2.4.0/models/handling-metadata/
this page shows the examples of retrieving metadata with peekRecord.

http://emberjs.com/api/data/classes/DS.RESTSerializer.html#method_extractMeta
advises to use a non-existing method store.setMetadataFor.

[sfbwalder]

@bmac, @wecc, or anyone else,
We would like to use the meta on a single record in the near future. Before we implement a workaround, could you share the status of any work in progress? We would like to build from it if possible. Thanks!

btw - using the high level meta on query is working great.

[henrymazza]

I'm using a meta: DS.attr('raw') on every record. But the thing is
challenging.

There is a difference between a set produced by one or other query. But not
between it's elements. 2 different sets may have different meta on itself,
but as elements may be shared l, the resource meta may be produced by the
first or the second. So 2 queries returning the same element may overwrite
one another.

I don't know what JSON API suggests to avoid this of how ED could implement
that.

[oswaldoacauan]

Any progress on supporting record-level metadata?

And anyone got a workarround to use for now?

[dynamiccast]

I'm also looking for per records meta. I ended up making my requests with jQuery and then push resulting data to the store.
Any better idea ?

[wecc]

Meta and links support is being worked on and there's a RFC from @pangratz available at https://github.com/emberjs/rfcs/pull/160 which will be the first step to enable a lot of these features.

[dbbk]

This issue is now 2 years old and Ember Data still does not support using self links for relationship saving. I can only assume this has been abandoned? It seems like a pretty important part of the JSON:API spec.

[matthewvalimaki]

@dbbk complete support for JSONAPI support does seem like abandoned. But take a look at https://github.com/pangratz/ember-data-meta-links-improvements as it implements some of the missing features as an addon.

[ef4]

For people interested in per-resource meta, I just published an addon that makes it possible: https://github.com/ef4/ember-resource-metadata

[escobera]

If complete support for JSONAPI is abandoned why is the project still defaulting to it?

[ef4]

It's not abandoned. Somebody who wants these features just needs to step up and implement them, or sponsor somebody else to implement them. This is open source.

I suspect the reason it hasn't happened yet is actually because the vast majority of apps are working fine already. Many many ember apps are using jsonapi every day happily, it makes no sense to say we shouldn't use it just because we haven't hit 100% of the spec yet.

[escobera]

I wasn't implying it shouldn't be used, but since it is incomplete and doesn't seem to have a developer involved in completing it probably shouldn't be kept as the default adapter.

All of the articles and issues about dealing with embedded records (which made me arrive here in the first place) should be enough evidence.

[ef4]

That is not accurate, ember-data is being very actively developed, and
jsonapi is fundamental to the entire modern ember-data architecture.

On Wed, Aug 16, 2017 at 10:59 AM Rafael Barbosa [e-mail]
wrote:

I wasn't implying it shouldn't be used, but since it is incomplete and
doesn't seem to have a developer involved it probably shouldn't be kept as
the default adapter.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/emberjs/data/issues/2905#issuecomment-322799870, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AATfMqGCJ1UcombpQeAsxMEgtAMCKQ0sks5sYwPagaJpZM4DxmdG
.

[lvjames]

As long as those other apps are working fine it's alright then. Nevermind the people still checking back two years after; clearly they're doing something wrong as this is functionality they shouldn't need. We are talking about the most official ember addon in existence and one of the biggest drives behind json-api spec. I can't for the life of me understand how you can keep a straight face when you say it's fine doing spotty coverage of the spec just cause this is the stuff core team members only ever needed in their personal projects, or that's how far they were paid to implement. And I know I'm only shooting the messenger here, so {{/rant}}

By the way, you've already implemented some much appreciated missing functionality in a separate addon. Why not a PR?

[aprescott]

I can't for the life of me understand how you can keep a straight face when you say it's fine doing spotty coverage of the spec just cause this is the stuff core team members only ever needed in their personal projects, or that's how far they were paid to implement.

I know you used {{/rant}}, but I struggle to see how this is in good faith. I don't think the maintainers see it as "fine" that there's spotty coverage. The issue exists to track the gaps, which says to me that they don't think it's fine.

Claiming the default adapter is in place "just because" the core team only ever needed a subset of the spec for personal projects is uncharitable, and pretty unreasonable. As someone totally unrelated to the project, the JSON API adapter and the spec are immediately useful even if I don't use all the corner cases of the spec. In comparison to a home-rolled half-baked media type of my own making, it's better to have something well-defined, even if the implementation doesn't cover 100% of the spec.

I understand the frustration with the implementation gaps. I share some of them, which is why I'm subscribed to this issue. But attacking the core team isn't going to get features shipped faster.

[lvjames]

I know you used {{/rant}}, but I struggle to see how this is in good faith. I don't think the maintainers see it as "fine" that there's spotty coverage. The issue exists to track the gaps, which says to me that they don't think it's fine.

I struggle to see how sitting on this issue for 2.5 years translates to "active development", and that's not turning a blind eye to the good work thrown into this addon over the years.

In comparison to a home-rolled half-baked media type of my own making, it's better to have something well-defined, even if the implementation doesn't cover 100% of the spec.

It doesn't even cover 80% of the spec, but that's not the point. Ember + Ember Data is supposed to be batteries-included, with all the tough decisions made for you so you can focus on building your ambitious web app. If you've followed this issue, how do you feel about the ugly hacks people had to resort to due to the holes in the spec, and how does that fit in with the previous sentence?

But attacking the core team isn't going to get features shipped faster.

Looking the other way doesn't exactly help ship them faster either.

If I wanted to "attack" the core team, you'd see a much lengthier response with simple points like: "please name what ember-data features have been implemented over the past year". Also, I think you failed to see my response in the context of the ongoing discussion. Yes, I'm frustrated and it reflects, but I'm not blindly accusing and would certainly never take it out on a 3rd party addon developer who works on her addon in her free time out of the kindness of her heart. Sure there's core devs doing the very same thing, but this isn't a 3rd party addon. I hate to repeat myself, but once again:

We are talking about the most official ember addon in existence and one of the biggest drives behind json-api spec.

This should be The reference implementation, not a sparse adaptation of the standard.

[ef4]

Please refrain from adding comments that add no new information. That just spams all subscribers. Everybody already agrees we want these features. If anyone wants to allocate some of their own time to working on any of them, but doesn't know where to get started, please email me and I will help you figure out where to begin.

If unproductive complaining continues I will have to lock this thread, and that would make it harder for everyone to collaborate on actually solving these problems.

If you simply _must_ register your feelings, consider adding a :-1: reaction to my comment and I promise I will feel appropriately chastened, while helpfully avoiding spam for all the other poor souls who are subscribed to this issue.

[lvjames]

@ef4 no need for that I'm done arguing. I was only expressing what I believe is a justified concern in the back of every subscriber's mind (only wish more people would overcome this herd mentality and speak up). If that was "spam" to you and everyone else, I apologize, but next time please stick to your rules and spare us the pr speak.

I would also appreciate an answer to my original question if that's not too much to ask:

By the way, you've already implemented some much appreciated missing functionality in a separate addon. Why not a PR?

[dbbk]

@wecc @ef4 there is an RFC that has been open for 16 months and ignored. How can we get this pushed through? Or is the RFC process not the right way to move forward?

[ef4]

@dbbk that RFC is incomplete and certainly doesn't address all the points in this meta issue. And it's not being ignored, its author stopped working on it. Nothing moves forward magically, it takes actual humans.

The main blocker here is people who prioritize this highly enough to implement it. Until somebody steps up and implements, people can complain here all they want and it won't matter at all. Everyone on this thread cares about this issue, but each of us is clearly prioritizing other things more highly.

Nobody needs permission to implement this.
But what if I implement and then people don't like my PR?
Then you still have the feature you wanted! Use it in your apps. Share it with other people who like the PR.
But I don't want to have to support the code.
Ah, you have just identified why this is harder than it looks. You're asking other people to do what you're not willing to do.
But I don't know how.
Neither do I, neither do any of us the first time we look. There's no shortcut.

[runspired]

I recently landed first-class support for links in all relationship situations, and did a bug-sweep for meta on relationships as well.

This RFC will additionally expose links and meta for single Records.

[uhrohraggy]

@runspired any thoughts or updates to whether this RFC is making any headway?

[mansona]

@runspired In relation to @uhrohraggy's question, I am assuming that this "Meta Issue" to improve JSON-API support now actually tracks the record links & meta RFC instead of the previously closed Links and meta improvements RFC?

If so then for people wanting to keep track of any progress or stuff that they might be able to help with should be following the RFC tracking issue, is this correct? I know it's not 100% filled out yet but I am curious where I should be looking 😂 Also I have a need to help improve the link support for pagination so I would be interested in contributing (if that actually is included in the RFC)

[runspired]

@mansona correct that this now effectively tracks the links & meta RFC, which would not on its own enable things like pagination but would at least improve access to the information needed to do so.