Data: Store#push doesn't normalize data

[alexspeller]

When using the store push method, the data is expected to be already normalized. This is weird because the data is presumably sent from a server that has the data in the same format that a normal rest endpoint would have. Wouldn't it make sense to normalize the data coming into the system in the same way that it would be if loaded via ajax?

Happy to work on a PR if this is agreed to be correct.

[wycats]

The idea is that you would normalize the data yourself in your adapter's normalize or extract* hooks. The push method expects already normalized data to simplify that part of the flow.

[alexspeller]

The use case is for pushing data via websockets, which normally wouldn't have a seperate adapter - you have a REST adapter that you use when explicitly loading via store.find etc, and then as more data streams in you load it in addition.

Perhaps there should be another method that allows pushing in non-normalized data, in case anyone searching finds this issue the workaround I am using is:

App.Store = DS.Store.extend
  pushData: (type, data) ->
    serializer  = @serializerFor(type)
    model       = @modelFor(type)
    serializer.normalizeAttributes(model, data)
    @push type, data

myPushClient.onMessage (data) ->
  store.pushData 'model', data

[alexspeller]

Also note that my workaround only covers normalizing attributes - if you actually need to do any more normalizing than that you will need to do more work, I'm not sure exactly what but reading the source code of the rest serializer and the json serializer is a good starting point.

[elucid]

I am using a similar workaround in my apps although my use case is for data preloading.

Ajax responses requiring custom deserialization go through the adapter so they are properly transformed before loading. Preloads (which are generated by the same server) are not.

push and pushMany don't necessarily have to change to also normalize data. However I think there is a case to be made for providing a public API available for out-of-adapter normalization.

[bradleypriest]

This is what my current version looks like, normalizing full JSON payload

App.Store = DS.Store.extend  
  pushData: (prop, data) ->
    type       = @modelFor(prop)
    serializer = @serializerFor(type)
    if Ember.typeOf(data) is "array"
      data = data.map (object) -> serializer.normalize(type, prop, object)
      @pushMany(type, data)
    else
      data = serializer.normalize(type, prop, data)
      @push(type, data)

[nragaz]

Here is an alternative version for parsing server responses in an AMS-style format, e.g. a hash that may also include sideloaded association keys:

# Normalize the provided model data and push it into the store.
load: (store, primaryType, data) ->
  type = store.modelFor(primaryType)
  serializer = store.serializerFor(primaryType)
  id = data[primaryType].id

  store.push type, serializer.extract(store, type, data, id, 'find')

# Normalize a hash with a data array and associations and push it into 
# the store.
loadMany: (store, primaryType, data) ->
  type = store.modelFor(primaryType)
  serializer = store.serializerFor(primaryType)

  store.pushMany type, serializer.extractArray(store, type, data)

[brandonparsons]

@nragaz How exactly do you use that? In an initializer? In store.coffee somewhere?

[lukemelia]

@brandonparsons and others coming upon this issue: You'll probably want to check out pushPayload

[brandonparsons]

@lukemelia Yeah - thanks! I literally _just_ found that. I'm having trouble using it though... Poking around in the console, the function appears to want both a type and a payload. However I'm confused why it needs a type if you are actually able to give it data like this:

{
      "posts": [{
        "id": "1",
        "title": "Rails is omakase",
        "author", "1",
        "comments": [ "1" ]
      }],
      "comments": [{
        "id": "1",
        "body": "FIRST
      }],
      "users": [{
        "id": "1",
        "name": "@d2h"
      }]
    }

What would be the type in this case??? I would have only expected to have to pass a payload into the pushPayload method, and it would pull out the various types from the top-level keys.

[dgeb]

@brandonparsons You can't push a completely arbitrary payload with pushPayload. The type should be the primary document type for the payload, which looks to be posts in your example. The type determines the serializer used for deserialization. Furthermore, other documents will be processed based on their relationships to the primary document.

[brandonparsons]

@dgeb Thanks - I'm starting to get to this realization as I continue to bash my head against the wall here. How would ember-data know that posts is the "primary document type" here? If you look at the JSON, they are all the same really.....

I'm basically trying to put some JSON into my HTML document to be preloaded with the ember application. This JSON data would include:

• Current user info (name/email)
• Model data associated with the user. user _has_one_ questionnaire .

How do I embed the questionnaire JSON such that it gets picked up in the ember-data store, along with the appropriate associations being set up? I'm having one hell of a time getting the association to load. There seems to be outstanding confusion in ember/ember-data with singular resources that isn't making my life any easier.

[alexspeller]

Ember data doesn't know - that's the point, that's why you have to tell it what the primary document type is.

[brandonparsons]

@dgeb @alexspeller I've tried using plain-old activemodel serializers to dump my user data (along with all other associations) to the JSON embedded in the HTML, and then done a pushPayload('user', myJSON) . It does not appear to be camelizing the JSON data on serialization, or picking up any of the associations. Any tips?

{
  "questionnaires": [
    {
      "id":2,
      "age":65,
      "an_underscored_attribute": 5
    }
  ], 
  "users": [
    {
      "id":1,"name":"Joe Blow",
      "email":"[email protected]",
      "questionnaire_id":2
    }
  ],
  "user": {
    "id":1,
    "name":"Joe Blow",
    "email":"[email protected]",
    "questionnaire_id":2
  }
}

This comes straight from UserSerializer.new(current_user).to_json

[brandonparsons]

Wondering if this is related to https://github.com/emberjs/data/issues/1173 ?

[alexspeller]

@brandonparsons that json is screwy, you shouldn't be including the user twice. Other than that, ensure that you are specifying somewhere to use the ams serializer, e.g. by using the ActiveModelAdapter:

App.ApplicationAdapter = DS.ActiveModelAdapter.extend();

[wycats]

@bradleypriest Do you think we can do a typeless form of pushPayload (with no primary record and everything considered "sideloaded")?

[brandonparsons]

@alexspeller I just found out upon upgrading you need to change the serializer - what is the difference between the ActiveModelAdapter you mentioned and the DS.ActiveModelSerializer extension? (i.e. App.ApplicationSerializer = DS.ActiveModelSerializer.extend({}))

The user showed up twice in the JSON because I had accidentally used include: true in AMS for both the user and questionnaire serializers. It didn't seem to affect anything (and could be useful if someone was requesting the questionnaire data directly, rather than the user) but I've removed it for now as it doesn't seem to be idiomatic.

[bradleypriest]

We've gotta consider the other side of the argument here too, linking https://github.com/emberjs/data/pull/1474

@wycats I guess we could make the type optional and use the ApplicationSerializer by default.
I don't know what sort of crazy stuff people are doing with serializers yet though.

[bradleypriest]

@brandonparsons the ActiveModelAdapter uses the ActiveModelSerializer by default, generally you'll be subclassing the ActiveModelSerializer in your model specific serializers

[wycats]

@bradleypriest I'm cool with defaulting to ApplicationSerializer if no type is provided.

[CodeOfficer]

Sideloading all the things when no type is specified sounds just dreamy to me.