Data: Support nested JSON objects

[devinus]

E.g.:

{
  name: {
    first: 'Devin',
    last: 'Torres'
  }
}

Thoughts:

1. name: DS.attr('object')
2. firstName: DS.attr('string', { key: 'name.first' })

Justification:

Nested JSON objects are too prevalent to require transforming them every time they're used. Models should model the data, not some transformed representation you morph in adapter, and hopefully not through hundreds of transformers.

[tchak]

Even if I was arguing with @devinus on the subject I am actually +1 on this :)
If the idea is approved I could submit a patch

[ghempton]

I'm +1 on this as well. This would be especially nice for people using non-relational databases like couch.

[devinus]

Or ElasticSearch in my case.

[devinus]

Another idea:

name: DS.attr({
  first: DS.attr('string'),
  last: DS.attr('string')
})

[tchak]

Hmm, the problem with this future is potential overhead on write. Read
is juste getPath, but on write, we would have to walk throu the whole
data hash, creating missing nodes...

On 19 janv. 2012, at 20:02, Devin Torres
[email protected]
wrote:

Another idea:

name: DS.attr({
first: DS.attr('string'),
last: DS.attr('string')
})

---

Reply to this email directly or view it on GitHub:
https://github.com/emberjs/data/issues/53#issuecomment-3572215

[devinus]

Could I have a quick comment on this from either @tomdale or @wycats so I could possibly move forward with a patch?

[tomdale]

Is this for read-only cases or do you expect to write them back to the server as well? How do you detect when the top-level object is dirtied?

You should be able to create multiple embedded models. Given the JSON data:

{
  name: "Steve",

  address: {
    street: "1 Infinite Loop",
    country: {
      short_name: "USA",
      long_name: "United States"
    }
  }
}

You should be able to model this with the following models:

Country = DS.Model.extend({
  shortName: DS.attr('string', { key: 'short_name' }),
  longName: DS.attr('string', { key: 'long_name' })
});

Address = DS.Model.extend({
  street: DS.attr('string'),
  country: DS.hasOne(Country, { embedded: true })
});

Person = DS.Model.extend({
  name: DS.attr('string'),
  address: DS.hasOne(Address, { embedded: true })
});

Are you arguing that this is too much ceremony? It seems like you actually _do_ want each of these conceptually to be separate models, so you can track dirty states separately.

[devinus]

I'm arguing that it can end up being too much ceremony, yes. For example, sometimes I may get back nested JSON that doesn't map well to hasOne attributes. For example:

{
  name: {
    first: 'Devin',
    middle: 'Alexander',
    last: 'Torres'
  }
}

Am I supposed to create a Name model for something like this? These things also aren't coming back with unique id's by the way, how does hasOne handle that?

[tomdale]

Since you are dealing with crazy, non-conventional JSON anyhow, couldn't you normalize the data in your adapter before you send it to the store?

[devinus]

This kind of nested JSON is all over the place: CouchDB, ElasticSearch, Neo4j... I'd argue that there's nothing crazy or unconventional about it. I'd also argue normalizing data in an adapter isn't what an adapter should be for either. I've had thoughts of maybe introducing some type of Transformer, but really--shouldn't the model model your data and not the other way around? Your thoughts?

[NickFranceschina]

+1 ... agree with @devinus . sometimes nested json properties are actually part of the "top-level object"... so then if any property inside of them changes, it would "dirty" the whole thing. and yes, for our app some of these nested-json sections are indeed being serialized and sent back to server

currently, can't use hasOne because nested models dont have "primary keys". could fake it in adapter, as mentioned... but that's extra work to make two unrelated concepts fit together (these aren't separate "database objects", which is really what hasOne/hasMany is doing for us.... so I'd be working around the framework)

will find another workaround for the timebeing...

[wycats]

Are you willing to write out the entire object when it changes, as opposed to being able to modify the object directly? Something like:

Person = DS.Model.extend({
  attrs: DS.attr('object')
});

var person = Person.find(1);

var attrs = person.modify('attrs', function(attrs) {
  attrs.age++;
});

// this would be illegal
person.get('attrs').age++;

The reason for this is that if you modify a conceptually whole attribute by mutating it directly, we don't know that the attribute has changed. We don't want to observe changes to the hash directly, because that may mutate the object in ways that harm its ability to be serialized back (for example, observers add on ES5 accessors to warn users who try to access them directly that they should be going through set/get; even without that, the object needs to be annotated with some extra observability information, which changes it from being a raw, easily serializable object to an extended object).

With that said, would you be comfortable with a .modify API (or something like it; open for suggestions here) for object-like attributes? If so, I think we could arrange it. Unfortunately, because we don't want to directly observe the object, it won't be easy for us to detect mistakes where people try to directly modify it :/ (we wouldn't want to make geting illegal, of course). One thing we could possibly do would be to use ES5 seal where available to try to trap mistakes, but the error messages won't necessarily be easy to figure out (because they'd come from the browser, not Ember).

Suboptimal all around.

[wycats]

Another idea is to return a proxy when you request the object (with [set]UnknownProperty), which would return the underlying value (or another proxy if it was nested data), and also mark the record as dirty when anything was set on it.

This would have the benefit of being mostly transparent, but @tomdale is still skeptical. I think it can work ;)

[devinus]

@wycats You're assuming we're only working with JSON that's nested one level deep.

[NickFranceschina]

I understand what you're saying... and agree that it is suboptimal. I think hasOne() actually works pretty closely enough... it would give me the control to build a Model for my nested JSON, which allows me to determine what sub-properties are being observed and how... if I want to do that. but currently that pattern:

• requires IDs for each model (I get errors if I dont fake the IDs... even with {embedded:true})
• I dont see a way for a nested-model to notify its parent that it has changed... no way to link the parent's "isDirty" to the child's "isDirty". I think maybe DS.belongsTo() might have done provided something like this??... but it doesn't seem to be in the 0.9.5 code

feels like there would be a way to change the association code a bit (hasOne, hasMany)... so then you're not inventing a DS.attr('object') where you dont exactly know what the developer wants you to do with it. we can make an "object" transform now... and do the work there on full set/get... but it feels like the tools are already here for the developer to be explicit with the model schema using DS.Model and build up a nested schema as complicated as they want

[NickFranceschina]

looks like a duplicate: https://github.com/emberjs/data/issues/100

[wycats]

@devinus nope. note that I said "or another proxy if it was nested data".

@NickFranceschina I also thought about reusing the hasOne code, but note that hasOne doesn't mark the parent object as dirty. Additionally, it wouldn't work for nested objects without even more shenanigans (you would want changes, however deep, to mark the parent as dirty). I think the proxy solution is the best solution we have so far.

[devinus]

@wycats Ah, my bad.

[garth]

I think that there is still a case for treating primitive arrays attributes seperate from object attributes. We can use syntax like DS.attr('date[]') and easily use transforms and manage dirty state. #96

[jfgirard]

+1 I'm about to start a project using couchdb. It makes a lot of sense to have object attribute type to support nested json objects.

[analytically]

+1

[wycats]

We haven't forgotten about this issue :D

[stefanpenner]

bump to restart discussion...

[leehambley]

What's the current status of this issue, looks like it's effectively dead?

If the issue won't be fixed, what is the recommended workaround?

[Neppord]

1+

[minznerjosh]

1+

Would LOVE to see a solution to this issue.

[julianalucena]

Any updates?

[arasbm]

I am also interested in progress on this issue

[minznerjosh]

Is there any way to manually force a custom attribute to generate the hash (using your attributes "from" function)? This would be a good workaround. Just setup a listener on the object's properties that regenerates the hash...

[ashugit]

Is there any solution to this, we are having Mongo as our backend and need to handle a lots of n level JSON data which may not have a predefined structure. An object like data could be the best solution.

[wagenet]

@wycats, @tomdale, what are your current thoughts on this?

[kelonye]

:weary:

[wycats]

@tomdale and I are working on this problem over the next few weeks for a client. We will have some progress on it soon.

[kelonye]

great :)

[adamantivm]

1+

[jonnii]

Is there any progress on this?

[maximegaillard]

Is there something new about it ?

[tomdale]

I am still against the notion of "object" attributes, but support for embedded hasOne relationships should be landing in the next few weeks.

[tomdale]

@mankind I do not understand your question.

[mankind]

@tomdale is this Merge embedded w/o ID tests?, a solution to the problem discussed here.

[pdf]

@tomdale Can you expand on your objection to 'object' attributes? It's a pretty natural pattern for applications backed on NoSQL documents.

[benmonro]

What's the status on this. Don't have control over the back end and I need support for nested objects, is this possible? It's been a real struggle for me. As @pdf mentioned, this is a pretty common use case.

[tedstrauss]

+1

[unrelentingtech]

My solution on jsbin (click "Run with JS").

[wagenet]

@benmonro You can do embedded associations. If the embedded object doesn't have an id then you'll have to auto-assign it one on the client. We don't currently support embedding JSON objects that aren't associated with a model. However, it may be possible to figure out a workaround like @myfreeweb.

[benmonro]

:+1: @myfreeweb looks like that works. My solution was similar, except it doesn't have the dirty tracking. Very cool, many thanks. @wagenet don't you think something like this should be baked into the framework? I mean there are a LOT of services out there (like mine) where you get back nested data like this and don't have control over them...

[wycats]

Auto-generated IDs for embedded relationships without a server ID are on the docket.

[benmonro]

:+1: cool, can't wait to see how that looks.

[wycats]

I'm closing this as out of date. The basic ideas in here continue to be part of our roadmap.

[buschtoens]

@wycats @tomdale
I'm really sorry to be nagging you, but what is the status on this?
There are a number of issues and pull requests that are all closed in reference to this issue.
The docs, which "should be accurate as of 1.0" contain no trace of these nested data structures.

Ember Data should really support nested objects and simple one-dimensional arrays.

Edit: Maybe I am just stupid and DS.hasMany("model") is exactly what I'm looking for.

[elwayman02]

I don't think DS.hasMany() supports primitives, does it?

[buschtoens]

Yeah, turns out that Ember Data still doesn't seem to support nested data.

The great thing about db systems like MongoDB is that I can do stuff like:

{
  id: 123,
  name: {
    first: "Jan",
    last: "Buschtöns"
  },
  address: {
    country: "Germany",
    town: "Herne",
    // etc...
  },
  products: [123, 456, 789]
}

I basically use _loose_ this gained flexibility, when using Ember Data.

[elwayman02]

How would you declare "products" in your model? It's just a primitive array, but there is no DS.attr('array'). Did you write a custom handler for this? I can't figure out how Ember Data supports something like this.

[buschtoens]

That exactly is the point. I don't know how to. :D

(For some reason I wrote "use" instead of "lose"... I need to focus)

[jasonkriss]

If you just want to have nested objects/arrays, you should be able to just declare the attribute as DS.attr(). This will pass the raw data through untouched. The catch is that dirty tracking will not work for that attribute.

[CodeOfficer]

A custom transform will do the trick. I had to do this once when modeling huge amounts of read-only map data.

App.RawTransform = DS.Transform.extend({
  serialize: function(deserialized) {
    return deserialized;
  },

  deserialize: function(serialized) {
    return serialized;
  }
});

App.MapFloor = DS.Model.extend({
  regionData: DS.attr('raw')
});

[slindberg]

After using ED for over a year, I finally decided to take a stab at making nested attributes observable.

My solution ended up splitting the difference between embedded records and raw object/array attributes. Model 'fragments' are basically treated just as normal models, just without any persistence logic. Modifying a fragment will update the parent's state, and persisting the parent will modify fragment's state. Note that this solution does not currently support arrays of primitives.

I've debated whether to turn this into a proposal on discuss.emberjs.com; if anyone else finds this useful I'll do so.

[wycats]

Is there some reason that DS.attr() (with no type) doesn't "just work" for MongoDB-style embedded data?

[wycats]

It seems @jasonkriss already raised this solution above. Note that while dirty tracking doesn't work, you can save() an un-dirty record.

[slindberg]

@wycats, I've found it to be extremely convenient to allow a the dirty state of a record to drive UI behavior. For example, disabling the save button when the record is clean, and enabling it to indicate that a user has made a change that needs to be persisted (or rolled back). Another example is halting a route transition after making changes to a nested object in order to display a confirm dialog.

In both of those cases, I previously put logic into routes or directly on the models themselves by overriding isDirty. After a few mixins and a little too much copy/pasting, I decided that record state management really just belongs in the data layer. Moving all of that logic out of my app has DRYed it up considerably.

[wycats]

Historically, the notion of "embedded dirty records" has been rife with edge-cases and confusion that eventually caused us to give up on it entirely. We started going down the path of allowing adapters to control dirtiness, but it became increasingly complicated to handle the edge-cases that people don't think about initially when working on this feature.

I would be perfectly happy to have a mechanism for explicitly marking a record as dirty, but trying to follow the white rabbit down a trail of embedded records to mark a potentially giant graph of records as "dirty" is much, much harder than it looks :frowning:

Would a "mark this record as dirty" mechanism help you?

[slindberg]

To be honest, I would likely use a ‘mark as dirty’ mechanism merely as an aid to implement the same kind of ED add-on I've started. Making a record dirty isn’t really the hard part of the problem :confused:

This issue is one of the most difficult I’ve encountered writing client side apps, but it keeps coming up because it is fundamental to apps with rich interactivity. Avoiding it in ED has just pushed the many edge cases and confusion into application logic, making for complex and brittle interdependencies. I’ve found that every time I put a stopgap in place I just end up revisiting it and performing refactor after refactor. IMO, that situation is not much better than dealing with it at the source (although it is true, people can't get mad at ED for bugs in their own code).

At the end of the day, it’s just a really hard problem — exactly the kind of common problem that I feel like Ember was created to tackle. For my part, Ember is the best client-side framework available specifically because of its aim to take the burden of seemingly simple yet highly complex problems away from the developer. Ember Data seems to follow this philosophy as well (god knows how complex a ‘simple’ data layer can be), and this is a prime example of those deceivingly simple problems.

[wycats]

@slindberg I totally agree with the sentiment behind what you're saying. The problem we encountered is that forcing people to think through the details of all of these edge cases to "write a simple adapter" when "I'm just trying to use MongoDB, how hard can that be" is a recipe for them to abandon Ember Data entirely.

I'm always open to strategies that can bend the curve by offering a simple, non-leaky solution for people until they discover that they need something more advanced, but our initial efforts on this front really ended up really complicating most of the Adapter and causing people to consider it overengineered.

To be perfectly honest, at this point in Ember Data, I'd rather if people fought with me because it didn't do enough than be constantly on the receiving end of complaints that it's overengineered. I'd love to chat more about ways we can bend the curve here, though.

Please let me know if you think of anything :smile:

[slindberg]

Damn, now I feel like I have to come up with something good :flushed:

I definitely respect your stance, having read through volumes of ED issues. In the specific case of managing the dirty state of models with nested structures, I don’t thing there’s any realistic curve bending. Perhaps a step in the right direction is to make it easier for people to author unofficial ED plugins (in this case, hooking into the model state and lifecycle). This might give people the opportunity to try out advanced behavior while deflecting frustration away from ED itself.

---

<belabor>
I believe it is possible to manage the dirty state of an object tree (not graph) sanely, despite the plethora of edge cases — especially if sub-objects are ‘modeled’ and only ever mutated through normal Ember channels. My hunch is that a good part of the original frustration of “I’m just trying to use MongoDB” folks came from the mismatch that @NickFranceschina brought up two years ago:

… nested models dont have "primary keys". could fake it in adapter, as mentioned... but that's extra work to make two unrelated concepts fit together (these aren't separate "database objects” …

Just having to fake nested JSON as real models with ids was hard enough without throwing in dependent state. Specifically addressing the needs of APIs with nested objects would (going out on a limb) not require the adapter to be involved, and therefore alleviate some of your concerns. Then again, any kind of built-in support should follow from the amount of actual use, and it's difficult to gauge what percentage of APIs out there expose document based idioms.
</belabor>

[buschtoens]

After a 3 week development pause at my project, I'd like to revisit this now. First and foremost I'd like to thank all of the participants for their insightful comments. Especially I'd like to thank @wycats for his continuous efforts, as well as @slindberg for sharing his "Model Fragments" code with us.

These are my opinions on the proposed solutions.

---

@jasonkriss mentioned that using DS.attr() without any further type specification will pass the raw data through, enabling the use of arrays and objects. However, this comes at the cost of disabling automated dirty checking for those fields.

@CodeOfficer suggested a new transform called RawTransform (DS.attr("raw")), which essentially is the same as DS.attr().

Both leave the state management up to the dev, which is a thing that ED ambitiously tries to do itself. I'd like to keep it that way. This is one of the main reasons we chose Ember and Ember Data.
For read-only data this would be okay though. Thanks for the hint!

---

@slindberg's solution enables nesting arrays of other models (DS.hasManyFragments("model")), just like DS.hasMany("fragment"), but propagating the state management up to the parent model.
For me, this is a step in the right direction.

Still, nested objects and arrays of primitives are impossible to implement in a dev-friendly way.

---

I feel like I misphrased my initial comment and didn't make myself clear enough. Sorry for that.

I don't want ED to handle nested objects with arbitrary fields. This would be insane. All fields _must_ be defined in the model to enable automatic state management. What I'd love to see, would be something similiar to this.

App.Customer = DS.Model.extend({
  id: DS.attr("number"),

  // a nested object
  name: DS.obj({
    first: DS.attr("string"),
    last: DS.attr("string")
  }),

  fullName: function() {
    return this.get("name.first") + " " + this.get("name.last");
  }.property("name.first", "name.last")

  // another nested object
  address: DS.obj({
    country: DS.attr("string"),
    town: DS.attr("string"),

    // nest-ception
    geo: DS.obj({
      lat: DS.attr("number"),
      long: DS.attr("number")
    })
    // etc...
  }),

  // an array of primitives or other Transforms
  loginDates: DS.arr("date"),

  // an array of objects: [ DS.obj({ .. }), DS.obj({ .. }), ... ]
  invoices: DS.arr({
    sum: DS.attr("number"),
    items: DS.hasMany("items")
  })
});

var obama = this.store.find("customer", 1);

/**
 * Nested objects.
 */
var address = obama.get("address") // returns a fragment
  , geo = address.get("geo");      // returns a fragment

geo.set("lat", 1.234);
geo.set("long", 5.678);
// ... is the same as ...
obama.set("address.geo.lat", 1.234);
obama.set("address.geo.long", 5.678);

/**
 * Arrays of objects.
 */
/**
 * This returns a subclass of `Array`, that implements most of
 * `Array.prototype.`
 */
var invoices = obama.get("invoices");

// getters and setters as usual
var firstInvoice = invoices.get(0);          // returns a fragment
// ... is the same as ...
var firstInvoice = obama.get("invoices[0]"); // returns a fragment

// this is not allowed.
invoices[0].set("sum", 1234);

// instead do this
invoices.set("[0].sum", 1234);
// ... or ...
obama.set("invoices[0].sum", 1234);

[slindberg]

@silvinci, I believe you can achieve almost everything in those examples with that model fragments gist. You don't get to use the DS.arr and DS.obj style sugar, but I've found that having a separate definition of model fragments is preferable since they become reusable and can have their own serializers.

@slindberg's solution enables nesting arrays of other models

This isn't quite true. In the case of a property like DS.hasManyFramgents('foo'), the foo model isn't a DS.Model, but a DS.ModelFragment (that's where all the magic happens).

Still, nested objects and arrays of primitives are impossible to implement in a dev-friendly way.

Nested objects are supported with DS.hasOneFragment(), and for the most part behaves like in your example. There's very little difference between obama.set("address.geo.lat", 1.234); and obama.get("address.geo").set("lat", 1.234);, and the latter may even be possible (it would be a feature of Ember.set if so).

Shortly after posting on this thread, I updated that gist to support arrays of primitives: just omit the model argument from DS.hasManyFragments(). There's no type checking or transforms (so dates wouldn't be treated specially), but I've been using it in my app for a while with much success.

Also, the bracket notation you are using in the nested array example isn't something that ember supports at all. Remember that you can always do someArr.get('firstObject') on any Ember array (e.g. var firstInvoice = obama.get("invoices.firstObject")).

[buschtoens]

@slindberg Thanks for your reply!

Nested objects are supported with DS.hasOneFragment().

I didn't skip through your code. I only read the synopsis from lines 1-54 which only showcased DS.hasManyFragments("fragment"). DS.hasOneFragment() looks absolutely magical.

I've been using it in my app for a while with much success.

So I can guess that your plug-in has no known bugs and is well-tested? How about turning it into a real repository with support for various build systems (component, bower, etc.). I'd be willing to do that. You'd only have to init the repo and I'll do some PRs.

Also, the bracket notation you are using in the nested array example isn't something that ember supports at all.

I haven't used ED extensively yet and this was just an API proposal, that popped to my mind. I claim no code validity. I'd say I'm an Ember rookie. :smiley:

[buschtoens]

@wycats What was the reason to not bake such a feature into core? It would not change the surface API in a backwards-incompatible way and folks who don't want to use nested data can happily go on with their business as they are not affected at all, but people like @slindberg and me could greatly benefit from an implementation similiar to this.

[elwayman02]

http://i.imgur.com/ssZfpXl.jpg

[wycats]

The right way, conceptually, to do embedded records in a built-in way is to define a transformation from embedded records to the current internal format, and build that transformation in.

One source of confusion is that there are really fundamentally two kinds of embedded records:

• Embedded records that are included with the primary record purely for efficiency. Otherwise, they have their own IDs and can be saved separately.
• Embedded records that are conceptually a part of the primary record. They may not have their own IDs and may not be saved separately.

These two cases are actually fundamentally different and need to be defined using separate APIs in Ember Data. People want to use separate record types for both cases for good reason, and using records would allow the second variant to get proper dirty checking, but we probably need to revisit making "embedded relationship" an application-level concern, in the sense that it affects saving as well.

We had originally tried to make refactoring between these two variants seamless at the application level (and purely a adapter-level concern), but I think that that kind of refactoring is actually rather rare, and not worth the added complexity (and missing feature for so long!).

[wildchild]

Ok, finally, can we expect id-less nested objects support in ember-data?

[buschtoens]

@wycats in here:

I think we probably will support embeddd records in the long haul; I outlined the issue in #53 (comment), and we should discuss it more.

Wouldn't it make sense to embed this functionality _before_ locking in on 1.0?

• Embedded records that are included with the primary record purely for efficiency. Otherwise, they have their own IDs and can be saved separately.
• Embedded records that are conceptually a part of the primary record. They may not have their own IDs and may not be saved separately.

I understand the difference and I second this. These are two totally different concepts and they should be treated seperately. That is why I'd like to see support for embedded records / nested JSON.

IMHO the nesting of objects is one of JS most used "features". It is a core concept of many JS applications.

Do you have any plans or ETA yet, when this will be addressed? Pre 1.0? What parts of Ember and/or Ember Data would have to be changed?

Sorry for being so extremely pushy.

[ksol]

For what it's worth, having something to model "Embedded records that are conceptually a part of the primary record" is exactly what I need right now. Something like the fragment-way proposed above would be a good fit, but I might not be used enough to ED.

There's no emergency here and I don't want to put pressure on anybody; I'm only saying this to show that some people need the feature.

[simonoff]

+1 Embedded records that are conceptually a part of the primary record. They may not have their own IDs and may not be saved separately. is a must-have case. other case can be done by saving each model separately without any issues.

[chrisvariety]

@ksol @simonoff have been using https://github.com/lytics/ember-data.model-fragments for that exact use case -- works great !!

[simonoff]

@chrisvariety I will try but I think what I will move to Backbone.js + Backbone-relational.js what is working fine without any issues.

[simonoff]

@chrisvariety i tried model-fragments and ember-data-extensions. None of them working.
The main issue what after get response from server it creates duplicates. Parent object includes hasMany items what was before(without id) and new items (with id) what is the same except ids. So need to call .reload() on parent to fix it. Why not check app attributes and if all equal except id and one record has null id then replace only id instead of adding new item?

[Glavin001]

+1

Embedded records that are conceptually a part of the primary record. They may not have their own IDs and may not be saved separately.

I am in need of this for a work project. I am going to try ember-data.model-fragments as it looks like what I want. Is there any progress for an official Ember-Data solution?

[buschtoens]

This could be mildly related: DS.EmbeddedRecordsMixin

[Glavin001]

Thanks, @silvinci. I will give that a try, too.

[ksol]

ember-data.model-fragments is working for me. My use case is pretty simple though, and involves consuming an API without modifying/updating the records or fragment. YMMV.

[Glavin001]

Ember-data.model-fragments was the solution for me.
I have created a Fork of ember-data.model-fragments at https://github.com/Glavin001/ember-data.model-fragments and it contains an example app that can be viewed at http://glavin001.github.io/ember-data.model-fragments/dist/

[gabelimon]

@wycats I believe that embedded records are supposed to resolve this issue but they don't actually support models with embedded objects. An I correct in my thinking or do I need to use a third party library like model fragments?

[alvincrespo]

:+1: On this. In my case, we have users who are able to define their own attributes - therefore establishing a model is not really ideal since these attributes can be anything from _first_name, first_name, firstName, _firstName, etc...

So our payload looks something like:

{
  attrs: {
    first_name: "Zoid"
  }
}

And our model:

export default DS.Model.extend({
  attrs: DS.attr()
})

So when the user updates an attribute, first_name, the dirty state of the model doesn't change. So having support for this out of the box in ED would be awesome.

[avaz]

Hi,
I read this whole thread and besides I got many insights on how to deal with embedded objects I can't discover the current status of this solicitation. The thread is closed, this feature was/will be implemented in ED or not?

Thanks!

[lolmaus]

So what is the recommended way for having an arbitrary JSON-like object as an attribute?

[simonoff]

@lolmaus use backbone.js :)

[bmac]

@lolmaus Currently the best way to have an arbitrary JSON-like object as an attribute is use DS.attr() as suggested by @jasonkriss.

If you just want to have nested objects/arrays, you should be able to just declare the attribute as DS.attr(). This will pass the raw data through untouched. The catch is that dirty tracking will not work for that attribute.

[lolmaus]

@bmac, say, from that JSON-like attribute i render a complex editing form with various types of controls: checkboxes, text fields, select lists...

After a user has changed a value of a control, Ember will update the object corresponding to the control's value. But the attribute containing that object will not be dirtied and the user's edit will not propagate.

So all i need is to dirty the attribute whenever the user modifies a control, and everything will be okay, right?

If yes, then the question is how do i dirty the attribute and how do i do it _after_ Ember saves the value of the child object of the attribute.

[alvincrespo]

@lolmaus Might be good to have a jsbin w/ an example use case?

[bmac]

@lolmaus Unfortunately I do not know of a good way to mark the parent object as dirty.

[elwayman02]

https://github.com/lytics/ember-data.model-fragments was the way to go for us, @lolmaus

[lolmaus]

@elliterate, model-fragments claims to be incompatible with the current version of Ember Data. I also find it risky to depend on a third party extension that hacks Ember internals: even after becoming compatible, it might break with next update of Ember Data.

I managed to come up with a solution using native Ember features: http://emberjs.jsbin.com/vapama/1/edit?html,js,output

Highlights:

• The model has a JSON-like attribute thedata.
• The template renders checkboxes for every item contained by the attribute. Each checkbox has checked set to the corresponding item.
• Checkboxes are customized Ember.Checkbox that send a stain action to the controller on change.
• When the action is triggered, the controller marks the record as dirty.
• For the controller to know which record to stain, i have to pass the record from the template to the checkbox, and the checkbox has to pass the record to the triggered action.

The latter will require a lot of utility code, so don't hesitate to suggest a more concise approach if you have something in mind.

[lolmaus]

Here's a slightly more compact version: http://emberjs.jsbin.com/vapama/3/edit?html,js,output

Instead of having the checkbox ask the controller to stain the record, the checkbox stains the record itself.

[lolmaus]

An even shorter version, now reusing the input helper rather than subclassing it: http://emberjs.jsbin.com/vapama/4/edit?html,js,output

Can you guys please review this and tell your opinion on whether it's an appropriate solution?

UPD: http://emberjs.jsbin.com/vapama/5/edit?html,js,output

• Used an observer of the checked property rather than a callback assigned to the change attribute. Not sure which way is better, though.
• I discovered that staining would happen before the change would be applied to the object. Resolved this issue by wrapping the staining into an Ember.run.later.

UPD2: http://emberjs.jsbin.com/vapama/8/edit?html,js,output

• I've realized that staining the record does not announce the change in the JSON-like property, so i've decided to replace staining the record with announcing the change in the property. But i've discovered that announcing the change in the property does not result in staining the record, so i had to have both staining the model and announcing property change.
• Extracted most of the code from Ember.Checkbox to Ember.View, so that it can be reused in other helpers.

[lolmaus]

Okay, this works in a JSBin, but in real-life scenario this approach requires so much utility code that it becomes simpler, more error-proof and definitly more Ember-way to use nested models.

[lolmaus]

Yay, Model Fragments have been updated to support Ember Data beta 14! :D

[KTKate]

I'm also trying to do a DS.attr('json') type thing to send back to the Rails backend. My situation is very similar to that of @alvincrespo where the user needs to be able to specify the key and the value. In my case we need the user to be able to specify multiple key/value pairs if needed. I'm hesitant to try ModelFragments because I'm wary of adding more dependencies.

[slindberg]

@JKGisMe I have the same need in my app for which model fragments works well. I understand your reluctance to add more dependencies, but model fragments was born out of your exact need. Ember Data may one day support nested id-less records like this, but that day is far in the future.

If you are interested, this is the rough pattern I'm using:

app/models/thing.js

export default DS.Model.extend({
  config: DS.hasManyFragments('config_field')
});

app/models/config-field.js

export default DS.ModelFragment.extend({
  name: DS.attr('string'),
  value: DS.attr('string')
});

app/templates/thing.hbs

{{each config as |config|}}
  <div class="form-control">
    <label>{{config.name}}</label>
    {{input value=config.value}}
  </div>
{{/each}}

This will serialize thing records with a config property that is an array of objects with name and value properties. If your API expects an object, you can override the thing serializer:

app/serializers/thing.js

export default DS.RESTSerializer.extend({
  serializeAttribute: function(snapshot, json, key) {
    if (key === 'config') {
      json.config = snapshot.attr('config').reduce(function(config, fieldSnapshot) {
        config[fieldSnapshot.attr('name')] = fieldSnapshot.attr('value');
        return config;
      }, {});
    } else {
      this._super.apply(this, arguments);
    }
  }
});

[igorT]

:+1: for model fragments

[delapuente]

Hi people. I'm following this approach. In my case I want to store a _info_ object for a _channel_ entity I'm working on:

var Channel = DS.Model.extend({
  name: DS.attr('string'),
  info: DS.attr({ defaultValue: () => Info.create() })
});

var Info = Ember.Object.extend({
  version: DS.attr('string'),
  description: DS.attr('string')
});

What is bad about this approach?

Thank you.

[igorT]

Well, I have no idea what happens when you put a DS.attr inside something that is not a DS.Model it definitely isn't a supported use case. You could have Info be a normal object, but you don't get dirtyness/rollback and some of the normalization/serialization niceties, but it definitely works for simpler use cases

[KTKate]

Thank you @slindberg I didn't see this earlier!. That looks very clean and probably would have worked great. The specs for our project changed a little and I ended up just making a whole new model & accompanying component in Rails & Ember which I then convert to customized JSON in the end from Rails.

[billpull]

Is ember-data-model-fragments now incompatible with ember-data 1.13.2

[igorT]

It will be compatible soon

[boehlke]

Hello guys, I followed this discussion because I have the same use case (nested documents).
After some thinking, I decided the solve the problem by creating a server-side component which flattens the data so that it can be used with ember-data.
My input to this discussion: Probably you do not want embedded records in the first place. Think about the complexity you introduce the your ember app. It is probably better handled on the server.

Having said that, I vote against having model-fragments in ember-data.
:-1:

[lolmaus]

@boehlke You assume that any data can be flattened, but that's absolutely not the case.

[boehlke]

@lolmaus Sorry, I cannot think of an example JSON object which cannot be flattened. Can you provide one? Probably that's easier than for me to prove that every JSON can be flattened. Thanks!

[wildchild]

This thread is turning into masturbation. Embedded objects are parts of JSON documents. If ember won't support it, fuck it. This framework is too "idiomatic" to be usable.

[lolmaus]

@boehlke Imagine a hash that has arrays which contain hashes which contain arrays of hashes.

You can of course undestructively flatten it by producing really elaborate key names. But working with such a flat structure from Ember will be a nightmare. You won't be able iterate over specific sub-arrays, etc. That's just not a working solution.

[stefanpenner]

The reality is, depending on the scenario people want both flattened and nested structures are ideal.
As with a DBMS, if one wants to keep data consistent, normalized structures are ideal. Many times though, this constraint isn't important and more natural graphs become easier to deal with.

Compromises exists, but as ED's goal was consistency, that explains its current state (not its final form).

If ember won't support it, fuck it.

Ember works fine with deeply nested data-structures out of the box.

Ember-data also works with deeply nested documents.

I believe the pain people are noticing. Is they want there nested blobs to behave like ember-data models, in both a read/write/serializer/deserialize fashion. Ember-data today, is largely the following units (on top of ember), and models have nice patterns, such as:

1. an identity map (to ensure you are always dealing with a single canonical entity)
2. a set of adapters (to normalize foreign endpoints/api)
3. a set of serializers (meant to contain all logic related to transformation of data to/from the server)
4. a mechanism of relationship syncing
5. a mechanism to detect changes, and the ability for a developer to commit said changes back a foreign data-store

Expanding these patterns to the nested id'less entities, is something we are interested in exploring. the model-fragments project is showing promise, and is a very intriguing interpretation of the problem. Unfortunately, it relies on private API and monkey patching to accomplish its goal. One of our goals moving forward is, to ensure a stable public API for model-fragments. This will prevent the upgrade turmoil currently introduced.

Now as for model-fragments becoming part of ember-data itself, it's too soon to know for sure. It is great to push this logic into user-space, as it will come with some caveats (and the problem space needs some more fleshing out) The important part today is for both model-fragments and ember-data to work better in tandem. This will allow the flexibility for authors to experiment, but also the stability to ship real projects with it.

[fivetanley]

@wildchild I would advise you to read the community guidelines. I do not mean to belittle your frustration but this kind of behavior is not acceptable in the ember community. http://emberjs.com/guidelines/

[devinus]

@stefanpenner

Ember works fine with deeply nested data-structures out of the box.

I just ran into this last night:

{{upload-field inputId="desktop-hero" accept="image/*" signURL=signURL uploadedURL=model.templateSettings.context.hero.src}}

Because model.templateSettings = {} and model.templateSettings.context.hero doesn't exist yet, the value is never updated.

I am forced to do this as a workaround in afterModel:

    if (isNone(get(model, 'templateSettings.context.hero'))) {
      set(model, 'templateSettings.context.hero', {});
    }

[stefanpenner]

@devinus a better place would be to add that to your serialization code.

Also, those questions are better asked on SO.

[devinus]

@stefanpenner What question?

(Also, this is before serialization even happens. I was just providing a counter-example to your claim that "Ember works fine with deeply nested data-structures out of the box." In my experience, it does not.)

[aivanov93]

@lolmaus I'm having trouble figuring out how to nicely flatten a JSON.
I have a similar problem to this SO question http://stackoverflow.com/questions/25220442/json-api-and-translations-with-ember-data and I really like the first answer with nesting everything under the translations field.
Sure, I could flatten everything to title_en, title_fr, but this way on a change of locale, I would have to manually change each field.
Any nice solution to this case?

[lolmaus]

@aivanov93 ember-data-model-fragments seem to fit nicely into your use case.

But I would attempt a completely different solution. I would split the model into two parts, e. g. post and post-translation. A single post would look like this then:

{
  id: '1'
  type: 'post',
  relationships: {
    translations: [
      {id: '1', type: 'post-translation'},
      {id: '2', type: 'post-translation'}
    ]
  }
}

{
  id: '1'
  type: 'post-translation',
  relationships: {
    post: {id: '1', type: 'post'}
  },
  attributes: {
    lang: 'en',
    title: 'Hello world!',
    body: "Wonderful weather we have today, don't we?"
  }
}

{
  id: '2'
  type: 'post-translation',
  relationships: {
    post: {id: '1', type: 'post'}
  },
  attributes: {
    lang: 'ru',
    title: 'Здравствуй, мир!',
    body: 'Чудесная погодка сегодня, не так ли?'
  }
}

This approach will also let you collect language-specific comments to posts.