V8-archive: Relational Repeater Interface

[theharshin]

Problems with current repeater interface.

• The configuration of child interfaces is saved as JSON in the database. Those fields are not available inside directus_fields. This makes those fields isolated from Directus system and prevents future updates being applied at fields level.
• The repeater child interfaces does not follow Directus' default field creation process, hence the default schema & configurations are not being applied.
  

  - Complex UX. The whole field creation process is placed under single accordion, which makes configuration difficult when there are many child fields.

Proposed Changes

• Consider repeater as an interface option instead of a separate interface.
• When user selects any interface and turns on the repeater option, the Field Type will be always an Array and MySQL Datatype can be changed to TEXT

genre:["thriller","comedy"]

• The app will handle the repeater field like this. (Just like tag interface but with a button and repeated inputs)
  

  

How to repeat multiple fields?

• Introduce a new interface called Interface Group
• Allow adding a child interfaces under Interface Group
  Just an idea how it would look:
  
• As the repeater is now an option of interface, Interface Group can also be repeated just like single field.
• The Field Type will still remain an Array and data saved in DB will be an array. Each item inside an array will be a JSON with child field as key.

experience:[{
 "company":"Directus",
 "years_of_service":1
},
{
 "company":"7Span",
 "years_of_service":5
}]

• Child interface should be treated as an alias field as it does not have an independent values but will be under parent interface.
  

  - directus_fields will contain all the information about child fields. To map the child field to parent field, we can use the group column.(which is already there)

There are already few issues with repeater fields which might be solved with proposed changes. directus/app#1994, directus/app#1985, directus/app#1965, directus/app#1903, directus/app#1864, directus/app#1861, directus/app#1845, directus/app#1818
This will be a breaking change.

[rijkvanzanten]

The repeater with multiple fields will be a relational interface in that case, correct? The way you describe it, making the group repeater a relational interface gives us the ability to rely on directus_fields for setting up the fields, and it'll take care of saving / retrieving the data, as it's now a "regular" one-to-many in a different visual style

[rijkvanzanten]

Also, thinking about that, I don't think there have to be 2 interfaces. A repeater with only 1 field is... just that: a repeater with only 1 field setup instead of multiple. It can still follow the same relational setup

[theharshin]

When we consider relational, there will be a separate table for the data right?
For repeater, there won't be any new table but just an array of data.

So if a single field is repeated, data will be saved like this:

"key":["value_1","value_2"]

For an Interface Group the data should be like this:

"key":{
 "child_key_1":"child_value_1",
 "child_key_2":"child_value_2"
}

And if the Interface Group is Repeated, the data should be like this:

"key":[{
 "child_1_key_1":"child_1_value_1",
 "child_1_key_2":"child_1_value_2"
},
{
 "child_2_key_1":"child_2_value_1",
 "child_2_key_2":"child_2_value_2"
}]

So we need only 1 new interface which is "Interface Group".

[rijkvanzanten]

For repeater, there won't be any new table but just an array of data.

Why?

Why not go relational?

How are fields / fields-config stored if it's not relational?

[theharshin]

The relational may not make sense every time and if you need relational interfaces we already have those. Let me give you some examples.

The Carousel
Consider adding a carousel inside a blog article. Each slide has _title_, _description_ and an _image_. Making a relational table would be overkill here.
Instead we can use Interface Group. Each group will have child interfaces for _title_, _description_, _image_. This would be a quick and straight forward to manage.

Social Links
An author bio can have dynamic links. We can create an interface group for _icon_ and a _link_ and make it repeatable.

Both the cases can be achieved via relational setup too. But that we can leave on developers how they want to manage.

The child fields will be an alias, so the directus_fields will contain all the information but a collection won't have a column of it. directus_fields contains a column group where we can define a parent field or we can also add a new key under options.

[rijkvanzanten]

Each group will have child interfaces for title, description, image. This would be a quick and straight forward to manage.

This is exactly how the current repeater works. The only problem is where the field options are stored. We need to store all the information that's normally stored in directus_fields somewhere. It can't be in directus_fields however, cause these fields aren't fields of any collection. They're "sub-fields" (for lack of a better term) of the one field that _is_ part of the collection.

The only way we can accurately store this information is if the sub-fields are fields of a related collection, and each row here is an item in the related collection (which also makes the most sense from a data storing normalization perspective).

I also don't agree with "Making a relational table would be overkill here.", as that's exactly what we do for One-to-Many fields, which are arguably the exact same setup as this.

[theharshin]

The "sub-fields" are same as fields and represented by interfaces, thus the Directus must have full control over it, so making a JSON is not a choice.
Is keeping "sub-fields" inside directus_fields a very bad thing? Can't we distinguish them with a new column, say parent which will contain the name of parent field?

The relational approach is undoubtedly the best way to manage data (except the setup 😅)
There will be a major difference in setup process for the approaches.

Non-Relational Approach

1. Goes to the collection.
2. Selects any interface.
3. Turns on the repeater mode.

• Needs Group interface for sub-fields.
• Data will be always an array.
• Won't have filtering control to sub-fields.

Relational Approach

1. Goes to the collection.
2. User selects repeater interface.
3. Sets up the relational table.
4. Goes to relational table.
5. Adds fields to it.

• Uses O2M
• Won't have select existing option.

I see there was a discussion on multiple types of repeater fields here: https://github.com/directus/app/issues/1205#issuecomment-480884171
Does that relate to what we're discussing here 🤔 @benhaynes

[benhaynes]

An interesting point... let's see what @rijkvanzanten thinks when he's available.

[vuchl]

Can't we distinguish them with a new column, say parent which will contain the name of parent field?

This is a common practice on how to store tree-like structures in the same table. Querying makes it a bit harder though. But in can be solved with a recursive CTE.

[sayore]

Using

• JSON for a Fallback Repeater(Which only includes a subset of "Pseudointerfaces" [Simple Text, Number and Static File(only carrying the full url and id), maybe Button/Boolean, maybe Dropdown]),
• and relational Repeater for the full-fledged Version.

I agree that there definetly needs to be a relational Version so that every Interface is able to be used.

But i (maybe) see an Issue when thinking about Query-chaining though. Making a JSON Repeater will flatten Querries a lot on bigger sets of data. Though ultimately for normal CMS usage this shouldn't matter. Just throwing this in right now.

[benhaynes]

Thanks @edenprojectde — those seem to me like two different repeater interfaces... which I think makes sense. We can build these separately so both are clean and organized, similar to checkboxes and checkboxes-relational.

[rijkvanzanten]

It would be a good idea to update the current repeater interface to hide the relational interfaces for that one seeing that it doesn't work with the relational interfaces to begin with. I think we might be able to scan for the o2m / m2o / translation types for each of the interfaces

[cupcakearmy]

Is this coming with directus 8 or 9?

[rijkvanzanten]

Yup! Hard to give an exact release date though.

[cupcakearmy]

Thanks for the quick answer! Do you think it's a matter of days/weeks/months?

[rijkvanzanten]

Hoping to be able to add it in to v9, which is closer to weeks than months, but definitely not days.

[daniel-upzzle]

Maybe it will be much like Wordpress Custom Fields work. Is a very flexible and fast way to put some data, in a objects database like style.

I think a key point here, and maybe I'm late for this, is to let choose between diferent possible interface groups in each item the user creates. If you take a look over Flexible type in WP Custom Fields is what it does.

It solves the option for a "M2MM" relation type. I think you called it like this, once ago when was announced in directus 6 times. I implemented a hack over directus 6 (while learning backbone? at that time) to achive this. It worked but with a couple of no user-friendly steps.

From strict relational database perspective it's maybe a strange, no so good, thing to do. But for content creation, that has really no need to be more than a document tree of different structs of data it's a pretty good option. Like when making web pages, in many cases.

I think it is currently not avaiable a relational field in a repeater's field types? Or I'm wrong? It will be possible soon?

[daniel-upzzle]

I don't have quite enough knowledge of how it works now. I will watch it closer to be able to help. But maybe this idea can help:

• Ability to create collections with a new NESTED boolean setting marked as true.
• Collections of this type are definitions but not current real database tables.
• These are the "interface groups".
• When selecting a new interface for a collection field, you are able to select Nested Collection. In it's setting select from avaiable NESTED collections.
• It's saved like a JSON.
• You can mark the new [repeater]. (array oj jsons or json with an array in its first level).
• If current data doesnt match last definition, you can ommit definition and leave it as free json.
• NEW interface FLEXIBLE NESTED COLLECTIONS. It's like the previous one, but in its setting many NESTED collections may be selected and with each new item you can choose between them.
• In fact, the first case can be ommited and only be this named as NESTED COLLECTIONS.

With all this, I think it is important to see that all that can be thought as only a way to help and constraint the creation of JSON fields. "Nothing more to check". And if current value not matches definition the user can avoid the helper o reset/merge values.

[daniel-upzzle]

Nested collections would work as standard collections but bypassing the steps implying create/alter/drop/delete/... for a real table in the user space. And are not elegible to query data from the API.