Lektor: Allow for querying descendants - children recursively

[nixjdm]

There is a need for this expressed https://www.getlektor.com/docs/api/db/record/children/ and on gitter.

To be clear, instead of returning only the direct, 1st generation children of a record, descendants would be recursive for the entire depth of the record and return all children of all depths.

We already have this.children recursive, but I think descendants still has a use, as it allows you to chain methods together more easily.

[runfalk]

I think descendantsis just less powerful. A recursive for can do both list and tree type structures with children. Do you have a particular use case where a recursive for is unwieldy?

[nixjdm]

I can think of a couple differences that arguably make it more powerful. Take these examples:
1)

{% for child in this.children recursive %}
  {% if child._model == 'page' %}
    {{ child }}<br>
  {% endif %}
  {{ loop(child.children) }}
{% endfor %}

vs. 2)

{% for child in this.descendants.filter(F._model == 'page') %}
    {{ child }}<br>
{% endfor %}

vs. 3)

{% for child in this.children.filter(F._model == 'page')  recursive %}
  {{ child }}<br>
  {{ loop(child.children) }}
{% endfor %}

The first two provide the same output; all descendants of this with the _model == 'page'. 3) actually fails, since if there is a page with _model == 'page' that is a subpage of a page with a different model, it will be missed.

1) vs 2): 1) uses built in Jinja to do the logic, and 2) would use the proposed new feature of Lektor. Which is easier to write is a bit of a matter of opinion, but I personally would say number 2) is easier. You could call 2) syntactic sugar. There's also something to be said for moving logic out of Jinja and into Python. However, I'm not aware of anything that descendants would allow that can't be done right now.

I also feel like descendants, and the other standard terms used in describing a tree are somewhat expected, and we should maybe add some the other missing ones as well, such as ancestors.

Side note: @Jossnaz in the same conversation also mentioned he'd like what I think are two additional features: A) A database api to perform a join on queries, and B) to be able to call pagination on any query set (such as a concocted joined one, or descendants) and have this generate pagination data for applicable pages. Tickets haven't been made for these yet. I bring this up because for his purposes descendants is somewhat of a stepping stone.

@Jossnaz, do you have any other specific arguments or examples supporting descendants? It took me some thought to think of it this much, and I'm betting you've thought about it quite a bit.

[runfalk]

I can see why 2 seems better than 1. I guess it makes a lot of sense for @Jossnaz DB query style use case. I haven't been able to come up with a normal use case where it's useful to flatten a tree of pages and not getting that tree back again. If you want them to be flat, shouldn't that be done in the original hierarchy?

[Jossnaz]

the main reason is better abstraction of route and model.

Say you have
/blabla/modelX

/projectYY/modelX

/projectYY/modelY

/modelX

/modelY

... etc the list goes on

now get me all modelX with working pagination.

the argument "well keep all modelX in one sub folder" is unsatisfyingly naive. Lektor should serve me, not vice versa. The original use case is this: you have a wordpress page that wants to migrate to lektor. This is common, tarquesian did the same. Wordpress allows things like hierarchical posts / pages. E.g. father has url father child has url father/child. Wordpress basically allows you a little bit to create the url the way you want. And think about it, why should that not be allowed? at the end of the day, the model type must not have limits because of its URL. Those are two logically separated things. Now create all pages without or almost without breaking or redirecting urls and without duplicating content in different urls (because there is a penalty from google on that). The recursive hack mentioned here does not create a working pagination. Or does it?

Descendants is an important way of filtering and querying objects. There were at least 4 people I saw on the lektor docs asking for it.

in detail, please use your mentioned recursive hack on:

[children]
replaced_with: ....

of course, maybe I am missing something. But feel free to shed some light.

[Jossnaz]

I now have to implement pagination for querying all records in subfolder resources. E.g. there is /resources/ /resources/videos/ /resources/white-papers/ /resources/pdfs/ . The models are res-video res-pdfupload res-link. Get me all those records, doesn't matter if link or video or pdf, get them mixed, sorted by date and make a pagination. Easy right?

[runfalk]

If you want to sole the particular case /resources/something/slug I think you can solve that with children and one sub-folder:

• You have the resources model
• You have a resource model with some kind of way to distinguish between the different types
• You set the URL slug to something like {{ this.type }}/{{ this._id }}, where type can be pdfs, white-papers, videos, etc. This will give you the URLs you want while still keeping a sane data model

I have not tried this, but in theory it seems very doable. You can read more about it on https://www.getlektor.com/docs/content/urls/

If that can be a solution to your particular issue I think adding descendants adds more complexity than it's worth.

[Jossnaz]

true that would solve this exact issue and something like that is what I
currently plan on doing.

now what about say, blog categories with sub categories?

e.g.

[fields.categories]
label = Categories
type = checkboxes
source = site.query('/blog-categories')
width = 1/2

I want not only 'blog-categories' but as well it's sub categories.

something is missing, yes, if there is just one level of blog categories it
works. But bad luck to you if you want to do something slightly different.

what about you want a resource in a different place than /resources as
well? Say I have some videos as well in /training I already tried you
cannot define the slug to be ../training/something
that was btw one of the first things I tried with lektor.

the question: "get me all video records" can only be solved in the view
via recursive function calls. I don't think thats cool.

I say I want to query all records never matter where they are. And I want
to have them available on model definition level. If it's only available on
view level, it limits in what I can do with it. And I want to run actually
nice queries over all records. That is not something insane to ask for
imho. By all means, if there is a way to achieve it, please tell me.

as soon as the structure gets a little bit more complex, the records are a
bit spread out, the air gets really thin for lektor. And that wasn't clear
to me to begin with. And I don't think it's cool. It should be simple and
fun to work with lektor.

On 19 September 2017 at 12:39, Andreas Runfalk [e-mail]
wrote:

If you want to sole the particular case /resources/something/slug I think
you can solve that with children and one sub-folder:

• You have the resources model
• You have a resource model with some kind of way to distinguish
  between the different types
• You set the URL slug to something like {{ this.type }}/{{ this._id }},
  where type can be pdfs, white-papers, videos, etc. This will give you the
  URLs you want while still keeping a sane data model

I have not tried this, but in theory it seems very doable. You can read
more about it on https://www.getlektor.com/docs/content/urls/

If that can be a solution to your particular issue I think adding
descendants adds more complexity than it's worth.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lektor/lektor/issues/445#issuecomment-330500226, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AB-d-sT_V2j1Zrq8Jq03GovuYU2iD2u1ks5sj5n-gaJpZM4PU3TC
.

[xlotlu]

I'd like to see glob patterns working in queries, and this would solve this here issue.

site.query('/blog-categories/*')

would be equivalent to the current behaviour.

But

site.query('/blog-categories/**')

would fetch everything recursively.