Geonode: GNIP-78: GeoNode generic "Apps" model to include pluggable entities into the framework

Created on 1 Dec 2020 · 11Comments · Source: GeoNode/geonode

GNIP-78: GeoNode generic "Apps" model to include pluggable entities into the framework

Overview

The scope of this GNIP is to present a technical proposal for the integration of generic geospatial entities into GeoNode as part of the available Resource Bases.

The main goal is to provide a way for a GeoNode User to manage and share "Geo Applications" similarly to the other GeoNode entities, such as Documents, Layers and Maps.

Such Geo Applications could be any kind of object provided by a plugged in client library, which is not a Layer nor a Map.
As an instance Dashboards, GeoStories, Charts, Geospatial Analysis and so forth.

Before moving forward with the proposal, we need to take into account some prerequisites and constraints:

We want to manage such “Geo Applications” by using the same interfaces we already have in place for Layers, Maps and Documents.

This new entity needs to be managed somehow like one of the GeoNode’s ResourceBase ones.
- It must be exposed through the GeoNode APIs
- It must be integrated with the GeoNode security subsystem
- It must be searchable through the GeoNode metadata model
We don’t want to change the GeoNode core model in order to add a new custom resources. Those “Geo Applications” must be somehow “pluggable” into GeoNode by adding external dependencies.
It is not possible, at this stage, a complete refactoring of the GeoNode model and APIs v1. Moreover we want those changes to be compatible with old GeoNode versions that, hopefully, should be upgradable to the new version without data losses.

Proposed By

@afabiani
@giohappy
@allyoucanmap

Assigned to Release

This proposal is for GeoNode 3.2.

State

[x] Under Discussion
[x] In Progress
[ ] Completed
[ ] Rejected
[ ] Deferred

Proposal

Given the prerequisites depicted above, the technical proposal is to introduce into GeoNode the concept of “Geo Application” (aka GeoApp).

A GeoApp in GeoNode, will be a container integrated into the core model able to be queried through both the standard (Tastypie) and REST APIs frameworks of GeoNode.

The GeoApp module, would be a new application of the GeoNode core project, containing the base abstract classes allowing any external plugin to define as many concrete applications they need.

Every GeoApp concrete instance, must define a unique “GeoApp Type” which will be translated into a polymorphic_ctype allowing GeoNode to seamlessly manage the concrete models and distinguish the GeoApp instances between the common ResourceBases and other GeoApp instances.

By default, GeoNode will enable a new menu on the navigation toolbar, whose name could be customized through the settings.

This new section will allow admins to present on the frontend all the GeoApp instances plugged into GeoNode.

It will also be possible to disable such menus, allowing the developers to define their own customized menus, each for every GeoApp concrete instance.

GeoNode Core Model: GeoApp django application

The new GeoApp model, extends the ResourceBase model by adding few more generic fields for any further GeoApp implementation:

Geospatial fields, like, projection, zoom and center coordinates. The BBOX polygon is already part of the base model.
data; a generic JSON blob field which will contain any kind of raw configuration associated with the app.

GeoApp Model

...

class GeoApp(ResourceBase):
    """
    A GeoApp it is a generic container for every client applications the
    user might want to create or define.
    """

    PERMISSIONS = {
        'write': [
            'change_geoapp_data',
            'change_geoapp_style',
        ]
    }

    name = models.TextField(_('Name'), unique=True, db_index=True)

    zoom = models.IntegerField(_('zoom'), null=True, blank=True)
    # The zoom level to use when initially loading this geoapp.  Zoom levels start
    # at 0 (most zoomed out) and each increment doubles the resolution.

    projection = models.CharField(_('projection'), max_length=32, null=True, blank=True)
    # The projection used for this geoapp.  This is stored as a string with the
    # projection's SRID.

    center_x = models.FloatField(_('center X'), null=True, blank=True)
    # The x coordinate to center on when loading this geoapp.  Its interpretation
    # depends on the projection.

    center_y = models.FloatField(_('center Y'), null=True, blank=True)
    # The y coordinate to center on when loading this geoapp.  Its interpretation
    # depends on the projection.

    urlsuffix = models.CharField(_('Site URL'), max_length=255, null=True, blank=True)
    # Alphanumeric alternative to referencing geoapps by id, appended to end of
    # URL instead of id, ie http://domain/geoapps/someview

    data = models.OneToOneField(
        "GeoAppData",
        related_name="data",
        null=True,
        blank=True,
        on_delete=models.CASCADE)
...

class GeoAppData(models.Model):

    blob = JSONField(
        null=False,
        default={})

    resource = models.ForeignKey(
        GeoApp,
        null=False,
        blank=False,
        on_delete=models.CASCADE)

GeoApp URIs

The generic GeoApp URIs will allow us to query and access the instances from GeoNode. Those can also be overridden for specific apps if needed.

js_info_dict = {
    'packages': ('geonode.geoapps', ),
}

apps_list = register_url_event()(TemplateView.as_view(template_name='apps/app_list.html'))

urlpatterns = [
    # 'geonode.geoapps.views',
    url(r'^$',
        apps_list,
        {'facet_type': 'geoapps'},
        name='apps_browse'),
    url(r'^new$', views.new_geoapp, name="new_geoapp"),
    url(r'^preview/(?P<geoappid>[^/]*)$', views.geoapp_detail, name="geoapp_detail"),
    url(r'^preview/(?P<geoappid>\d+)/metadata$', views.geoapp_metadata, name='geoapp_metadata'),
    url(r'^preview/(?P<geoappid>[^/]*)/metadata_detail$',
        views.geoapp_metadata_detail, name='geoapp_metadata_detail'),
    url(r'^preview/(?P<geoappid>\d+)/metadata_advanced$',
        views.geoapp_metadata_advanced, name='geoapp_metadata_advanced'),
    url(r'^(?P<geoappid>\d+)/remove$', views.geoapp_remove, name="geoapp_remove"),
    url(r'^(?P<geoappid>[^/]+)/view$', views.geoapp_edit, name='geoapp_view'),
    url(r'^(?P<geoappid>[^/]+)/edit$', views.geoapp_edit, name='geoapp_edit'),
    url(r'^(?P<geoappid>[^/]+)/update$', views.geoapp_edit,
        {'template': 'apps/app_update.html'}, name='geoapp_update'),
    url(r'^(?P<geoappid>[^/]+)/embed$', views.geoapp_edit,
        {'template': 'apps/app_embed.html'}, name='geoapp_embed'),
    url(r'^(?P<geoappid>[^/]+)/download$', views.geoapp_edit,
        {'template': 'apps/app_download.html'}, name='geoapp_download'),
    url(r'^', include('geonode.geoapps.api.urls')),
]

Notice that the views templates will invoke the client_tag_library tags in order to dynamically render the final template. Such mechanism allows the plugged-in client library to, eventually, override the templates by editing the hooksets.

As an example, the app_list template will be defined as follows:

{% load i18n %}
{% load base_tags %}
{% load client_lib_tags %}

{% block head %}
    <style>
        #paneltbar {
            margin-top: 90px !important;
        }
    </style>

    {% get_geoapp_list %}
{% endblock %}

The generic client hooksets are defined as below instead:

    # GeoApps
    def geoapp_list_template(self, context=None):
        return 'apps/app_list_default.html'

    def geoapp_detail_template(self, context=None):
        return NotImplemented

    def geoapp_new_template(self, context=None):
        return NotImplemented

    def geoapp_view_template(self, context=None):
        return NotImplemented

    def geoapp_edit_template(self, context=None):
        return NotImplemented

    def geoapp_update_template(self, context=None):
        return NotImplemented

    def geoapp_embed_template(self, context=None):
        return NotImplemented

    def geoapp_download_template(self, context=None):
        return NotImplemented

The client library must implement the methods above in order to return its own templates, specific to the GeoApp implementations (see later how the mapstore-client library can plug in the GeoStories to GeoNode).

GeoApp REST APIs (v2)

:warning: Depends on GNIP-79: GeoNode REST APIs (v2)

Similarly to the models and client tags, the REST APIs too will be fully pluggable. The generic GeoApp application, will expose a set of predefined and generic api endpoints along with JSON serializers. Those ones could be overridden and extended in order to expose the APIs for the specific GeoApp instance.

The GeoApp attaches its own generic endpoint to the GeoNode REST API default router.

from geonode.api.urls import router

from . import views

router.register(r'geoapps', views.GeoAppViewSet)

To complete the bundle, the GeoApp generic app will provide a set of predefined viewsets and serializers, which can be eventually extended by the specific implementation.

Use Case: MapStore GeoStories

ℹ️ demo instance available at https://dev.geonode.geo-solutions.it/

The MapStore client library for GeoNode, currently implements the concept of “GeoStory” as a concrete instance of the GeoApps.

GeoStories App Model

As explained above the library should override and extend several methods and templates in order to achieve that.

First of all, we will need to include the MapStore library extension as part of the GeoNode INSTALLED_APPS bundle.

"""
To enable the MapStore2 REACT based Client:
1. pip install pip install django-geonode-mapstore-client==1.0
2. enable those:
"""
if GEONODE_CLIENT_LAYER_PREVIEW_LIBRARY == 'mapstore':
    GEONODE_CLIENT_HOOKSET = os.getenv('GEONODE_CLIENT_HOOKSET', 'geonode_mapstore_client.hooksets.MapStoreHookSet')

    if 'geonode_mapstore_client' not in INSTALLED_APPS:
        INSTALLED_APPS += (
            'mapstore2_adapter',
            'mapstore2_adapter.geoapps',
            'mapstore2_adapter.geoapps.geostories',
            'geonode_mapstore_client',)

The new model must extend the GeoApp one and define a concrete app_type which will be translated by GeoNode as a new polymorphic_ctype.

from django.utils.translation import ugettext_noop as _
from geonode.geoapps import GeoNodeAppsConfig

class GeoStoryAppsConfig(GeoNodeAppsConfig):

    name = 'mapstore2_adapter.geoapps.geostories'
    label = "geoapp_geostories"
    default_model = 'GeoStory'
    verbose_name = "GeoNode App: GeoStory"
    type = 'GEONODE_APP'

    NOTIFICATIONS = (("geostory_created", _("GeoStory Created"), _("A GeoStory was created"),),
                     ("geostory_updated", _("GeoStory Updated"), _("A GeoStory was updated"),),
                     ("geostory_approved", _("GeoStory Approved"), _("A GeoStory was approved by a Manager"),),
                     ("geostory_published", _("GeoStory Published"), _("A GeoStory was published"),),
                     ("geostory_deleted", _("GeoStory Deleted"), _("A GeoStory was deleted"),),
                     ("geostory_comment", _("Comment on GeoStory"), _("A GeoStory was commented on"),),
                     ("geostory_rated", _("Rating for GeoStory"), _("A rating was given to a GeoStory"),),
                     )

default_app_config = "mapstore2_adapter.geoapps.geostories.GeoStoryAppsConfig"

class GeoStory(GeoApp):

    app_type = models.CharField(
        _('%s Type' % settings.GEONODE_APPS_NAME),
        db_column='geostory_app_type',
        default='GeoStory',
        max_length=255)
    # The type of the current geoapp.

That’s it, there’s not much more to do here, unless we need to further specific fields to our model. Currently MapStore is able to store the whole GeoStories configuration as a JSON blob into the schema, therefore the data field of the generic app model is more than enough.

GeoStories Client Hooksets

We will need to render the GeoStories list, details and view templates. As explained before, this can be easily done by implementing the GeoNode client library generic Hooksets methods.

    # GeoApps
    def geoapp_list_template(self, context=None):
        self.initialize_context(
            context,
            callback=ms2_config_converter.convert)
        return 'geonode-mapstore-client/app_list.html'

    def geoapp_new_template(self, context=None):
        self.initialize_context(
            context,
            callback=ms2_config_converter.convert)
        return 'geonode-mapstore-client/app_new.html'

    def geoapp_view_template(self, context=None):
        self.initialize_context(
            context,
            callback=ms2_config_converter.convert)
        return 'geonode-mapstore-client/app_view.html'

    def geoapp_edit_template(self, context=None):
        self.initialize_context(
            context,
            callback=ms2_config_converter.convert)
        return 'geonode-mapstore-client/app_edit.html'

    def geoapp_update_template(self, context=None):
        self.initialize_context(
            context,
            callback=ms2_config_converter.convert)
        return 'geonode-mapstore-client/app_update.html'

    def geoapp_embed_template(self, context=None):
        self.initialize_context(
            context,
            callback=ms2_config_converter.convert)
        return 'geonode-mapstore-client/app_embed.html'

The MapStore client library can now define its own templates by injecting all the specific JavaScript code needed to manipulate a GeoStory.

GeoStories REST APIs v2

:warning: Depends on GNIP-79: GeoNode REST APIs (v2)

The final step is to plug GeoStories specific APIs to the GeoNode router. This requires a few more steps.

Since the MapStore client and adapter are dynamically plugged into GeoNode, we will need to declare at initialization time which new Django urls we want to include into the resolver.

This is possible by including them from the app __init__ class of the MapStore Adapter.

def run_setup_hooks(*args, **kwargs):
    from geonode.urls import urlpatterns
    from django.conf.urls import url, include

    urlpatterns += [
        url(r'^mapstore/', include('mapstore2_adapter.urls')),
        url(r'^', include('mapstore2_adapter.geoapps.geostories.api.urls')),
    ]


class AppConfig(BaseAppConfig):

    name = "mapstore2_adapter"
    label = "mapstore2_adapter"
    verbose_name = _("Django MapStore2 Adapter")

    def ready(self):
        """Finalize setup"""
        run_setup_hooks()
        super(AppConfig, self).ready()

In this way we don’t need to touch GeoNode at all. Everything will dynamically plug in at initialization time.

Now, we can easily define a new api model for the GeoStories which will eventually extend the generic viewsets and serializers provided by the GeoApp module of GeoNode.

from geonode.api.urls import router
from mapstore2_adapter.geoapps.geostories.api import views

router.register(r'geostories', views.GeoStoryViewSet)

urlpatterns = [
    url(r'^api/v2/', include(router.urls)),
]

Backwards Compatibility

Compatible with GeoNode 3.x and above, by using the new bbox model provided by Catalyst.

Future evolution

More concrete GeoApp implementations, like MapStore Dashboards

Feedback

none

Voting

Project Steering Committee:

Alessio Fabiani: 👍
Francesco Bartoli:
Giovanni Allegri: 👍
Simone Dalmasso:
Toni Schoenbuchner: 👍
Florian Hoedt: 👍

Links

enhancement gnip

Source

afabiani

👍1

All 11 comments

We will use a mixture of MapStore2 Apps like Dashboard, MapStories and there like and 'third party' applications completely decoupled from GeoNode like shiny applications running on their own server. The third-party applications will most likely just hold some hyperlink to get there but should have some of the ResourceBase metadata info to get them displayed in searches. Since those third-party applications would be out of the scope of the GeoNode system, authentification could not be used - or has to be done by OAuth scopes (TBD).

:edit: To get other developers use this generic model (eg. cartologic), we could provide documentation on how to built a simple generic app.

In the long run, we think about containerizing those third-party applications and host them as part of the GeoNode stack. But that's far-far-away.

gannebamm on 3 Dec 2020

@gannebamm as you will know we already have several options for AuthN of third party apps:

Use an external Oauth2 server. We did it for a client, which needs to authenticate in GeoNode from QGIS using Oauth2 provided by their WSO2 instance. Here GeoNode is configured to "hop" the authentication to WSO2 and create an authenticated session on top of it.
GeoNode already offers on Oauth2 provider, which is internally used by Geoserver to exchange access tokend with GeoNode.
One of our customers is using this support to obtain access tokens for a third party app of them. They registered this app along with Geoserver inside GeoNode, and it seems it is working perfectly.
On the base of this use case, I want to investigate the support for refresh tokens.

Going in the direction of using GeoNode as a REST backend for multiple third party apps, even served from different domains, the documentation for these scenarios should be extended.
And probably there will be some more work to make all this work as smooth as possible...

giohappy on 3 Dec 2020

👍1

Cartoview related to this GNIP

is probably covering all the issues suggested in this GNIP already.
is very easy to install as an app with any running GeoNode
it can also be included by default as part of the Geonode install
already offers a much higher entry point to work with Apps as per the GNIP proposal and it is ready to use now

Cartoview is managing a couple of app types:
App Type A) Resource based, like dashboards, storymaps, viewers etc
App Type B) Function based or business based, like data loaders, data processors, disaster management and so on. In this type there is no resource created it just a single app instance

Additionally Cartoview is offering app management from the UI, start, stop, suspend apps and so on
There is also an app store for deploying apps at run time which is also configurable if you want to use your own
It is good for System Admins, you can maintain the apps without requiring a developer to install or upgrade apps

Having a base App or Generic App type is kept to a minimum, for example a map catalog app, training material app, a pdf maps app are resource based apps however it does not require properties related to maps or layers so you need to keep the app definitions to the bare minimum (just enough to be a resource) which is what CartoView is doing

If the app requires geographic properties like starting extent and layers, then use a map or a number of maps to define the app properties, a number of app property definitions are already available as definition wizards to ease the app development process

deploying story maps or dashboards based on mapstore should be very simple already with Cartoview

Before embarking on this GNIP I suggest spending sometime evaluating the capabilities of CartoView,

suggesting how to make CartoView better
evaluating best integration scenarios with GeoNode

Looking forward to hear back from you all

ahmednosman on 4 Dec 2020

Dear @ahmednosman the only way to evaluate CartoView apps it to propose a PR to GeoNode.

afabiani on 4 Dec 2020

@ahmednosman the idea behind CartoView is instereseing, and it is pretty similar at its minimum to the concept that we're proposing here. The key differences are:

this proposal wants to integrate the "Apps" concept with a lightweight support directly inside GeoNode Core
the genarl plan is to make GeoNode Core headless, so without templates to drive its functionalities. The transition will be slow, and probably some admin UI will remain for the time being. Functionalities like Cartoview's App manager could be provided by an external module which could use the core App utilities
as far as I can see CartoView currently doesn't support GeoNode 3.x

In brief, integrate the basic stuff in the core and let everyone build fantastic appls ontop of it :)

giohappy on 4 Dec 2020

👍1

@giohappy

CartoView is supporting 3.x will be released shortly https://summit2020.cartoview.net/ you can see it here
if you are going with a GeoNode Core that is headless. How do you envision having an app framework integrated into the headless core?
This will require the apps to be split deploying the app API inside the core and the App UI somewhere else, correct?
How about we integrate the app framework or CartoView as a registered app with GeoNode the same way like GeoServer is integrating with GeoNode, it will be a unified login across GeoNode/GeoServer/CartoView all we have to do here is create a Generic GeoAPP Model in GeoNode for the apps to be searchable and treated as a resource but the app will infact run on a separate server this will bring a lot of flexibility, you will have a minimum geonode core and you will not have a lot of dependancies related to apps to release.

ahmednosman on 4 Dec 2020

By the way if you are using headless core GeoNode, a number of different User interfaces will be built by the community from single page applications to geoportal to an Enterprise GIS
What will make good practice is to write up and SDK/framework/template that will allow the initiation of such work for the developers to start building apps, frameworks, mobile etc

ahmednosman on 4 Dec 2020

@ahmednosman we have two distinct conecpts here:

what we're proposing with the "core" Apps concept is to support GeoNode plugins. The natural context for thie is the GeoNode Project, where the core GeoNode and additional custom apps and plugins can be hosted.
what you're saying is more geared towards third party apps. This is a scenario that we want to support more and more by:
- extending the REST API further, and make GeoNode really API-first, so that third party apps can leverage all GeoNode features. The plan is to extend CRUD methods, so that allowed third party apps can fully interact with GeoNode models
- clarifying and improving AuthN (and AuthZ) support for third party apps. Here we already have some options, including Oauth2 for external apps, but this opportunities have never been treated as "first citizens". Now it's time to elevate them to foundamental components for GeoNode's apps echosystem

From our side we are only committed to support GeoNode core development, and related critical components (geonode project, Geoserver integrations, etc.)
I really wish you will be able to contribute to GeoNode core on this side.

giohappy on 4 Dec 2020

🚀1

I think we are in agreement, let there be an API first GeoNode with apps treated as resources
As such I suggest to limit the proposal of the GNIP to the APP API implementation only.

Referring to the case study GeoStories APP can be deployed as an app outside the GeoNode Core
The App will appear as a resource listing in GeoNode with meta data and search etc
MapStore library extension or other such libraries should not be included in Core GeoNode

You can consider the following parameters

App redirection URL upon successful login
App type: Mobile, Browser, Server, Desktop
App key: invoked by the APP owner

correction: CartoView is already released for GeoNode 3
https://github.com/cartologic/cartoview/releases/tag/v1.30.0

ahmednosman on 4 Dec 2020

@ahmednosman the proposal is already taking into account generic APIs and models.

No specific app will be included into the GNIP. The concept is exactly this one, creating a structure in geonode allowing to plugin any concrete apps implementation from outside geonode.

The "MapStore GeoStory" use case would like to demonstrate how it could be possible to plug in a concrete app from an external package (that said MapStore Client library).

About the cartologic tag, nice, but, again, if you want to backport and include any stuff into GeoNode core you will need to follow the contribution agreement as established by the community rules, i.e.:

Propose a GNIP along with description and technical details
Discuss the GNIP and ask for PSC votes
Attach a PR to the GNIP with the implementation so that developer can inspect and evaluate the code
Make sure there are both documentation and test cases

afabiani on 4 Dec 2020

After talking about this feature with some of my colleagues we thought that a simple 'hello world' example would be awesome. Something like an ultra-slim leaflet app maybe? @giohappy @afabiani Would this be possible? I think this would give the concept a big push.

gannebamm on 22 Dec 2020

Was this page helpful?

0 / 5 - 0 ratings