Redash: Add front-end extension capability

[emtwo]

This is similar in concept to https://github.com/getredash/redash/pull/2354/files

[arikfr]

Some thoughts to start the discussion:

1.

We already have some dynamic functionality around adding custom pages, visualizations and components by auto loading any file in client/app/{components,pages,visualizations,services} and executing the exported function (while passing the app module to it).

webpack's require.context function can't take a variable for a path, but we can use webpack's alias configuration to be able to specify additional folder to load code from in a similar way. This folder path can be defined using an environment variable, so we no longer have to put it in the existing folder structure (which gets overridden when deploying a new version).

1. The above will allow for adding new functionality to Redash (like new pages/components/visualizations), but we also need extension points in the code, to be able to modify and extend existing functionality.

This is what we did in the Policy pull request. There are two types of extensions there:

• The Policy object which you can replace with a different implementation, that allows you to set custom logic around whether certain operations are permitted.
• References to non existing components (like <users-list-exta> here) which you can implement to show additional information in various places in the application. We can add similar "extension points" to implement some of the custom UI Mozilla has (like data source version or documentation link).

This is what I have so far, some additional half baked thoughts I have:

• It might be interesting to have the ability to load extensions at "runtime" (specially visualizations), but I'm not sure how useful that can be.
• It can be useful to be able to replace components with a different implementation. Not sure how to achieve this though.

Also we need to keep in mind that any solution we implement, needs to keep working when we migrate to React...

[arikfr]

I stumbled at flask-webpackext, which in their README lists the following as one of the features:

Collect bundles: If you Webpack project is spread over multiple Python packages, Flask-WebpackExt can help you dynamically assemble the files into a Webpack project. This is useful if you don't know until runtime which packages are installed.

Might be interesting for us as well.

[jezdez]

@arikfr Interesting indeed since that'd allow us to ship bundle configurations based on Python entrypoints and have that bridge to the backend extensions that we're looking for. Behind the scenes this uses pywebpack and the flask package is just the integration layer for Flask best practices/patterns.

In fact there seems to be a stub already to do the entrypoint based loading there. Just not used anywhere it seems. This smells like not fully baked but maybe we can talk to @lnielsen and see if he's able to shed some light for the plans of the library?

[lnielsen]

I'm happy to provide more info. https://getindico.io is using the plugin, and next week we are starting a three-week sprint to integrate it into Invenio, and push the pynpm, pywebpack and flask-webpack out into final releases.

[lnielsen]

We have an initial prototype of the implementation available here

There's also the bundle test case

E.g. to load bundles from entry points:

setup(
    entry_points={'my_bundles':['module.bundles:abundle']}
)

#...
project = WebpackBundleProject(
       # ...
        bundles=bundles_from_entry_point('my_bundles')
    )

[jezdez]

Thanks @lnielsen, that looks super useful, thanks!