Horizon: Plugins

[bopjesvla]

Reading through the discussion on document validation, it seems to me that if everything is implemented as a fixed API, horizon will only be used by people with specific use cases.

I don't think document validation should be part of the core, for the simple reason that no one solution will suit every app. You might be thinking about storing the schemas in the database. What if I want to validate login information before the web socket is opened? The schemas might be read from JSON files. What if I want to use YAML? You might pick ajv as the json-schema implementation because it's fast. What if I want to use jsen because it's three times as small? What if my app doesn't need any fancy APIs whatsoever, and I'm just using horizon for the realtime feeds?

---

This is what I imagine a document validation plugin would look like:

let validate = require("json-something").validate
module.exports = function(hz, opts) {
    let schemas = hz("schemas").fetch();
    let validate = (table, doc) => validate(schemas[table], doc)
    hz.on("insert", validate)
    hz.on("update", validate)
}

An auth plugin:

module.exports = function(hz, opts) {
    let users = hz("users");
    if (!users) {
        // initialize
    }
    hz.on("connect", credentials => users.findOne({user: credentials.user, password: credentials.password}))
}

Plugin configuration would be done in config.toml and/or a configuration module like webpack's config.js.

[deontologician]

345 is also related to freeform customization of the server.

Ultimately, we have the fallback of allowing the user complete control by importing horizon as a library. I think if you're interested in the different between size and speed of json-schema parsing libraries, then you're in the realm of writing your own custom backend. The default backend should focus on working for 80% of use cases and allowing the user to take over once it isn't sufficient

[bopjesvla]

I think if you're interested in the different between size and speed of json-schema parsing libraries, then you're in the realm of writing your own custom backend.

That particular comment was about document validation on the client. And yeah, it wasn't the best example. The first and the last one are more important.

A slightly more concrete proposal that depends on #345:

• Every X in [plugins.X] in config.toml is required and called with the horizon instance and config options. This can also be done programmatically from server.js.
• The horizon server exposes events which plugins and code in server.js can hook into.
• The client also exposes things like insert hooks, which client-side code can hook into.

I'll expand on why I think auth and validation should be implemented as plugins later, but I'd like to hear your thoughts on this first.

[bopjesvla]

Disorganised notes:

• Unofficial plugins have the luxury of making assumptions about what stack they're used in.
• Lacking that luxury, horizon's APIs are forced to make assumptions about the nature of the app they're used in.
• The most simple use case for horizon is public real-time display of data, which requires authentication nor document validation.
• The auth API is going to have to provide a way for custom validation logic. *CAPTCHA validation and checking against an IP blacklist are examples of use cases. If custom validation logic is implemented, the auth API will be functionally the same as an onConnect hook with a preset for OAuth.
• "If you can't do it with our APIs, write your own backend" kinda feels like a cop out. The thing that makes horizon special isn't the auth API or the document validation API, it's the data system. If the data system works for me, it shouldn't matter how obscure my OAuth provider is.
• There's no reason a hypothetical plugin-supporting core can't accommodate to the current implementation of auth and document validation. Decouple everything that can be decoupled. If some aspect of either API is tied to the core in an inconvenient way, keep it there for the time being. Hell, you don't even have to document the available hooks while the API is in flux. Just allow folks who are willing to maintain a plugin by looking at how your plugins work to do just that.
• You already have a nice monorepo so you can keep everything in one place.

[bopjesvla]

Two more things:

• If you plan on turning horizon into an extendable data system, it's best to do it before the first release. If hz gets known as a platform apps eventually grow out of, it's going to be hard to change that perception.
• It'd be nice if plugins could be configured in the web UI.

[danielmewes]

Linking the work-in-progress RFC for Horizon plugins: https://github.com/rethinkdb/horizon/pull/588