Hugo: Support plugins for extending funcMap

[natefinch]

Note: I would be willing to implement this if others think it is valuable

In working with another generator, I needed a function that didn't exist in the funcMap, and adding it required a PR to the generator. It occurred to me that it would be very easy to implement a plugin system to allow users to implement their own functions in the funcMap.

Here's my suggestion - plugins are very simple executables that follow this pattern:

If a plugin executable is run with no arguments, it should output to stdout a newline delimited list of function names. Hugo would run all the plugins[1] at startup and keep a reference of which plugin produced which function name, for when they're called from a template.

When a template calls one of the plugin's functions, the plugin executable is run with with the function name as an argument, and any further arguments are sent as a json array to the plugin's stdin[2]. The result of the program should be json formatted and written to stdout. If an error is encountered, the executable should exit with a non-zero exit code, and write any error text to stderr.

Bam, now any user can create a plugin in any language they want to extend the functionality of Hugo to support any function they want.

[1] How plugins are discovered by Hugo is left to the developer. I don't really care if it's a magic directory or specially named executables in the $PATH or whatever.

[2] We do this to avoid OS character limits on executable arguments.

[digitalcraftsman]

It would be a very powerful feature for users to be able to extend the funcMap with custom functions.

But from the POV of a theme maintainer I would be facing the following problems:

Plugin distribution: Most themes are publicly accessible as a Git repository which users should only have to clone in order to use it. As themes should work out of the box it would be necessary to track executables in a CVS as well unless there is some dead simple package manager.

Furthermore, executables are mostly platform dependent. Writing a plugin in Go isn't hard but I would have to provide executables for all major platforms (incl. 32 and 64 bit versions). This would create even more bloat.

Versioning: Plugins can change over the time, e.g. through the addition of new features or bug fixes. Sometimes a piece of software can change it's behavior which might cause errors. This makes it harder to detect errors because you can say plugin xx with version 1.xxx is not compatible with the current state of the theme.

[natefinch]

This is a very interesting point. I hadn't thought of this from the vantage of requiring a plugin for a theme, but that does make sense. I'd just been thinking about hacking up an existing theme and needing different functions for it.

Of course, if it's a script (python or otherwise) it could be included with the theme, but that does then require that the host have python installed, which is a problem for Windows users.

I wonder if something like embedded lua would be a way to avoid that - https://github.com/yuin/gopher-lua of course, then you have to write the functions in lua :/

[bep]

Doing template func plugins via os/exec sounds tempting and simple, but there are lots of issues floating to surface once you start thinking about it (performance, security, distribution, support). I have always thought about potential plugins in Hugo on a more coarse-grained level; piping a template string into a python trim func does not sound very fast for 10K pages.

Embedded Lua does sound interesting and is the way a big project like Wikipedia has gone with their end-user provided scripts.

[digitalcraftsman]

Using an embedded language might be the sweet spot. Distributing executables is a hassle due to the vast amount of platforms and unflexible. Interpreted languages like Python would require a runtime environment.

Embedded languages also might be easier to use than RPC-based plugins (see #1239).

While glancing over the README of the gopher-lua repo I saw the DoFile method to load scripts. It would be neat to have scripts as simple text files in a special directory which is watched by Hugo. This way functions could be added, removed and updated ad hoc.

However, I don't know how goper-lua will perform / slow down Hugo, nor how much the Hugo executable would grow. @bep also mentioned other aspects that have to be considered as well.

While this is just wishful thinking and definitely needs a more precise specification it could go beyond template functions.

Edit: Just toyed around a bit with gopher-lua. It looks pretty good and reasonable fast. But that has of course to be benchmarked :wink:

[natefinch]

FYI, we just built a plugin system like this for Gnorm: https://gnorm.org/templates/plugins/

It's basic now, just running an executable in a specified directory. But I intend to enhance it so it can understand how to run common languages like .py .rb .go etc... yes it'll require that you have that language on your machine, but it's a lot easier to implement this than embedding a whole new language in your binary, and requiring people to write plugins in that language.

Not that I wouldn't also like lua in Gnorm, it's more work.

[natefinch]

Hmm... skylark might be better (python is more popular than lua). Could be distributed as plaintext with a theme.

https://github.com/google/skylark
https://godoc.org/github.com/google/skylark#ExecFile

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.
This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

[sijad]

this issue should be reopened, i need a way to format date to jalali, embeded script seems a very good solution.

see #5510

[natefinch]

POC here: #5521

[moorereason]

@sijad, create a separate issue for your calendar/date issue.