Tiddlywiki5: Call for thoughts on improving plugin handling

[Jermolene]

This ticket is a call for input. I'll update this post with any suggestions

The work for adding support for dynamic loading/unloading of plugins in #4259 requires touching many parts of the core concerned with plugin handling. I'd like to take the opportunity to address a few other long standing issues/limitations that are in the same area.

For example, in the control panel plugin listing and plugin library we need to add an indication of whether a plugin contains JS modules and hence will require a restart if installed or removed. I'd like to take the opportunity to address one of the concerns in Improve the plugin dropdown information by elevating the portion of the title after the final slash to be the shorthand name of a plugin.

Other proposals:

• In client/server configuration, hide the official plugin library from $:/ControlPanel; it isn't the recommended way to install plugins so that they are available in both the browser and the server

Existing issues that might be worth revisiting:

• Support plugins/publisher in wiki folders
• Move plugin library from modal to tiddler
• Proposal: The $ControlPanel provides possibilities for plugins to promote important content

[pmario]

@Jermolene ... There is something wrong with your links.

[Jermolene]

Thanks @pmario

[twMat]

• [quote] "Move plugin library from modal to tiddler"
  +1 ...and make it take substantially fewer steps to reach it.
  ...and, related, prepare plugin library for Federation i.e to show multiple libraries.
• I'd like a simple plugin packer. Drag'n drop titles onto a dropzone and they become a plugin. Plus some additional post-editing options (remove, add, edit shadows).
• I want update alerts, i.e to know if I have the latest version of a plugin ideally both when I look at the plugin list but also alerts
• (See image below)
  
  
  
  1. The open/close slider is unclear and easy to miss. One idea is to put the arrow to the left of the box shadow... (and make the button have a border itself?)
  
  
  2. The "technical" path title should not be seen here, it should be when the slider is opened
  
  
  3. Version should be much bigger
  
  
  4. It is confusing that clicking on this area is the same as clicking on the title. Clicking on this area should possibly open the slider, if anything. Only by clicking the blue link should one go to the actual tiddler.
  
  

[twMat]

ref

Harmonize appearance of plugin library titles with local listing of titles, i.e

vs

[Jermolene]

Thanks @twMat

Move plugin library from modal to tiddler

One can open $:/core/ui/ControlPanel/Modals/AddPlugins directly in the story river. The reason why I made it a modal was to make it harder for people to confuse the plugin listing with the plugin chooser. They look almost exactly the same (the listing is blue because each box is a link, while in the plugin chooser everything is black because we can't link to plugins that haven't been installed).

I'd like a simple plugin packer. Drag'n drop titles onto a dropzone and they become a plugin. Plus some additional post-editing options (remove, add, edit shadows).

With the support for dynamic plugins plus a few more action widgets anyone will be able to write a plugin packer. Eventually we should definitely do it in the core.

I want update alerts, i.e to know if I have the latest version of a plugin ideally both when I look at the plugin list but also alerts

Good idea.

(See image below)
[snip]

Great feedback, thanks.

[twMat]

The reason why I made it a modal was to make it harder for people to confuse the plugin listing with the plugin chooser.

I had not considered this. But things requiring scroll often feel claustrophobic in modals. Firefox approaches it this way (BTW - anybody knows how to make images smaller than this?):

Installed add-ons:

"Plugin library"

I like the bold blue buttons. Very clear indication IMO. I also like the "..." for a small popup menu to e.g disable the plugin.

---

With the support for dynamic plugins plus a few more action widgets anyone will be able to write a plugin packer. Eventually we should definitely do it in the core.

If plugins are a way to protect tiddlers and make them manageable as a unit between wikis, I think there's reason to consider it as a native feature (or an add-on). It would also further simplify sharing in the community. One idea would be to use a system tag to create plugins like so: $:/tags/plugin/myplugin. The prefix $:/tags/plugin triggers "bundling" of all tiddlers tagged such.

[morosanuae]

@Jermolene I don't know if what I'm going to say is related to the current issue but I'm interested mainly in having a single central repository/library for users made plugins, something similar to Notepad++. Currently it's very hard to find plugins made by users, it's more like a matter of luck.

[Jermolene]

Hi @morosanuae

@Jermolene I don't know if what I'm going to say is related to the current issue but I'm interested mainly in having a single central repository/library for users made plugins, something similar to Notepad++. Currently it's very hard to find plugins made by users, it's more like a matter of luck

We should discuss in a separate issue. The issues are more logistic than technical: the plugin library is capable of serving third party plugins, but we don't have policies and procedures for accepting contributions.

[Jermolene]

Here's where I am with the plugin chooser:

@twMat I'd also been thinking that perhaps the plugin chooser should diverge from the plugin listing. In particular, I'm interested in adding more metadata (like screenshots) to the plugin chooser.

[twMat]

How about this (top=current, bottom=suggestion). I think it looks better and is easier to understand.

[Jermolene]

Thanks @twMat I was concerned that moving the arrow to the bottom will consume vertical space to avoid overlaps which might make things look lopsided. I've experimented with changing to use a more prominent double chevron:

[twMat]

Thanks @twMat I was concerned that moving the arrow to the bottom will consume vertical space

If the version number remains where it is, I'd think there is space to have such a button float in the same vertical position. Also, ellipsis ("...") could be used which is a tad less vertically spaced.

A button on the left: I'm just afraid the user just won't understand it is a button. It looks more like decoration to me... a bit like the bullets in a bullet list.... but with the pointing-to-right saying "hey, look at this!" (pointing to the plugin icon)

Another aspect: The Palette chooser has a different and more subtle design that is cleaner more neutral. It makes sense to stick to one style for all item lists in the Ctrlpanel and perhaps especially so for the Plugin Library because the items there are not a buttons so there is no real reason to show them as square boxes.

Here's a design idea for the plugin listing. It is not the same design as the Palette chooser, but anyway:

[morosanuae]

@Jermolene and @twMat IMO the first suggestion made by Mat is the best (clearest), the one with the arrow/chevron in the bottom center. I would make it a full width button with some hover animation and make the icon smaller. I would make it just like the TiddlyWiki unfold tiddler bar.

[AnthonyMuscio]

I think sharing this hacking of the plugin mechanism may contribute to this thread.

I have started using plugin type info to capture tiddlers and make them into shadow tiddlers. This is not unlike the suggested opening up of the plugin mechanism. I note however that the fact a tiddler is named $:/plugin.. and has a plugin-type field is a way of differentiating plugins and related tiddlers, including to delete them from the wiki. If the mechanism is to be opened up we need to ensure the traditional plugins and other uses are well differentiated otherwise the plugin list will get long and it may be hard to differentiate.

Currently I package my info tiddlers by dropping one or more tiddlers or json files on my wiki, rename $:/import to another name, change the plugin type from import to info and remove any other fields from the new tiddler.

As a result I created a new Plugin tab to list info plugins.

The ability to avoid reload on installing a plugin with simplify the above hack.

Perhaps the idea of having plugin types of data/bundle or package, or something that we do not even call a plugin, that installs shadow tiddlers would be wise.

My use of such an info tiddler is published here as a demonstration to solve a wikis performance issue. The main design issue is list widgets may need to refer to shadow tiddlers.

[AnthonyMuscio]

Link added in above post https://tiddlywiki.psat.com.au/ESPDIC/

[AnthonyMuscio]

In the example at the above site, the Info plugin stores 36,000 word tiddlers. I believe we could provide a little more functionality to assist in plugin management. Most likely using an optional tool(s).

Provide a mechanism by which the subset of edited (no longer shadow tiddlers) can be;

• Optionally Deleted from the current wiki in a click
• Rolled back in to the "plugin" to replace the shadow tiddler below them and become the new shadow tiddler
• Exported as a set, for application after the plugin is updated, perhaps even step through and review differences.

All of the above suggestion are driven by a desire to build continuous improvement processes, development work flows and tools for integrating input from disparate wikis.

[twMat]

Another "thought on improving plugin handling" (which, as we note, is a pretty wide ranging question):

There are several problems when creating plugins that depend on yet other plugins:

• When creating and packing my plugin, I want to include the functionality of those other plugins but they are, obviously, separate entities. I wish they could be treated more like tiddlers in my plugin - without at the same time not mutilating them (out of respect for the original authors).
• When distributing my plugin, I don't want the "fetcher" to have to get multiple plugins (or even worse, go to multiple sources) - but again I don't want to mutilate them by, for example, adding a common tag to create a shared DnD tag pill. (One solution might be #2843 )
• Maybe "sub plugins" could have a separate info tab among the other plugin tabs (readme, contents, dependants) where proper credit and source links (automatically) appear.
• How should my plugin deal with updates in those subplugins?

[Jermolene]

I've just added a minor tweak in 92340d81ccca94cb3a866ab44c605e7d3e448779: an improved implementation @BurningTreeC's idea in #4109 from @BurningTreeC, hiding addon plugins under their parent:

Note that addon plugins still appear in the search results as usual:

[Jermolene]

In e752ba7c6 I've added support for automatically installing dependencies when installing a plugin. It uses the existing "dependents" field (which is a list) and the new "parent-plugin" field (which is a single title) to track the dependents that need installing. Note that it does not recursively look at the dependencies of dependencies (we're not rebuilding npm yet!), so plugins will have to list all of their dependencies themselves.

[AnthonyMuscio]

Jeremy,
As a result of this discussion could I suggest the ability to install plugins without reload and in fact the import process, be moved into plugins that come with the standard distribution but can be removed if desired by the designer?

This can help by simplifying some wikis published but also gives people the ability to remove them and thus reduce some possibilities of miss use.

I am happy to put this argument in more detail if it does not seem obvious.

[Jermolene]

Hi @AnthonyMuscio

As a result of this discussion could I suggest the ability to install plugins without reload and in fact the import process, be moved into plugins that come with the standard distribution but can be removed if desired by the designer?

There's a bit of a chicken and egg problem if we make the import mechanism be a plugin, because the import mechanism is how plugins are installed, so without it we can't install the plugin.

Note that importing can already be disabled in a published wiki (albeit it currently requires overwriting shadow tiddlers to get rid of the page dropzone widget).

[Jermolene]

Hi @twMat

There are several problems when creating plugins that depend on yet other plugins:

The dependency installation changes I committed yesterday addresses these problems. Plugins can list other plugins in their "dependents" field, and the plugin library will automatically install those plugins alongside itself.

This simple mechanism allows dependencies to be shared, and maintains the current update logic.

When creating and packing my plugin, I want to include the functionality of those other plugins but they are, obviously, separate entities. I wish they could be treated more like tiddlers in my plugin - without at the same time not mutilating them (out of respect for the original authors).

We can't really do that while still allowing those dependent plugins to be shared between several plugins that depend on them.

When distributing my plugin, I don't want the "fetcher" to have to get multiple plugins (or even worse, go to multiple sources) - but again I don't want to mutilate them by, for example, adding a common tag to create a shared DnD tag pill. (One solution might be #2843 )

The changes discussed above address the case of installing via the plugin library. For drag and drop installation, the publishing site should create a draggable that references all the required plugins.

Maybe "sub plugins" could have a separate info tab among the other plugin tabs (readme, contents, dependants) where proper credit and source links (automatically) appear.

Sounds a bit complicated.

[AnthonyMuscio]

There's a bit of a chicken and egg problem if we make the import mechanism be a plugin, because the import mechanism is how plugins are installed, so without it we can't install the plugin.

Understood, perhaps we can just have the equivalent. A switch that disables the import or drag and drop mechanism (reversible) and a switch that disables the "install plugins without reload?" Perhaps always saying save and reload as it currently does. The main thing is the designer can influence how the wiki will be used, even if strictly not enforceable.

In a similar vein,
I currently set the autosave mechanism off and hide the save changes button from the read only visitors so they are not disappointed, get errors when autosave fails or get errors hitting save. A simple switch would help here.

Perhaps all of these can be Control Panel Saving Options defaulting to the most permissive.

Note that importing can already be disabled in a published wiki (albeit it currently requires overwriting shadow tiddlers to get rid of the page dropzone widget).

Could we document this if not adding the above switch?

I plan to develop a method where if the wiki is opened by URL it takes on a read only behaviour, and if a special parameter is on the url it will open in a permissive state with autosave on with import and plugin without reload permitted. Of course the "right to write" to the host is independent.

These config issues could have even greater use with the local browser storage option to keep user state information. I would like to hide the save button when they do not need to save, because the only thing they can save is the state information in local storage.

Regards
Tony

[AnthonyMuscio]

Post script with the proposed "switches" they can be set according to conditions detected in Startup actions (The more generous startup actions plugin that is).

[Jermolene]

Hi @AnthonyMuscio

OK, I think you're suggesting several customisations that are already possible should be made available as switches in control panel so that they are easier to control.

There's a couple of up-front implications to adding settings to control panel that should always be borne in mind:

• Every entry in control panel adds exponentially to the complexity of the control panel from the perspective of the user
• All strings used in the control panel have to be pulled out as separate shadow tiddlers so that they can be translated

But I think a deeper problem is expressed by your understandable uncertainty about the precise behaviour of those switches. The fact that there are several plausible options means that any single choice we make is unlikely to satisfy all users.

Another way of saying this is that TiddlyWiki is customisable not via the MS Word route of a massive "options" dialogue with thousands of switches but from having a transparent, pluggable structure that can be understood and changed by end users.

[AnthonyMuscio]

@Jermolene

Having a transparent, pluggable structure that can be understood and changed by end users.

Totally!

Whilst such settings may make the control panel more complex their judicial use will help users make use of tiddlywiki in all its possible configurations more simple. These are not normally concerns of app or website users.

I have a production wiki in my client at present, and briefing an editor on how to use it is unnecessarily complex, because I cant (yet) use such settings to simplify it for them. Perhaps these could be hidden settings?

Do hidden settings need the same level of language support?

The most important issue is in my mind that the plugin mechanisms themselves support such settings internally or have a workaround, and they are documented.

[Jermolene]

Hi @AnthonyMuscio

Whilst such settings may make the control panel more complex their judicial use will help users make use of tiddlywiki in all its possible configurations more simple. These are not normally concerns of app or website users.

I'm pointing out what I believe are fundamental problems with repeatedly adding more settings to the control panel:

• That adding more settings adds to the complexity of the control from the user perspective, making it harder to understand and use
• That fixed settings that embody complex behaviour cannot satisfy the needs of all users (e.g. disabling autosave and hiding the save button, which I think you're suggesting should be a single setting)

I have a production wiki in my client at present, and briefing an editor on how to use it is unnecessarily complex, because I cant (yet) use such settings to simplify it for them. Perhaps these could be hidden settings?

Why can't you just give them a simplified UI that doesn't include the complexity you don't want? As a hack, I often just use CSS to hide things.

Do hidden settings need the same level of language support?

No, they do not. One of the motivations for having hidden settings is to keep the control panel simpler (they are all clearly documented on tiddlywiki.com)

[AnthonyMuscio]

By the way, A lot of plugs do not contain the source field and even fewer provide a github issues or communication channel. Can we improve this?

[twMat]

@Jermolene wrote

[...] adding more settings adds to the complexity of the control from the user perspective, making it harder to understand and use

Please don't rule out things/settings based on if they make the Ctrlpanel more complex. I believe it is already too complex so a totally different approach to it is warranted (and fully possible) as discussed here. I hope to make some demo soon but it should not be too difficult to merely imagine something like how e.g Chrome deals with it's settings (and search therefore)

[AnthonyMuscio]

Perhaps config tiddler idea we have discussed is a different thing to control panel settings. My Question about hidden only relates to plugin and save switches.

I and mat have discussed a generic way config tiddlers and there possible values can be set. Our idea is to collate all these config settings introduced by plugins and macro solutions into a single page within the control panel. What is more complex than a busy control panel is important settings existing everywhere and not able to be found without serious exploration in tiddlywiki. I don't recall how many times I am busy looking for that setting I really want to change and I do not recall what introduced it.

[Jermolene]

On reflection, let's try to keep this thread focussed on the issues around the plugin library. We can debate the control panel elsewhere.

[Jermolene]

Hi @AnthonyMuscio

Perhaps the idea of having plugin types of data/bundle or package, or something that we do not even call a plugin, that installs shadow tiddlers would be wise.

How would it differ from an ordinary plugin?

[AnthonyMuscio]

It would be like my current use of info plugins. Import / Rename the $:/import tiddler, removing status and changing plugin type to info. Then a new tab under plugins to list info "plugins". See this demo https://tiddlywiki.psat.com.au/ESPDIC/ where I packed 36,000 words in esparanto into a "plugin". Regards
Tony

[AnthonyMuscio]

Just working with CdK On something and using the Modloader plugin which may be relevant to this thread. https://evanbalster.com/tiddlywiki/formulas.html#Mod-Loader%20Plugin

[Jermolene]

Hi @AnthonyMuscio so do you mean you want a plugin whose shadow riddlers are not activated? You can do that already, but don't use plugin-type "info", that's already used by the core.

[Jermolene]

@pmario I think I recall you mentioning elsewhere that there were some unexpected dependencies between the CodeMirror add-on plugins? I might well be mis-remembering!

[pmario]

You are right. The different functionalities for CM have some dependencies. But we don't have a configuration option atm. ...

I was thinking about a little algorithm like this one to resolve the dependencies. ... https://gist.github.com/RubyTuesdayDONO/5006455

So every CM module should contain a little configuration for the minimum requirements, that is needed for it to work.

[Jermolene]

Hi @pmario I don't think we need to incorporate a dependency resolution algorithm, we just need to fill out the "dependents" field correctly for the CodeMirror add-on subplugins (see e752ba7c6).

[pmario]

That's probably true, if we define the parent-plugin field. Would be worth a test.

[AnthonyMuscio]

Is it as simple as disabling a plugin and listing its critical missing dependencies and a warning for not critical plugins?

basically should we differentiate dependencies.

Get Outlook for Androidhttps://aka.ms/ghei36

---

From: Mario Pietsch [e-mail]
Sent: Sunday, September 29, 2019 12:15:57 AM
To: Jermolene/TiddlyWiki5 [e-mail]
Cc: Anthony Muscio [e-mail]; Mention [e-mail]
Subject: Re: [Jermolene/TiddlyWiki5] Call for thoughts on improving plugin handling (#4263)

That's probably true, if we define the parent-plugin field. Would be worth a test.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/Jermolene/TiddlyWiki5/issues/4263?email_source=notifications&email_token=AHQ7DETGCULSHM2ZRSM6OYDQL5RJ3A5CNFSM4IW3YCAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD722UDY#issuecomment-536193551, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHQ7DEXNPKEUJZHBBQSVBLDQL5RJ3ANCNFSM4IW3YCAA.

[Jermolene]

Hi @pmario

That's probably true, if we define the parent-plugin field. Would be worth a test.

The parent-plugin is in use now, we use it to tie the CodeMirror sub-plugins to their parent.

The question remains: beyond the parent-plugin field (which is already filled in), what other interdependencies are there between the CodeMirror sub-plugins? @BurningTreeC can you illuminate this?

[Jermolene]

Hi @AnthonyMuscio

Is it as simple as disabling a plugin and listing its critical missing dependencies and a warning for not critical plugins?

basically should we differentiate dependencies.

I'm not sure what you mean, can you elaborate?

[BurningTreeC]

Hi @pmario

That's probably true, if we define the parent-plugin field. Would be worth a test.

The parent-plugin is in use now, we use it to tie the CodeMirror sub-plugins to their parent.

The question remains: beyond the parent-plugin field (which is already filled in), what other interdependencies are there between the CodeMirror sub-plugins? @BurningTreeC can you illuminate this?

Hi @Jermolene, I'm currently figuring out the interdependencies and I'll create a PR

[AnthonyMuscio]

@jermolene with code mirror there may be a parent and children arguably the child must have the parent installed, but the child is only optional for additional features

but in other plugins there may other plugins that are desirable and some essential.

Rather than have parent child relationships just have essential and desirable.

The plugin mechanisium could disable plugins without an essential and list them.

Then list missing desirables.

If people could submit plugin tiddler titles, source, essentials and desirables to a registry we could avoid a lot of potential issues and build a good resource. This could be done by JSON

[AnthonyMuscio]

I am currently hacking both the plugin and Import mechanisium exploring some ideas.

I would like to be able to export a plugin but do so with the edited/changes tiddlers being used not only the original packaged tiddlers.

Somehow I need to inspect all the JSON tiddlers defined in the plugin tiddler + those edited and generate a new plugin (or JSON file) with the edits applied.

Why? - Import a plugin with designer tools. Edit and enhance them in the current wiki and export the new version of the plugin with edits included. This provides a continuous improvement development mechanism.

[AnthonyMuscio]

Without putting the full argument, I would like to see the introduction of additional plugin types rather than pollute the existing plugin-type of plugin which has come to have a specific meaning. I would also see additional plugin tabs appear within the control panel plugins listing these plugins if they are present.

• Plugin type "data" (For shadow tiddlers containing data tiddlers) sharing test data or shared data sets. Allow extract or hide
• Plugin type "json" (for collections of tiddlers intended for distribution as json) my use data but can be dragged and dropped to another wiki with a import or save option.
• Plugin type "macros" (Non JS plugins that provide macros to extend tiddlywiki functionality) Enables disable a set of macros with a click
• Name your own plugin type.

[Jermolene]

Hi @AnthonyMuscio the word "plugin" has a very specific meaning within TW5: it is a JSON tiddler with a "plugin-type" field set to "plugin", "language", "theme" or a custom type. I think you're using the word "plugin" to mean "something downloaded from the official plugin library".

The plugin library mechanism could indeed be used to distribute arbitrary tiddlers, they don't have to be plugins.

[AnthonyMuscio]

@Jermolene I am NOT using the word "plugin" to mean "something downloaded from the official plugin library".

We could say abstract definition of a plugin is a tiddler containing other tiddlers in JSON format that presents to tiddlywiki shadow tiddlers, can be enabled and disabled, and its tiddlers overwritten.

Traditionally these have being used to distribute features in what we call plugins, themes and languages. As you say we can use "custom types" of plugins. I am simply suggesting provide a set of additional "plugin types" and provide the handling (listing) for sensible use rather than leaving this undefined. In many ways these alternate uses are still plugins because they are plugged into a wiki.

If we don't do this with the plugin enhancement there will be more and more "plugins" with various uses appearing in the the current, single, plugin list, and no selection or sort mechanism.

A key example is macros. People are sharing macros all over the place, typically there is no rules beyond adding the macro tag. There is no formal distribution method, and now you are providing a mechanism for plugins that do not need reloading, I think more macros will be moved into "plugins" and I think this is a good opportunity to set some standards (plugin types) that reflect these other uses rather that having them complicate the list of plugins.

We could arguably provide libraries of macro (plugins}, libraries of test data plugins, For example Bimlas's Kin operator could come with a dependence on a test data tiddler, one can then delete or disable the test data. Where do you find such data plugins? under the plugin type/tab "Data plugin".

Regards
Tony