Rack: Module whitelist

[AndrewBelt]

This proposal has two purposes:

1. Users would like to add individual modules rather than entire plugins into their Module Browser.
   Many users have requested the "Favorites" feature, but I believe that what most users actually want is to remove/hide modules they don't use.

1. Commercial plugin developers (including VCV) would like to sell modules individually in order to increase customers' value / price ratio for their purchases. In some cases this increases profit, and in all cases it is more consumer friendly to have this option available.

Proposed implementation

This proposal is to be added to Rack v2.

Settings and Module Browser

A .moduleWhitelist setting should be added, e.g.

    "moduleWhitelist": {
        "Grayscale": [
            "Supercell"
        ]
    }

If a plugin slug is present in this JSON object, only the listed modules should be visible in the Module Browser. If not, all modules in the plugin should be visible.

The whitelist only affects the Module Browser.
If a patch is loaded that contains a module that is not available in the whitelist, it will be used anyway.
This allows users to try others' patches without requiring the lengthy task of adding all relevant modules to their VCV account.

Syncing with VCV API

When plugin updates are synced (plugin::syncUpdate()), the whitelist is overwritten.
The .moduleWhitelist object is not intended to be manually edited.

VCV Library website

On https://library.vcvrack.com/, a button labeled "Add Geodesics Entropia" for example should be added to each Module page and each module entry in the index page.
On the plugin page, a button labeled "Add all Geodesics modules" should be added to the top. Its behavior is complicated, but the following seems to be the only good way of doing this.

• If no modules are added, it should say "Add all Geodesics modules".
• If all modules are added, it should say "Remove all Geodesics modules". The system should consider the user to be "subscribed" to the plugin, so they should obtain new modules as the plugin developer adds them.
• If some but not all modules are added, it should say "Add all Geodesics modules". The user is not subscribed to the plugin. In this case only, Rack will sync a .moduleWhitelist entry.

Commercial plugins

Regardless of the whitelist in settings.json, commercial plugins that allow individual module sales should do their own validation that the user owns the license for that particular module upon use.
rackdrm.hpp will serve this purpose, which will be available before Rack v2 to interested commercial plugin developers.

[MarcBoule]

I agree with the proposal.
Just for terminology though, in the three-point list near the bottom, should this

• "If no modules are added, it should say "Add all Geodesics plugins".

instead say this:

• "If no modules are added, it should say "Add all Geodesics modules".

Same in the other two items I think.

[FunkybotsEvilTwin]

As an end-user, I would be more inclined to download everything from a collection, then hide the items I don't want/need after having had some time to check them out. Example: "I already have a clock divider I like better than this one, this LFO doesn't add anything new, I like this sequencer though, this quantizer is neat, etc."

The most intuitive method for handling that on the user-side would be to allow users to hide items in the browser itself. Right click module in Browser -> "hide from browser." Then maybe a filter/checkbox to "show hidden modules" within the browser itself.

[AndrewBelt]

@MarcBoule Thanks, fixed typo.

@FunkybotsEvilTwin Bidirectional control between Rack and the VCV Library would be messy. Also, if users can hide modules from the Module Browser, they'll want to show modules from the same interface too, which is also messy. Using the VCV Library to hide modules is more straightforward, because the page doesn't disappear when you click "Remove Geodesics Entropia".

[MarcBoule]

@AndrewBelt the three point itemized list has the same typo in each of the points, I believe. Sorry to be pesky :-)

[FunkybotsEvilTwin]

Andrew, some questions that spring to mind: what would I do if I bought a third-party set of modules like those from KlirrFactory and wanted to hide some of those? What if I want to quickly unhide a module (e.g. "I don't use ATOMS much and hid it, but maybe I should give it a quick try today")? Why make users leave the VCV Rack application or plugin to hide a module? Does this really address the favorites issue?

You don't want to develop a solution that ultimately doesn't meet user expectations and end up in a situation where you've got rebuild an alternate solution. My suggestions:

Proposed implementation

1. Users add plugin collections to their libraries as per current process
2. Authors could still sell individual modules
3. Within the VCV Rack plugin browser, users can still search for modules based on text, filter by tag, and manufacturer
4. New options are added to the VCV Rack Browser for A) star rating (or a simple favorite yes/no), B) sort by rating/favorite status or name, and C) the additional ability to hide/unhide modules
5. To hide a module, user right clicks module in browser and selects "Hide" option - perhaps consider option to allow users to select and hide multiple items in the browser at once
6. To show hidden modules, user clicks a checkbox in the browser itself called "show hidden modules" - this allows users to quickly toggle between hidden and unhidden modules
7. Users can sort the browser results based on favorite status or star rating putting top modules on top - consider the ability to allow users to add a favorite status to modules all at once
8. "Hidden" and "Favorite" settings are written to a locally stored file - allows users to backup and restore
9. No need to sync favorites, hidden/unhidden status etc. to VCV Rack account
10. Because this is happening locally, modules like KlirrFactory, Floats, etc. can all still function within this framework
11. Users don't have to exit application or plugin to make these types of changes - only to add plugin collections
12. Bonus points if module thumbnails could be stored/cached in a local database for faster browsing

Just an alternative way of trying to accomplish a very similar result but I think the above type of approach would improve the overall Customer Experience and workflow, while being more intuitive. I'd always suggest on taking a customer-centric view of design, and I just don't think managing which modules appear in the browser outside of VCV Rack or the plugin accomplishes that as well as just doing it within the browser itself.

[AndrewBelt]

What you're suggesting is at least 10x more complicated than my proposal. It also doesn't address the use case:

• User sees Geodesics Entropia on the VCV Library and just wants to add that to their account.

I believe this use case alone is more important than everything else combined. This is how buying a Eurorack module works. You see a module you like on your favorite store, you purchase it, and you have the ability to add it to your rack at home. It can't be simpler than that. Nobody I know organizes modules in a drawer by a "rating" or "favorite status" that they sit down to maintain. They just grab the module that fits their needs when they need it.

You've asked a valid question though.

what would I do if I bought a third-party set of modules like those from KlirrFactory and wanted to hide some of those?

This proposal wouldn't help you here. You'll just have to deal with using all modules (install plugin) or no modules (uninstall plugin), but I don't think this is a blocker for the proposal.

[FunkybotsEvilTwin]

I don't mean to trivialize the amount of work that would go into what I'm proposing. I'm sure it's a TON of work (I remember the U-he folks taking a long, long time with their browser). I'm just trying to look ahead to when the user base blows up after a VST version of Rack is available. A large number of users will be looking for a more traditional VST-type experience (Browser with favorites/ratings, module management - see U-he, Arturia, Reaktor, etc.) as opposed to a hardware type experience in software form. I'd wager there will eventually be a large, very vocal percentage of users pushing hard for that type of browser experience within Rack.

What's the data tell you? You probably have data on it already, but how many users have >1,000 modules installed? How many are very selective in what modules they download? I suspect a lot of people will download just about every module they can get their hands on, and won't care much about the ability to add modules one at a time (hell, an "Automatically Add Everything" button would be wildly successfully). I know I want to check out everything in a plugin even if only one or two modules initially appeal to me. If that sounds about right, that data point would be indicative of users preferring to check things out after installing them and instead potentially wanting to hide after the fact. If this sounds like a large percentage of the user base already, then I imagine it will only get worse as adoption increases. And this would indicate that users don't want to add things one at a time, so much as they want to hide them after the fact.

Anyway, I'm not trying to be argumentative for the sake of it or anything, just trying to offer a different perspective/food for thought. It may not make sense to get ahead of it right now with everything else on your plate, and maybe it's not a Rack 2.0 but maybe a Rack 3.0 (or later) feature, but my experience tells me "ye average plugin user" isn't going to love having to manage modules elsewhere and will want favorites and the ability to hide modules anyway. And if you accept this basic premise as maybe being true, maybe there's a way you can build an incremental approach that eventually gets you there without having to go in years later and having to rebuild the process.

[AndrewBelt]

I could discuss the details all day, but the reason I don't like most of the features in your proposal comes down to the following choice.

We can have a functional website at https://library.vcvrack.com/ where users can "shop" for modules they want to learn/use, both free and commercial, and immediately sync them to all their computers.

Or we can force users to download tons of platform-specific ZIP files, overwhelm them with 1600 modules in their Module Browser they've never seen, spend 30 minutes hiding modules they don't think they'll need, and then spend several minutes per day managing the time-dependent variable of "how much they like" each module. If this is how users will install modules, what's the point of https://library.vcvrack.com/?

It's clear to me that the first is the obvious choice. A common complaint about VCV Rack is that the number of modules is extremely overwhelming. It's difficult to learn 1 module until you're comfortable with it if you know there are 1,600 others installed waiting for you. If a user can go to a web page to shop for modules, they'll give more attention to the modules they install.

So I'm not really interested in "from-scratch or overhaul proposals". I'm interested in feedback to make positive changes in my original proposal.

[gabrielreygoodlatte]

I like the proposal. To me, it feels kind of separate from the favorites/tagging issue. I think ideally both would exist. I don't know how they should be prioritized versus each other, and other stuff, but I don't get why it would be a choice between the two. A big collection of virtual instruments and sample content feels like a relevant analogy. You want control over what to install and what not to install. When dealing with that much content, I also want some ability to categorize and sort it on my own terms. I have every free module (and a few I've paid for) installed, and I like having them on hand when patching. On the one hand, I want to be able to quickly find the stuff I use all the time, and also to curate stuff I want to try out soon, so that it's easily visible. But I also don't want to lose the ability to try some weird module I've never tried before, on a whim, in the flow of patching.

A different approach would be to have something like Ableton's "user library". It could be another section of the browser that is based on files and directories, for users to organize their presets (and sub-patches, if that eventually becomes a feature). That would sort of solve the favorite issue -- we could create folders with presets for whatever modules we want, which effectively creates a custom list of modules. Would also be cool to be able to directly load a module with a particular preset loaded (and/or to load strips of modules directly).

I don't know much about the technical challenges involved, so this is just my 2 cents, with a lot of love and appreciation for this free software and all the incredible work that goes into it.

[AndrewBelt]

it feels kind of separate from the favorites/tagging issue

I agree. It's not orthogonal, although there are many intersecting features. But @FunkybotsEvilTwin's proposal brings in a ton of non-related featured into this proposal, which is why it was rejected. This proposal is about adding an "Add/buy module" button to the module page and its potential implementation.

[jonheal]

I'm for whatever works best for you, Andrew, but I would definitely like to be able to pick and choose between modules in some fashion rather than having to grab/browse an entire collection.

It's actually paralyzing having too many choices.

[stoermelder]

I think it is the right direction to switch to a module-centric view instead of the current plugin-centric view. I also want to add two thoughts to consider in this proposal:

1. I propose supporting multiple whitelists. In this way I could organize my modules as "My Favourites" but can also create a list "Sounddesign" or "Synced LFOs" if I want. Every user has a default whitelist and more lists can be added if someone likes to. This feature would complicate the website a bit but users wont't have to care about it at first. This feature has not to be added to the initial release of Rack v2 but should be considered in the design stage to be implemented in a future update.
2. It is a kind of special case but I like to mention it: While developing new modules I promote nightly builds or send them beta testers from time to time. Using this proposal it sounds impossible to me for users to access modules which is not available in the library and therefore can not be whitelisted. Maybe an attribute "ignoreWhitelist" or something shoukd be added to the plugin manifest which can be used if a module is distributed outside the library. Obviously this attribute can not be allowed for plugins in the library.

[AndrewBelt]

multiple whitelists

Too esoteric.

(2) brings up a more general problem with my proposal. If you add a module to your VCV account, download the plugin, and remove the module from your VCV account, Rack will not have an entry for the plugin in the whitelist, so all modules in the plugin will appear at next launch, including ones you didn't add at all. How do we solve this?

[MarcBoule]

Currently, it is understood that when we de-select a plugin in the plugin library, this does not delete the plugin in our plugins directory and we must delete it ourselves to stop it from showing up in the module browser. Could it be the same in your scenario above and nothing actually needs to be solved? i.e. a user has one module added from a plugin in the library, later they remove that module in the library, then they have to go delete the plugin themselves? Not sure this is user friendly nor what you are after, just bouncing an idea around.

[FunkybotsEvilTwin]

How about a blacklist instead? On the back-end, it can work similarly to what you're proposing, but it would work to exclude modules from the browser rather than include them. So if a user goes and removes a Plugin from their VCV Rack Account, the entire Plugin gets added to the blacklist on next Rack launch, and all modules remain hidden in the browser. Now, the flip side of that means, to add a single module to the browser, you'll need to keep track of all modules in the collection and blacklist the others.

So if there's a plugin collection called "Variables," with 4 modules named A, B, C, D and I select to only add D my VCV Rack account, you blacklist A, B, and C. Then if I remove D from my Account, D also gets added to the blacklist. Now with NYSTHI that could mean writing hundreds of modules to the blacklist, if a user only wants a few, so maybe not ideal.

[stoermelder]

multiple whitelists

Too esoteric.

(2) brings up a more general problem with my proposal. If you add a module to your VCV account, download the plugin, and remove the module from your VCV account, Rack will not have an entry for the plugin in the whitelist, so all modules in the plugin will appear at next launch, including ones you didn't add at all. How do we solve this?

Library could keep records for every plugin that had been added to the user’s account anytime in the past. Deselecting a plugin would not remove the plugin completely from an account, instead all modules are removed from the whitelist and an empty whitelist for the deselected plugin is sent to Rack.

[AndrewBelt]

@MarcBoule Unfortunately I don't think this solves the problem in https://github.com/VCVRack/Rack/issues/1619#issuecomment-554795971

@FunkybotsEvilTwin I don't think blacklists offer any advantages or disadvantages over whitelists. The server would just need to do more work in computing all modules in plugin - modules added to VCV account. The problem I mentioned above still remains. If you add a module in a plugin to your VCV account and then remove it from your VCV account, the API would delete the module blacklist entry for that plugin, so Rack would display all plugins, giving the same problem as the above.

@stoermelder The problem remains that if you add Geodesics Entropia from the VCV Library and then want to try a non-VCV Library build (e.g. from source), you'll only see Entropia in the Module Browser. With your suggestion, if you remove Entropia from the VCV Library, you won't see any module in Geodesics because the whitelist will look like

    "Geodesics": [],

[FunkybotsEvilTwin]

@FunkybotsEvilTwin I don't think blacklists offer any advantages or disadvantages over whitelists. The server would just need to do more work in computing all modules in plugin - modules added to VCV account. The problem I mentioned above still remains. If you add a module in a plugin to your VCV account and then remove it from your VCV account, the API would delete the module blacklist entry for that plugin, so Rack would display all plugins, giving the same problem as the above.

I covered that...

So if there's a plugin collection called "Variables," with 4 modules named A, B, C, D and I select to only add D my VCV Rack account, **you blacklist A, B, and C. Then if I remove D from my Account, D also gets added to the blacklist.**

So in that case, all 4 modules would now appear on the blacklist meaning nothing appears after module D was removed.

Note: I don't think this is the most elegant solution, but was just brainstorming. My other thought was to create a "plugin collection" class of white list entry, and an "individual module" class, but that also felt cumbersome and difficult to maintain.

[AndrewBelt]

@FunkybotsEvilTwin Doesn't solve the problem. See my response to @stoermelder. Your blacklist suggestion is just a dual of the whitelist suggestion. It's a different but equivalent solution but still has the same problems as the whitelist.

[stoermelder]

How about my suggestion from above for distributing packages outside the library?

Maybe an attribute "ignoreWhitelist" or something shoukd be added to the plugin manifest which can be used if a module is distributed outside the library. Obviously this attribute can not be allowed for plugins in the library.

[jonheal]

I suppose it's too much to just dispense with module collections altogether, and make grabbing modules from VCVRack.com a get-them-one-at-a-time proposition?

Granted this would make for an incredibly long list on the site, given some prolific authors like NYSTHI, but I personally would not mind scrolling through a long list if that meant I could get just the ones I wanted.

[MarcBoule]

For plugins distributed outside the library, Ben's idea would be one way to handle it, but perhaps there is a different way also. Just thinking out loud, as each method would have their advantages. Here are my summary+thoughts on this:

A) if the plugin does not already exist in the library, it will show up in the module browser since there is no whitelist for it, so this case is not problematic.

B) if the plugin does exist in the library, perhaps we can have it such that when the entire plugin is added in the library, no whitelist entry is created. Testers would be informed that when testing and doing manual installs of plugins that we send them, they should always have the entire plugin added in the library, in order for it to show up in their module browser.

I guess we are discussing two issues: the problem AB has mentioned that is still not solved, and the request that Ben and I (and others for sure) have that we be able to send plugins to testers. Currently it works quite well; when we make dev builds, we keep the same version as in the plugin library so no updates are triggered when they install the dev version of the plugin.

[AndrewBelt]

@MarcBoule

if the plugin does exist in the library, perhaps we can have it such that when the entire plugin is added in the library, no whitelist entry is created

This is the proposed behavior.

Testers would be informed that when testing and doing manual installs of plugins that we send them, they should always have the entire plugin added in the library, in order for it to show up in their module browser.

I believe this should be fine, so I'm no longer concerned about this issue. I'll inform plugin developers to tell testers that users either need to click "Add all modules in plugin" or "Remove all modules from plugin".

In order for the last point to work, we need to add the following behavior to the proposal.

• If your VCV account has no modules in a plugin, the API will not send an empty whitelist (e.g. "Geodesics": []) but will instead not include the plugin slug in the whitelist at all.
• On the other hand, Rack's settings will include an empty whitelist for the plugin by "merging" the JSON with the logic if settingsWhitelist[pluginSlug] exists but apiWhitelist[pluginSlug] does not exist, set settingsWhitelist[pluginSlug] = []. This means that if you previously had a module but removed it, the client (Rack) holds this state, but the server (VCV Library) doesn't need to keep track of it.

[stoermelder]

I’m not sure but doesn’t this behavior fail on the following sequence?

Add Entropia from Geodesics to account / API sends whitelist, Rack creates setting.
Remove Entropia from account / API doesn’t send whitelist, Rack holds last state and keeps empty whitelist.
Add all modules in Geodesics to account / API doesn’t send whitelist, Rack holds last state and keeps empty whitelist for Geodesics?

[AndrewBelt]

Oops, meant to say users either need to click "Add all modules in plugin", or "Remove all modules from plugin" and remove the "Geodesics" object from their whitelist.

Without the JSON merging behavior, the latter wouldn't work because the empty Geodesics whitelist would be overwritten on launch.

I like having this alternative option (removing rather than re-adding the plugin), but the caveat is that users must be informed to remove the whitelist manually with a text editor.

[AndrewBelt]

I am accepting the following proposal based on the discussion in this issue thread and internal discussion with @milholen.

Terminology

This will be used at https://library.vcvrack.com/.

• Subscribe plugin: Tells Rack to download the plugin and not use a whitelist for that plugin. This effectively enables all modules in the plugin in the present and future, in case the plugin developer adds new modules.
  
  
  
  • Note that Rack sync/updates plugins regardless of subscribed state.
  
  
• Unsubscribe plugin: Keep all current modules enabled in the plugin but don't allow new modules to be enabled in the future. This is implemented by creating a whitelist containing all current modules in the plugin.
• Enable module: Adds the module to the whitelist.
• Disable module: Removes the module from the whitelist. If subscribed, unsubscribes before removing.

VCV API

When Rack requests updates, the VCV server responds with two data structures.

• plugins = [...]: List of plugins for Rack to sync. This includes both subscribed plugins and plugins with at least one module enabled.
• moduleWhitelist = {pluginSlug: [...], ...}: List of modules to display in the Module Browser. If a plugin's whitelist does not exist, all modules in the plugin are displayed.
  
  
  
  • The VCV API will never send an empty [] whitelist for a plugin. The whitelist will simply not exist.
  
  
  • Note that patch loading does not use this whitelist. If a patch requests a module in a loaded plugin, it is added to the rack regardless, in order to avoid unnecessary "This module is not enabled in your account" messages.
  
  

Merging

When Rack requests updates, it will merge the remote whitelist with the local whitelist (in settings.json) in the following way per plugin slug.

• If remote exists, local is set to remote.
• If local exists and remote does not exist, local is set to [], not removed.

If manually installing a plugin that you have previously enabled and then disabled from the VCV Library, you will have to manually edit settings.json to remove empty whitelist entry.
This is a "nasty" corner of the proposal, but I believe the above criteria will be rare.