Rack: [Feature Request] internet access for modules

[gbrandt1]

what about modules accessing online resources, like radio streams or weather data?
no, or is there an API or we can do whatever we want?

[danballance]

My gut reaction to this is security. Imagine a botnet of rogue VCV modules launching DoS attacks. I'm only half joking.

It's an interesting idea though for experimental work.

[sebastien-bouffier]

I made some tests with curl in december 2017 and I stopped for 2 main reasons the first one being the same as danballance, opening http stream can lead to security issues and the second one is that if you want to decode mpeg you quickly face licensing issues. That was my conclusion at that time so I trashed my test module but I am curious to read other opinions, tech information etc...

[gbrandt1]

i think we have that security issue already simply by allowing closed source modules

[danballance]

Perhaps security concerns should be thought through in a separate ticket? This is just one aspect of those concerns. If VCV takes off in a really big way then pushing out a new module that clones a famous Eurorack module, such as Make Noise Maths for example, and obfuscating some code in there to do some nasty stuff is a real risk. Security conscious developers may be willing and able to audit some code, but for your average punter they are just going to install and run the next new module they just must have.

I'm sure I saw someone in another ticket raise the issue of sand boxing - but I just did a quick search on security here and didn't turn up much that seemed relevant. It could be too much to re-architect for this at this stage. I'm sure my VST plugins could be insecure as well, but it would be a really nice feature for the platform to have in an ideal world.

[squinkylabs]

Am I missing something? Can't a module today open a socket connection to wherever it pleases?

[danballance]

Quite possibly @squinkylabs - I only develop in (easy lol) high level languages so not currently involved in any VCV development. I've not looked at how modules are built and what API constraints there may or may not be.

[AndrewBelt]

Would be entirely possible, but you'll need to statically link CURL + OpenSSL into your plugin, which is no easy task. CURL + OpenSSL is already loaded by Rack, but because Windows .dlls cannot access symbols from other .dlls loaded by the same application, your plugin cannot call the CURL library bundled with Rack. Let me know if you have ideas to solve this issue.

[sebastien-bouffier]

It is feasible on windows, Once I made a plugin that was able to read content from url with curl. I was using html code to generate sequences -> char to volts. I was not satisfied with the result so I trashed it :) So I confirm curl allows that. I also tried to read mpeg streams but I didn't find any free open source library for decoding and according to my understanding it is due to format licensing.

[AndrewBelt]

@sebastien-bouffier Not sure what MPEG you're talking about, but there are a few out there, like https://www.mpg123.de/index.shtml. Most patents are expired already, most notibly all for MP3.

What CURL did you use? Did you statically link your own?

[sebastien-bouffier]

thanks for the information. I am not sure but I do not remember having used anything else than libraries provided with rack. Again not sure but I only remember having done a simple include in my test. If some users are interested in I can retry to go in that direction. The reason why I tried curl at that time was to read webradio's streams ... so

[bontric]

@danballance

Perhaps security concerns should be thought through in a separate ticket? This is just one aspect of those concerns. If VCV takes off in a really big way then pushing out a new module that clones a famous Eurorack module, such as Make Noise Maths for example, and obfuscating some code in there to do some nasty stuff is a real risk. Security conscious developers may be willing and able to audit some code, but for your average punter they are just going to install and run the next new module they just must have.
I'm sure I saw someone in another ticket raise the issue of sand boxing - but I just did a quick search on security here and didn't turn up much that seemed relevant. It could be too much to re-architect for this at this stage. I'm sure my VST plugins could be insecure as well, but it would be a really nice feature for the platform to have in an ideal world.

This is certainly an important point which should be seriously considered (pinging @AndrewBelt). While sandboxing the whole application (#638) is a valid option, plugins can still affect the application itself.

A more sophisticated approach to archive this would be to run plugins in separate processes which don't have any privileges (privileges can be dropped when forking a process). Those plugins would communicate with Rack's core process via shared memory or maybe sockets (This might be problematic due to overhead). Limiting network access might be another issue but should certainly be possible.
This will require a revamp of the plugin architecture to some degree and a fair amount of work and most importantly a lot of testing and planning. I would strongly advice to approach these issues sooner than later since Rack is already a quickly growing project which can, as of now, easily be abused by mean people.

This would have other advantages as well. Multiprocessing is a planned feature anyways (#195 ) and separating each module would prevent single, faulty, plugins from crashing the whole application. Performance might be an issue but this would need further evaluation.

In general it's important to note that installing any software on your system creates a risk since you always need to trust the developers if you don't audit the code yourself. As of now, I would advice anyone to be careful when installing 3-rd party plugins especially if it's closed source.

[WIZARDISHUNGRY]

Little back of the envelope calculation of the overhead of out-of-process modules: Some people suggest scheduler latency is on the order of 30M instructions (src). Assuming a 41k sample clock and 10 modules on a 2.4ghz machine:

30 million * 10 * 41000 / 2.4 billion = 5125

we're performing 5000x too many operations per second. So even if we buffered 4k audio samples (unacceptable!), we'd be over our compute limit just from task switching.

This is without considering the overhead of shared memory segments, other processes, and module computation. If we performed this inside a dedicated module process using pthreads, presumably this would be much faster, but I still believe you'd need to buffer from the figure of 100,000 pthread context switches given here. Please correct me if I messed up the math or assumptions.

It is currently possible to spawn a worker pthread inside Rack and consume it in step() if you don't synchronize on data structures every clock.

[AndrewBelt]

As for security, only one thing needs to be considered: Don't allow arbitrary code execution and you'll be fine. Make all the networking plugins that you want. Just do the networking code in another std::thread of course.

@bontric Overhead is too high for IPC between modules. Multiprocessing is not planned. The issue you linked is about multithreading.

[bontric]

@AndrewBelt

Don't allow arbitrary code execution and you'll be fine.

Arbitrary code execution is exactly what plugins do. Executing 3-rd party plugins within racks main process will give them the same privileges the executing user has which is basically an open door for a malicious plugin. Running plugins (or Rack) in a non-privileged process is a way to restrict malicious plugins.

Overhead is too high for IPC between modules.

Using shared memory it might be feasible. Looking at other architectures which use a plugin based approach with security in mind might be a good idea.

Multiprocessing is not planned. The issue you linked is about multithreading.

You are right, what I meant is that the approach could be used to tackle two problems at once since multiple processes would also utilize multiple cores.

[AndrewBelt]

Arbitrary code execution is exactly what plugins do

We're talking about data a non-malicious plugin downloads from the internet, not malicious plugins (you're welcome to open a discussion).

[WIZARDISHUNGRY]

@bontric #666 (mostly) works for isolating Macs from malicious plugins. I'm still looking for Windows and Linux solutions.

[sebastien-bouffier]

Hello.
I just made a module (antN) that uses mpg123 to decode mp3 streams with Curl.
It works pretty well on Windows (still needs some improvements) but I have an issue on Linux (and maybe on OS X too). The compiler fails linking with Curl under Linux ... W10 is OK.
Here is my makefile https://github.com/sebastien-bouffier/Bidoo/blob/master/Makefile
Any help is welcome.
Best.

/usr/bin/ld: cannot find -lcurl
collect2: error: ld returned 1 exit status
../../compile.mk:43: recipe for target 'plugin.so' failed
make: * [plugin.so] Error 1

[AndrewBelt]

You might need to add a linker path to $(RACK_DIR)

[sebastien-bouffier]

Thanks a lot I'll try.

[AndrewBelt]

libcurl is now statically compiled into Rack (v0.6 branch as of a few weeks ago), so plugins can directly call the libcurl API without supplying their own libcurl binary on Windows. This was not an issue on Linux and MacOS.

[sebastien-bouffier]

Received, I will test. Thanks