Electron: Idea of runtime mode

[zcbenz]

Currently developers using atom-shell have to ship the whole atom-shell binaries when distributing their apps, but since now we have asar as atom-shell's app format, we may add runtime mode to atom-shell like Adobe Air or Chrome Hosted Apps that developers only need to distribute packaged app in asar format as long as end users have the runtime mode of atom-shell installed.

The runtime mode can just be another app based on atom-shell (let's call it atom-runtime for now), and has deep platform integrations:

• On Windows, we can provide installer and auto updater for atom-runtime, and register the .asar file type to be opened with it.
• On Linux, we provide official software sources of atom-runtime for some popular distributions, and also register atom-runtime as handler of .asar files.
• On Mac, we need to provide a tool to help users package their apps into Mac bundles. We can reference how Steam created bundles for downloaded games, or how Parallel created bundles for applications in hosted OS.

We can even provide a tool or service to help generating installers for developers' apps which can automatically download atom-runtime if it is not installed on user's machine, just like many .NET applications.

[Subash]

+100, That would be very very helpful to reduce the size of app.

[ghost]

Can the .asar extension be changed? It sounds really weird in hungarian. Basically it would mean the shit in a bad way (link).

[YurySolovyov]

.apak? with all seriousness, there are tons of languages, so almost any extension can sound weird in some of them.

IMO runtime installation and updating should be as easy as possible. Many runtimes have very annoying notifications. Also, if you are going to do this, some additional manifest metadata should be provided, because newer app versions may not work on old runtime, so you need to compare current runtime version with version in app manifest( possibly even versions range as in node packages ), and then if runtime needs to be updated, make this process as easy as possible. I don't know what easy is for most people, but for me it is just window with progress bar or status messages ("Downloading... 57%", "Updating...")
I'm not sure but background updates may also be considered. It may be some update service or just runtime download while app is running and update when app closes.

[malcolmalex]

+1 for this. I'm working on an application that I have to deliver for mac and windows, and this would simplify the delivery considerably.

[joshuawarner32]

This would be awesome!

A few thoughts:

• As @YuriSolovyov said, we need a way to declare what versions of the atom-runtime a particular asar is compatible with. Preferably, this should also allow specifying an exact version, since this lessens the burden on QA when releasing an asar-based app.
  
  
  
  • To support that, we'd need to allow having multiple versions of the atom-runtime installed side-by-side, potentially invisibly to the user.
  
  
  • Note that this is somewhat at odds with the ideals of limiting download sizes. However, a company producing multiple asar-based products could standardize internally so all their apps still use the same version.
  
  
• You didn't mention support for auto-updating atom-runtime on Mac, but that would be _very_ useful.
• Rather than having source-only installations on linux, it would be good to support installing / updating binaries, both through .deb/.rpm packages as well as some method that doesn't require sudo rights to install (i.e. per-user). For runtimes installed per-user, auto-update would also be very useful.
• Would there be any support for keeping installed asar files up-to-date - or is that something that'd have to be implemented independently in each app?

Lastly, what's the timeline on this effort? What would be the best way to get started, if I wanted to begin implementing parts of this? I'm open to contributing.

[zcbenz]

@joshuawarner32 Currently this is just a rough idea, I plan to work on it after solving some major issues of atom-shell, but it could be months later.

[YurySolovyov]

I think it is also worth looking at existing runtimes and how they are working/updated, how disturbing they are and what other pros and cons.

[steelbrain]

Currently this is just a rough idea, I plan to work on it after solving some major issues of atom-shell, but it could be months later.

It's been about 5-6 months. It'd be great to have this implemented.

[joshuawarner32]

We've built up an internal implementation of this that fits our particular use-case very well, and we might be able to upstream it. The danger would be that it's actually too specific to our use-case, and nobody else would be interested in the extra complexity.

Here are the highlights of our design:

• Deploy .asar.gz files for several different apps on a server
• Distribute a single build of atom-shell to users, which doesn't bundle any code for specific apps
• On app startup, download the appropriate .asar.gz file (if it's not there, or there's a more recent one on the server). Extract it and run the contained app.
• Our atom-shell build accepts a --app argument to specify which app to download/run

Is that sort-of what atom-shell folk had in mind? Would something like that be useful to other people?

[tonypee]

Like the JRE it could be the ERE (electron runtime environment), with .eapp files (electron app)

[steelbrain]

-1 for .eapp

[sergeysova]

+1 I need it!

[anaisbetts]

On Windows, we can provide installer and auto updater for atom-runtime, and register the .asar file type to be opened with it.

Install Steam on Windows and check out how they create game shortcuts, they actually use a protocol handler, it's relatively clever

[zcbenz]

:+1: on the Steam model.

There is no technical obstacle on implementing this, the whole thing can just be an Electron app itself. But the problem is it is not easy to implement neither, taking all the details into account, I think it requires a small team working full time on it, I'm not sure whether it would ever happen.

[anaisbetts]

But the problem is it is not easy to implement neither, taking all the details into account, I think it requires a small team working full time on it, I'm not sure whether it would ever happen.

There are a lot of questions in my mind that, while certainly aren't insurmountable, make this plan Difficult. For example, say "App 1" requires 0.30.x and cannot run on 0.31.x, and "App 2" requires the opposite. Who wins in this scenario?

It seems like a lot of effort and infrastructure / trickiness to optimize for something that isn't our biggest problem - disk space and bandwidth, at the expense of developer complexity (i.e. now developers have to create this crazy chain-installer to first install/verify Electron, then install their app)

I don't mean to be hatin' on an idea! I just think that with the time we'd take to do this, we could instead spend it on our big problem - making it super dead easy for new devs to create and publish Electron apps, from their first line of code all the way to shipping it to users

[steelbrain]

@paulcbetts But hey, it's benefits are worth it, aren't they? When updating an app, users will have to download less than an mb instead of ~70mb. Which means we won't even need auto-updating frameworks and all, we will just have to download and extract a micro-zip archive with a couple of files.

I've spent the last three months in a remote area where the speed was less than 128kbps and trust me, the upgrade size is a huge pain in the arse, the worst part is that there's still tons of people still in similar situations

[anaisbetts]

Which means we won't even need auto-updating frameworks and all, we will just have to download and extract a micro-zip archieve with a couple of files.

I don't think that's true though, because almost certainly at some point you'll want to move in an update to a newer version of Electron and then kerplowie, you're in a similar sitch (or at least one that is similarly ugly)

[RIAEvangelist]

Basic apps now are between 30 and 50 Megs unless you have a crap ton of videos inside your package. I don't think that's a stretch for most people to download. I believe the average youtube video is between 30 and 40 mb...

[JoshTheDerf]

I'd just like to make a note here, in China, the connection speed averages about 20kB/s (Granted I've got a connection slower than ~92% of my city.) Updating Atom/electron usually requires a full day of wget -c'ing. unless the taobao mirrors are used.

For version compatibility, electron development moves too fast to have a catch-all version either, perhaps some sort of wrapper/helper could be used to use/download the requested major version when the app starts?

[RIAEvangelist]

20 kb/s? how can you survive? I can't imagine how difficult that must make things!

[RIAEvangelist]

@steelbrain not my intention

[JoshTheDerf]

@RIAEvangelist Getting on at the fast times and lots of patience. The speed gets really fast late at night, so just leave downloads running or scheduled while you're sleeping. :3

Anyways, I think it wouldn't be too much work to write a (basic) third-party wrapper, anyone want to try?

.asar files, however, can't easily declare their electron version. Perhaps another program descriptor system would be needed?

[YurySolovyov]

How about some kind of runtime version dispatcher that (downloads if needed and) launchers required version of runtime for your app? We can add an entry to package.json for that.

[JoshTheDerf]

@YuriSolovyov Except that we can't have said runtime handle .json files by default. I suppose it could read the asar file first and view package.json, but that would slow it down a bit.

[YurySolovyov]

@Tribex yes, logic flow is like:

-> associate .asar with runtime
-> user launches .asar app
-> dispatcher reads package.json in .asar
-> (optional) downloads required runtime version if needed
-> launches app

[hokein]

Ideally, the run-time mode should work like the way of Steam or Chrome App Launcher. The main issue is how we handle apps when the run-time upgrades to new version.

Electron apps suffer from the upgrade of Electron, especially node native modules. The native modules are required to rebuild for each upgrade of Electron due to the C++ ABI issue(There are several forgot-rebuilt crash issues reported by users).

Steam doesn't suffer from the pain since every game on Steam is a stand-alone program, actually you can launch the game without Steam Client. Neither does Chrome App Launch, because Chrome apps are written in HTML/JS/CSS which are kinds of interpreted languages, there are no gap running in different versions of Chrome.

[steelbrain]

@hokein how about we have a lightweight "client" in a native language, that downloads multiple languages, whenever it encounters an app that depends on an uninstalled version of electron it downloads a copy of it, then runs each app with it's specific electron.

[etiktin]

There's a new Electron release about once a week, so the chance that 2 apps have exactly the same version of Electron are slim.

To support a run time approach, we will probably need to start separating releases to LTS (long term support) versions and bleeding edge versions. The LTS will be for production and the bleeding edge will be for developing and testing. LTS will only get bug fixes and maybe some changes that don't replace the major io.js and content shell versions, so native modules should still work.

[RIAEvangelist]

given what @etiktin has said, unless a user has many electron apps on their system, there would likely be no benefit in download time since they would still have to download the run time most every time.

In the two weeks I have been looking at electron now I think I have seen 4 releases.

[JoshTheDerf]

@etiktin Ideally, if electron follows semver, then an app could declare version ^0.31, and it would use the latest 0.31.x release. That might help things a bit.

[jorangreef]

We have an app in production with a semver auto-update mechanism that does the following:

1. GET a simple HTTP end-point on the server giving the current version (0.3.0, 0.3.1 etc.) and platform (darwin-x64, win32-ia32, win32-x64 etc.) of the app.
2. The server returns "304 Not Modified" or else a JSON manifest if there is a newer version.
3. The JSON manifest is essentially a nested list of folders, files and symlinks in the newer version.
4. A folder entry in the JSON manifest looks like this: [name, array of child entries].
5. A file entry in the JSON manifest looks like this: [name, hash of content, array of hashes of content split into 64KB chunks, an integer flag indicating whether the file is executable or not].
6. A symlink entry in the JSON manifest looks like this: [name, destination path].
7. These JSON manifests are created and cached on the server for each version. For each file in the JSON manifest, the server runs a simple deduplication algorithm to split the file into variable length chunks using content-dependent chunking, and then hashes the chunks using SHA256. This ensures that most chunk hashes for different versions of the same file remain the same, even if content is shifted around by a few bytes. The average chunk length is roughly 64KB for optimum deduplication + compression efficiency. If the chunks are larger they will not deduplicate as well, if the chunks are smaller they will not compress as well. The average chunk length will also be increased for very large files to avoid having too many chunk hashes in the metadata.
8. When the app receives a new JSON manifest from the server, it checks available free disk space and creates a new temporary directory with a universally unique ID. Using its old local manifest as a starting point, it then copies chunks from local files or downloads missing chunks from the server to recreate the new manifest. After creating the new version, it then verifies the directory structure and signatures of all file content. If anything was corrupted in the download, it removes the update.
9. If the new update verified successfully against the manifest, it then launches the new version of the app at the new update path.
10. The older version notices that another version is running, and shuts itself down.
11. The new app waits for the older version to shut down, then copies itself to another temporary directory, renames the older version to an archive directory, and renames the temporary directory to the location of the older version (/Applications/Ronomon.app). It then runs itself again but now from the location of the older version (/Applicatons/Ronomon.app) and then cleans up the archive directory.
12. If anything goes wrong during the transition, the older version remains at the canonical path until switched by the rename call.

Overall, it's working very well in production. The deduplication and compression means that only about 33% needs to be downloaded for a typical major version update and minor updates require just 1-2 MB depending on the app. The auto-update handles Electron and app code as one atomic unit, and never violates any code signatures in the process.

[sergeysova]

+1 To idea
+1 To rename asar

[championswimmer]

I came down to make an issue regarding this. Found this already existed, so I'll put my + :100: to this
We totally need a JRE or .NET or Adobe Air like solution, because packing the runtime with each app makes them really bulky.

[PierBover]

Any updates on having a runtime separated from the apps?

Is this closed because a solution has been developed?

[championswimmer]

@PierBover this issue is open.

this - https://github.com/KeitIG/museeks/issues/48 - got closed. in someone else's repository.

[PierBover]

Oh thanks @championswimmer and sorry for the confusion.

So are there any news on the independent runtime? Is this something that is being worked on?

[etiktin]

@PierBover it's probably not going to happen any time soon due to the current version fragmentation (it's hard to find 2 electron apps that are using the same exact version). Maybe after v1 is released.

@jorangreef do you use your differential update mechanism on all platforms? It would be awesome if you could release that code as a standalone module.

[martpie]

@championswimmer Yep, I've managed to create a build script for my app, so I'm ok distributing my app now. But I'm keeping an eye on this because it would be interesting.

[jorangreef]

@etiktin Yes, the differential auto update currently works on all platforms. If it also included a minimal installer which does the download, then the installer could be shared by several apps, and use a cached version of Electron if already available. This would serve to make installs much faster. I would like to open source it if people are interested, and will try to release the code in the next few months.

[louy2]

For example, say "App 1" requires 0.30.x and cannot run on 0.31.x, and "App 2" requires the opposite. Who wins in this scenario?

Then they should install the two versions and there will be two runtimes, just like VS C++ 2008 coexists with 2005. I really like how VS C++ runtimes are installed, the installer would install a required version if there is none, and others can depend on it later, and different runtimes can coexist. Enforcement of version requirements also seems to be more easily done in the installer.

[martpie]

@louy2 There is a difference between the Visual C++ example (3 years between releases), and Electron (3 releases per month at least)

[championswimmer]

I'll quip in with a 'this is a broken system' quote here, if majority of
the apps in the Electron ecosystem are not compatible with +- 0.0.1 or
0.1.0 version differences

On 20 February 2016 at 18:55, Pierre de la Martinière <
[email protected]> wrote:

@louy2 https://github.com/louy2 There is a difference between the
Visual C++ example (3 years between releases), and Electron (3 releases per
month at least)

—
Reply to this email directly or view it on GitHub
https://github.com/atom/electron/issues/673#issuecomment-186595285.

[iamtekeste]

+1

[notpushkin]

I've packaged electron to a .deb package using fpm. This could be a fair starting point on Linux (Debian- or RPM-based, at least). You can take a look here: iamale/electron-deb; also an example app is there. (It will work with asar's as well, just need to add a MIME entry into the runtime package.)

It would be really great if we had something like that on the other platforms as well! I could probably try to package it for Windows, but I don't use it actively anymore, and din't really have had many experience with it.

(Thanks to @louy2 for pointing out this thread to me!)

[notpushkin]

By the way, what's the MIME type for .asar? Would application/x-asar do?

[steelbrain]

@iamale Instead of fpm, have you had a look at AppImageKit. The main benefit that you get is that instead of creating multiple libraries and all, you can target all linux distros with one executable. You also no longer have to ask the user to install any dependencies as they are bundled with the executable. It's a nice project, please take a look.

and about the mime, how about application/x-electron?

[notpushkin]

@steelbrain That's a nice idea. However, this wouldn't solve the original problem: the Electron runtime would have to be packaged with each app (because the images have to be self-contained), which is exactly what this issue is about. Although I think it should be relatively easy to make AppImages out of Electron apps, and I believe this would be great when traditional package management can't be used (rare distros or unpriveleged users). So, thanks for pointing it out!

application/x-electron sounds great by the way, probably will stick with it.

[notpushkin]

Added asar support! Not in the repository yet, will rebuild it later today or maybe tomorrow. For now, you can get the package on transfer.sh (link will be valid for next 2 weeks)

[sapioit]

You need a asar.json inside the .asar, to tell:

• recomanded version
• compatible versions
  Once that's done, you need one app using electron (any version of it) to check the web for (1) updates for itself (the checker) and (2) the existing versions in your computer, then if a compatible version exists it parses the opening path to the opened .asar, otherwise download the recommended version and check again. The app, itself, can be written in Electron, or python (as it would only require minimal or no UI).

[sapioit]

The idea is: The app which checks if the electron version required is downloaded haves nothing to do with the electron versions needed.

If need arises, you could update the content of default_app, as well as the rest of the app, excepting it.

And in there, you can have the versions saved with app.setPath("userData", __dirname + "/ElectronVersions"); then those versions could be left there alone, and new ones installed as need arrises.

[probonopd]

Instead of fpm, have you had a look at AppImageKit.

There is an AppImage of Atom available at https://bintray.com/probono/AppImages/Atom/_latestVersion#files - just download, chmod a+x, and run on any not-too-outdated x86_64 Linux distribution.

[sapioit]

The problem is not just linux, we need a way to make it cross-OS, which means we need to be able to have it runing on Linux, Windows and Mac OSx.

[YurySolovyov]

Some kind of os-specific launcher that looks up if runtime is available and up-to-date/compatible with asar package

[yan-foto]

(many years later) Let me summarize the discussion:

1. @zcbenz proposes a runtime which manages local Electron instances so .asar packages are only distributed and bundling Electron binaries are avoided. The runtime manages also execution of .asar packages.
2. @YurySolovyov suggests enhancement of manifest file with additional information (e.g. required Electron version) so runtime knows which version is to be used for which .asar package. An overview of this approach is given here.
3. @joshuawarner32 introduces the idea of multiple runtime versions, binary packaging (e.g. as .deb), and autoupdate for .asar packages. Moreover the idea of a distribution server is considered.
4. @paulcbetts suggests to take a look as Steam model and implement something similar. He points out the complications bound with the idea of a local runtime which supports different Electron versions, etc.
5. [people discuss internet speed in china and their childhood affairs]
6. @hokein points out the complications when the runtime is upgraded. Specially wrt. native modules which are required to be built again against Electron headers.
7. @etiktin wants to have LTS and bleeding edge versions so production-ready apps use LTS rather than bleeding edge.
8. @Tribex says if we stick to semantic versioning we don't need to have a local instance of _all_ versions, but only those which are incompatible.
9. @jorangreef provides an example of differential auto-update system already in production for their own application.
10. @iamale has built an fpm package of Electron to execute .asar packages.

_disclaimer: I tried to include only main points of discussion. Please excuse if I missed anything!_

[yan-foto]

I read the discussion up to here, tried to summarize the main points and wanted to address some of them. The idea of a runtime with "deep platform integrations" goes beyond a simple local instance. A minimum viable product is to be define, which acts as a system-wide runtime. I also added a nice-to-have feature which might come handy. The last section describes what changes/features needs to be implemented.

Rationale:

What is being done by most of developers (i.e. bundling Electron binaries) is like linking all shared libraries statically! It's like delivering an oven with every pizza we sell.

MVP:

• Environment manager: keeps track of locally available Electron instances. It can be used to decide which package is compatible with which instance (using enhanced package.json [see bellow]). It should also take care of setting correct environment variables for packages to run.
• Update manager: the possibility of of automatic updating (+ notification) and differential auto-update has been discussed. It would be nice to have it as a daemon which takes care of updating local instances to latest versions.

Nice to have:

• Package/Dependency manager: just like Atom's apm, a package manager is needed to download .asar packages (from GitHub or a given URL, etc.) and install them locally (comparable with apt-get in Ubuntu or brew in Mac).

Requirements:

• Enhanced manifest files: package.json should also contain a semver (or semver range) to denote which Electron version is required.
• Defined installing/uninstalling procedure: ATM the .asar package contains everything needed to run an application. It is, however, possible to deliver only the source and download dependency modules at installation. If we get too courageous we can have some kind of _global_ directory containing dependencies for all applications (npm 3's flat structure might help). This, however, can get really complicated, really fast (e.g. when uninstalling an app).
• Multiple local runtimes: if semantic versioning has been applied up to now there are only two incompatible versions out there 0.x and 1.x. The minor versions and patches should be backwards compatible, so we only need the latest version of 0.x and 1.x on the system.

---

Inspirations:

• Environment/dependency manager: Python's virtualenv, Ruby's rbenv, and also see ruby-build.
• Package manager: Python's pip, Atom's apm.
• Update manager: Chromium's zygote process has an interesting approach to _silent, background_ updates.
• Code translators (to adapt to new APIs): Pythons 2to3.

[Simran-B]

epm (electron package manager) seems to be taken already as name...
Maybe call the runtime elec and the apps .tron? There are probably some trademark issues with TRON but it would sound cool ;)

[sapioit]

Maybe uepm (universal electron package manager), since it-s meant to be run on all OS-es supported by Electron... or epmc (electron package management center) and the file .epack (electron package). The extension, at least, sounds better than .tron and cannot be confused with a movie ("gimme that tron thing, dude" _hands you the movie_).

[kapouer]

There are also a host of words in the theoretical physics field related to electron

• fermion
• lepton
• charge
• coulomb
• pauli

And, not related directly, but somewhat more accorded to what's happening with a package manager,

• boson
  ;)
  (i believe photon has already been taken)

[YurySolovyov]

@kapouer +quark? not sure if it is taken though

[sapioit]

Quark is taken, the rest I don't know. But why must we keep the status quo, when this suggestions list is suggesting NOT maintaining the status quo? But since we're at it, why not name it Hauldron? Taken fron LHC. Collider is not a good idea, it's giving the product a negative shade, because it's meaning is tightly linked to war...

[YurySolovyov]

Particle(s) ? Quite long though.
List of particles in wiki

While naming is important, I still think we need to investigate runtime/update/packaging/distribution semantics first.

[gunn]

lepton seems perfect since its the family of particles to which the electron belongs.

[notpushkin]

A bit out-of-scope idea: a more generic runtime manager would be even more awesome (and not really harder to implement). For example, we could include @nwjs binaries as well and manage it in the same version-manager-thingy as well (just like @asdf-vm, but for GUI runtimes).

[notpushkin]

Quick and dirty: https://github.com/iamale/lepton. Supports only fixed version for now (semver is the next thing to do), and apps are unpacked (i. e. no .asar-s yet).

[sapioit]

Nice!

[pavankumarkatakam]

+1

[sapioit]

Even if no .asar, @iamale, as long as you have any way to know which file to open, I think it should be alright. Like, for example, you could have a file.something with json code linking to package.json so it knows what to open...

[YurySolovyov]

This is quite crazy idea, but what about using electron field in package.json to make
any (well, not like ANY, but any prepared) npm package executable?
Like:

electron: {
  "main": "index.js",
  "runtime": ">= 1.2"
}

This could be achieved by adding ability for electron to install npm packages and making them executable and visible for host system.
Just like

electron install atom

For this to work, we would need to split electron command and actual runtime bundles.

[sapioit]

but what about renaming the package.json to package.electron and have it's content something like this?

run:{
  "name"    : "zDillinger",
  "version" : "0.0.1-1",
  "main"    : "main.js"
},
electron: {
  "recommended": "1.2",
  "compatible": [
    "1.2.1-1.2.5",
    "1.3,1.3.0.1",
    "1.1"
  ]
}

In this example, it would recommend using electron 1.2, if it is not found also checks if there is any version between 1.2.1 and 1.2.5, if not checks for 1.3.1, then for 1.3.0.1, if not it checks for 1.1 and if not, downloads 1.2.

Maybe there could be something like it automaticly downloads the recommended version, and it it cannot, then it checks for the compatible versions. And take each and every one of the compatible ones and check them, and if none work, ask the user if they want to not use a incompatible version, and if they press no, it would run with a incompatible version found on the PC, and if none is found, instead of asking to use an incompatible version, show an error pop-up saying that there is no Electron instalation detected.

[JoshTheDerf]

@sapioit Having a separate manifest sounds like a good idea to me. Having a custom file extension would simplify things quite a bit. (Though .electron seems a bit long.)

Maybe according to @iamale 's lepton, .lept?

[YurySolovyov]

I don't see any benefit from not using package.json, electron field does the isolation part. Babel and ESlint use this technique and doing just fine.
This makes it much easier to publish apps directly to/via npm.

[lenovouser]

Yeah, I second @YurySolovyov - I think putting it in the package.json is more appropriate

[JoshTheDerf]

@YurySolovyov The advantage of an additional file is that the runtime can register a handler for the additional file's extension and run automatically as the default opener of the file. Whereas to do this with package.json would require registering the handler for every JSON file, which would be a pain for users.

[YurySolovyov]

@Tribex fair point.
Maybe package.electron can be just some kind of link that tells the runtime "Hey, take a look at package.json, it should contain the info on how to launch it". This was a "crazy" idea, so yeah, I'm fine with tweaking it.

[sapioit]

I agree with @tribex, we do need EITHER a separate file, or to rename the package.json. to something else.

[sapioit]

@wurysolovoyov you can't do that. The only thing you can open a file is having an extension attached to it, in which case every extension will open with the same software. So unless you're ok with being unable to open a .json with anything else than electron (which would just error out, unless a proper electron app is opened), you can still do that, but let us, the ones who want to be able to open .json files, be able to open them with the program we want to open them with.

[sapioit]

Otherwise you won't be able to edit the file, without either opeining the editor first, or changing what to open .json with every minute or so.

[sapioit]

I mean @YurySolovyov not @wurysolovoyov

[YurySolovyov]

Maybe I don't stated it clear: the purpose of package.electron is only to serve as a marker that this directory is an app, so that runtime can refer to package.json that is in the same folder, and launch an app. If you open package.json directly, it will open up in whatever is associated with .json in your OS.

Sort of like shortcut system.

[JoshTheDerf]

@YurySolovyov In that case it's just a matter of taste as to if we define that information in package.json or package.electron. I would personally keep it separate simply because it seems cleaner to me. Perhaps the engines property in package.json would suffice, though.

[YurySolovyov]

@Tribex it is also a requirement for publishing app to npm - having valid package.json

[JoshTheDerf]

@YurySolovyov Right. If we used package.electron package.json would still be present. .electron would only be used to define runtime information, or, in the case of your suggestion, just be a marker.

[sapioit]

Basicly, the package.electron would only be there as a launchable _marker_, so the package.json would still be present with it's own data, but that's why a separate file would be needed: _double click opens the app_. I would still like to be able to rename the package.electron to my-app.electron or my-app.electron.

[YurySolovyov]

@sapioit sure, that's the whole point of having .electron file, you can name it whatever you want, as long as it has .electron extension.

[JoshTheDerf]

It still doesn't seem very clean to me to have the runtime handler have to look at *.electron, then at package.json to determine which version of electron to run, which then reads package.json again.

[YurySolovyov]

@Tribex why do you think package.json would be read twice?

• If we put runtime info into .electron file, runtime would launch app using package.json assuming everything is correct and fail-fast if its not.
• If we put runtime into package.json directly or under electron section, that would also require only 1 read. This way, I'm not sure what to put into .electron file (ideas?), I'd be fine even with it being empty.

[sapioit]

@tribex The .electron could point to the package.json, just in case, for some reasons, the package.json is renamed. This way, you could have a package.json for NPM and a different package.json (maybe electron.json) for the Electron. Why? Maybe you want your app have the package.json to know what things to import with the package manager, but once imported, you want to run it without needing to move or rename files.

_app.electron_

{
  "package": "package.json",
  "recommended": "1.2",
  "compatible": [
    "1.2.1-1.2.5",
    "1.3,1.3.0.1",
    "1.1"
  ]
}

or

_myapp.electron_

{
  "package": "electron.json",
  "recommended": "1.2",
  "compatible": [
    "1.2.1-1.2.5",
    "1.3,1.3.0.1",
    "1.1"
  ]
}

[sapioit]

Or, if we put everything into package.json, we could have the .electron only link to the .json we want to use to open the app, so:

_app.electron_

{
  "package": "package.json"
}

or

_myapp.electron_

{
  "package": "electron.json"
}

[sapioit]

Also, maybe you want to have two launchers in the same directory, one for the client and one for the server, with the client using the files of the server to launch a local server with only one player. This way, you won't need to duplicate all 99% of the files (assuming you have a game that uses 500+ MB of files) and 5-6 files different, and have them in two separate directories, but just have one extra file to be able to launch the server on it's own. Literally, the same as having multiple executables in the same directory.

[StreetStrider]

Separate package.electron should be named package.electron.json or something that has extension .json for the sake of editors and IDE to determine syntax properly.

[YurySolovyov]

@StreetStrider I'd rather teached IDEs that .electron is actually valid .json.
And this also brings us to point we discussed with @sapioit about .json associations. I don't think any popular OS supports double extension associations. (.foo.bar)

[sapioit]

You can have the .json extension added when you edit it. or right click and open with an editor, but just like Adobe Air, which, by default, opens the file with their launcher, allowing the user to open the file with an editor, that would cause no problems.

[JoshTheDerf]

@sapioit Fair point, I guess I agree with you then about pointing to the package.json.

[baconbrad]

To me a runtime and a player are two separate things. This is coming off more as a player now. And it doesn't seem like it would get any widespread use outside of a technically inclined community that are used to the concepts of package managers.

The issue at hand is to create a runtime. Kind of like JRE and .Net. Not a player.

[JoshTheDerf]

@baconface Perhaps you could elaborate? It's unclear what the distinction you are making here is.

[YurySolovyov]

@baconface yeah, but both JRE and .Net are installable too.
Without JRE, .jar files are as useless as packages without electron runtime.

[JoshTheDerf]

Ideally this tool would be distributed with or as the primary method of installing electron, ideally alleviating the spread issue.

[sapioit]

For this you would only need a tiny tool to make an executable which would launch a certain file, namely the .electron imputed. Simple as that.

[notpushkin]

.lept looks awesome, thanks!

I personally think there's no need to invent any new configuration, pkg.engines looks quite straightforward to me here (it should allow for some complicated semver expressions though, like 1.x || >=2.5.0 || 5.0.0 - 7.2.3 from npm's docs, so no need for { recommended, compatible, etc }-structures like in @sapioit's examples, I believe). (I've added a lepton field though, because scripts.run is usually invoked by npm start, and lepton somewhat breaks semantics of it by adding its own $VARIABLES.)

[notpushkin]

Or, if we put everything into package.json, we could have the .electron only link to the .json we want to use to open the app

Doesn't seem to me like a great idea either. If we have a package with multiple entry points, we still could have them all in one package.json, like we have a scripts map now. I'm thinking of something like so:

// package.json
{
  "engines": {
    "node": "4.x",
    "electron": "1.x"
  },
  "lepton": {
    "run": "$ELECTRON .",
    "server": "$NODE ./server.js",
    "server-debug": "$NODE ./server.js --debug"
  }
}

# lepton-run [DIR OR TARBALL] [CONFIGURATION]
$ lepton-run . # the default configuration is `run`
$ lepton-run . server
$ lepton-run . server-debug

---

Then, if you really want this as executable files, on Linux / macOS you can use simple shell scripts (3 lines including #!shebang), and on Windows... well... ¯_(ツ)_/¯

[sapioit]

First, you might want to take a look at this article, which offers a better alternative to semver, while still being compatible with pretty much everything out there: Release.Breaking.Feature.Fix or why the Semantic Version should be replaced with Explicit Versioning as soon as possible

Second, the question is how to actually differentiate them. I mean, if server.js already accepts the --debug argument, why bother with that and not allow the people to have their own file configs. Maybe the server requires different configurations than the client, like the client working on 2.2-2.4 and the server working only on 1.4-1.9.2.1...
Doing as you sugested, we would only be more limited.

Also, what about using more variables, not only --debug? How am I going to run it with more variables like it? Is it safe to do that? After all, if I wanted one file to do that, I could have done it even having separate files, but this way it messes up with the possible outcomes of runing the .lept file with parameters.

If I wanted to run the server with the debug parameter, wouldn't I simply launch the .lept with the --debug argument? Or I could make another file to launch that file with the --debug argument. Isn't it that simple?
I hope I'm making sense...

[notpushkin]

First, you might want to take a look at this article, which offers a better alternative to semver

I think this is a great idea! I don't think Electron is going to switch to this versioning scheme, though. I also believe that while it's great for end-user products, we don't really need it for libraries and engines and runtimes and stuff. Ideally, user shouldn't even know which version of Electron they are running (unless they want to).

Maybe the server requires different configurations than the client

This could be a problem, but I can't imagine a project which would need more than one version of a runtime.

If I wanted to run the server with the debug parameter, wouldn't I simply launch the .lept with the --debug argument?

Yeah, why not:

# lepton-run [DIR OR TARBALL] [CONFIGURATION] ARGS...
$ lepton-run . server --debug --myoption=5
# resolves to:
$ path/to/node ./server.js --debug --myoption=5

This dual-config package.json is great for cases when the server a) shares a lot of codebase with client, and b) is intended to be run by end users. In other cases it would make sense to separate it all into two repos/packages, each with its own package.json.

[sapioit]

It's not for Electron, but for the projects you build. Electron, likely, has it's own system derived from the Semantic Versioning to fit their needs, and it's unlikely that they're going to change it, but for Lepton and other projects that aren't so far in _development hell_, it might be a good idea to skip Semantic versioning altogether.

If I wanted to run the server with the debug parameter, wouldn't I simply launch the .lept with the --debug argument?

Yeah, why not:

# lepton-run [DIR OR TARBALL] [CONFIGURATION] ARGS...
$ lepton-run . server --debug --myoption=5
# resolves to:
$ path/to/node ./server.js --debug --myoption=5

No, I don't mean in the config, but through the launcher. So this:

# lepton-run [DIR OR TARBALL] [CONFIGURATION] ARGS...
$ lepton-run . server --debug 

But launch the .lept with the --myoption=5 argument without touching the files (besides opening them). Since I'm pretty sure you can "_push_" the argument through the launcher, you can also remove the need for --debug, too, since you'll launch it with the custom argument if you want to.

[sapioit]

But on one thing I think we can agree: The users do want to be on a stable version that they are getting support for. That's why I made the "Release" it's own number, because the people are unlikely to remember many numbers, but they usually do remember the first one.

[YurySolovyov]

@sapioit if "Release" field is for end-users, what are other ones for? For end user this should not matter at all, if new version available and it requires updated runtime(via bump in package.json), then runtime should be updated too.

[sapioit]

Here are the problems:

1. Users want to know which version they're being offered support for.
2. They don't want to remember a lot of _useless numbers_, so if they choose to remember the version, they're more likely to remember version 13 than version 1.7 (for example).
3. Some people don't want to update untill a certain things happens, like how quite a bit of nVidia Drivers are not being updated because since the Thunderbolt compatibility was added, their graphics card(s) get randomly ejected (depending on the user/case, either after booting the computer, after opening a game or a few minutes into playing the game). This is a good example of a reason (but there are many others, like the KMPlayer's ads or locked on automatic updates) which keep the users from using the latest version of a product. This makes it easier to look at the changelog, by removing complexity from it.

Those are just some of the reasons... if you could go into more details, I could also go into more details and explain you better why those choices were made.

[YurySolovyov]

@sapioit Isn't semver ranges like >= 12.1 flexible enough for declaring support?
If there is a new version of runtime available, and it satisfies version range, all next launches of an app should use latest possible runtime.

[sapioit]

Should they? But what if they decide not to, due to technical limitations (like the thing with KMPlayer's ads and nVidia Drivers)?

[YurySolovyov]

@sapioit Then just make sure you stated your runtime range properly

[JoshTheDerf]

@sapioit @YurySolovyov Is this discussion relevant here? A runtime manager would have to follow whatever versioning system electron is using. I believe a discussion of what versioning system electron _should_ be using belongs in another issue.

[YurySolovyov]

@Tribex somewhat. I was trying to say that we should not overcomplicate things that are already basically solved. But I agree that runtime manager should do simplest possible thing in this regard.

[sapioit]

@yurysolovyov So why not simply allow as many sections in the version number as the player wants?

[notpushkin]

@sapioit

It's not for Electron, but for the projects you build.

Lepton doesn't have a versioning scheme assigned (yet), and packages (apps) can use any versioning scheme they want. We need to handle this (for example, if we want to support updating apps through the Lepton itself).

Runtimes, ideally, should use semver, as they're not products, but again, any 3-number versioning is supported now (developers would need to be careful with specifying ranges, though). If we ever need to support more than 3 numbers (like with your Explicit Versioning proposal), we'd need to extend the parser (for Python, I suggest forking k-bx/python-semver or podhmo/python-semver — the former has updated more recently, while the former follows the node-semver API more tightly and is what I'm currently using).

[championswimmer]

Can I just say that having a runtime package manager written in python for a nodejs project is just a bad idea.
Just let's not do it. Whatever it is called, lepton, or whatever. Let's write it in node, or use something else. And let's not spam this issue with needless mails about how you think you know better than the well thought semantic versioning scheme

[JoshTheDerf]

@championswimmer It would be interesting to build the runtime manager with a version of electron packaged with it, and a UI to show when other versions are being downloaded.

[notpushkin]

@championswimmer iamale/lepton is sort of kind of throwaway reference implementation: I just put it up to test various ideas floating around this thread, nothing more.

[YurySolovyov]

@iamale maybe you should state it in readme to avoid confusion then?

[notpushkin]

@YurySolovyov Done!

As for the real implementation, we have two options: either we use node.js, or a compiled language like C/++ or Go; and for node, we'd have to bundle node.js with it (or maybe just run it on one of electron instances downloaded? We'll need to ensure that it runs on _any_ sane electron version, though).

[JoshTheDerf]

@iamale Not really, we just have to run off a default installed version. After that updates can switch it to newer version in-place.

[sapioit]

I'd rather choose to use one certain version, because if needed, everything can be updated...

[enlight]

Lepton might not be such a great name now that it's an image format.

[sapioit]

So... what do we call it now? Hadron?

[JoshTheDerf]

Quark? If someone wants to take the name "Muon" I can rename one of my projects.

[YurySolovyov]

Isn't naming actually the least important part here?
Did we agreed on install/update/runtime semantics?

[jobukkit]

Why not just electron...

Op 15 jul. 2016 om 13:22 heeft Joshua Bemenderfer [email protected] het volgende geschreven:

Quark? If someone wants to take the name "Muon" I can rename one of my projects.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[sapioit]

@yurisolovyov I think we agreed... more or less.
@job-v That's what I've been suggesting all along. Just let's pick one and stick with it.

[YurySolovyov]

@zcbenz are you observing this thread?
If so, it would be nice to have some vision on how this should work to get some directions here.

[ghost]

hi
any updates about this electron runtime ?
@zcbenz what do you think about all the discussion about :

• selecting electron version
• installing the appropriate version
• prepare it to work in win, linux, mac
• what tool use to build with it (python, node, go, c++) ?
  @YurySolovyov @sapioit thanks for your awesome ideas
  @iamale is your solution work for debian based destros 'ubuntu ...'

[notpushkin]

@alnour-altegani
Yeah, it would work for any dpkg-based distributions because we don't depend on outside packages. RPM packages can be generated in a similar way, why not?

I'm thinking of starting a central repo for all the Electron things, so i. e. you could run:

sudo sh -c "echo 'deb https://epkg.example.org/ /' > /etc/apt/sources.list.d/epkg.list"
sudo apt-get update
sudo apt-get install min-browser  # or whatever

[ghost]

@iamale I think make electron as deb package and use it as dependaces in application to be installed manually or from centeral repository for many electron version deb packages .
and then use this package after installation to run the user application

[sapioit]

If anyone starts working on something for this, maybe link to it here...

[notpushkin]

@sapioit I've started working on a Linux (APT/RPM) solution. A simple service that you upload .asar to and it gets converted to packages and added to the repositories. Should be extendable to Windows and macOS as well, but no promises on that for now.

[sapioit]

@iamale Cool! Let me know when you get it working on windows.

[moosingin3space]

What about making a Flatpak runtime for Linux? This way, the runtime can be shared between multiple apps, and you get sandboxing too.

With Flatpak removing its dependency on systemd it'll probably become universal on Linux for upstream packaging.

[notpushkin]

the runtime can be shared between multiple apps

I thought that with Flatpak it wouldn't be possible?..

[moosingin3space]

@iamale Flatpak is designed to share runtimes - app developers choose a runtime, install the SDK for that runtime, then build an app with flatpak-builder or similar. Then when the user installs the app, Flatpak installs the associated runtime.

[notpushkin]

@moosingin3space Thanks for explanation! It is definitely worth considering then.

[moosingin3space]

The other thing I like about Flatpak is that it has been designed around the use case where a user adds a repo run by the runtime dev and app dev (transparently) and this makes updates automatic.

The only things I can see being issues are related to security - need to modify system libraries to use the "portals" interface in Flatpak to allow safe exiting of the sandbox.

[moosingin3space]

There appears to be work on Flatpak here: https://github.com/endlessm/electron-installer-flatpak

[ebassi]

As a side note, Flatpak is based on the OSTree project, which does content addressing for file system trees. This means that files with identical checksums are automatically de-duplicated on disk, and that updates are performed using static deltas. This means that not only Electron could provide a share runtime that can be independently updated and shared across applications, but also that applications built on identical files would automatically share them on disk, and that once an application and/or a runtime are installed, updates would not require a full download.

[CxRes]

Forgive me if this is a bit naive, I am wondering if the possibility of a building a desktop environment as a natural logical progression of the runtime environment has been considered. The idea being that there is a launch app which electron loads on startup (instead of say, atom) with the ability to launch other electron apps (given the large growing library) by spawning new instances of the renderer or maybe even reusing it and have apps open in a new tab.

Chrome as in the OS in my understanding seems to be exactly this (but limiting itself to webapps). Having desktops apps with full fs access through nodejs (as opposed to limitations of chromeAPIs) with electron as a runtime environment on top of Linux (or even replacing or at least augmenting in the beginning explorer.exe in Windows) would be very exciting imho. I think this is a cleaner solution than ChromeOS which is loading what is effectively a runtime and then a browser engine and then hosting an app - here one will load apps directly in the runtime and browser itself will be just another app (and there are so many innovative electron browser projects out in the wild).

Again, I am wondering if this has been considered or if there is some project doing this already. If not, perhaps a fairy can put a word in the ears of Github management for taking a lead????

PS: One thing I am not suggesting here is porting OS.js (which is a cute little project in itself) to electron but creating an environment for electron apps (Though I wouldn't be averse to code reuse from OS.js).
PPS: I'll create a separate issue a issue if it is considered worthy enough or if it is a distraction to the issue of a runtime environment.

[sapioit]

I do think that a built-in runtime environment would make Electron apps much better. I think packing them should be just zipping the files and renaming the extension to asar, but that archive should have two folders, one called asar, with the content of the default runtime folder, and one called enviroment with files that should overwrite the default config.

For example, running a new app would copy the required version of Electron in a temp folder and move the runtime (contents of the asar folder) into the default files location, while the optional enviroment folder (with 3 subfolders, windows, linux, macosx) should contain files that would be pasted over the default folder, overwriting it's contents.

This way, we only need an app or something to assess the .asar extension to an app, and have that app access the folder with the downloaded versions of Electron (archived), copy (extract) them into a temp folder, and use the contents of the .asar to modify what's needed, then run the Electron instance from that temp folder and wait for it to die in order to force delete the temp folder. For starters, even only getting the default configuration to work with an archive with a custom extension on all supported operating systems would be leaps and bounds more than we already have. After that, we can worry about having a third folder, config, where to keep the current config settings, and maybe a _config folder inside it, for the default config settings (if we need them).

[bryc]

With Chrome packaged apps being phased out soon, people are going to be looking for alternatives. A runtime would be great for those minor projects that aren't always going to have builds/releases available. Since apps are made with web tech just like Chrome apps, it'd be valuable for a user to simply run the app from source. I can't justify porting countless minor projects to Electron if the only way a user can run them is to download a large binary release for each individual script or app, and users cannot take advantage of running the latest commit without building it or downloading Electron's IDE themselves.

[dezzeus]

Hi to all, even if I'm not in the JS-centric world, I recently had an idea quite similar to _Electron_ (which I happened to know, but not use for development) but with a broader audience; to put it in a very simple way: _just limit the web technologies to the GUI and provide APIs._

So keep goin' with a semantic-versioned shared library / runtime (built on top of _Blink_, _Skia_, etc; perhaps in – "modern" – _C++_) and let the developers choose the language by using bindings…
e.g. you want _JS_, use _Node.JS_; you want _C++_, no problem; you want _Python_, with the standard _CPython_ should be possible; anything else ? maybe _SWIG_ can help. In (almost) any case, we win.

Once we have the shared (OS-dependent) library / runtime, you resolve the main issue and can elaborate whatever way you think it's best to deliver the rest (other than revolutionize/modernize the GUI-programming approach for the rest of us; a huge step forward!).

So the starting point should be the "split" (should even have a physics-related name, given the fact that you seems to really care to it) of the _Electron_ project… :)

[sapioit]

We can't even get done an addon to Electron, but I think that will not have that much success, unless lots of advertising.

[CxRes]

@dezzeus There was an effort on this lines before which fizzled out. Have a look at the thrust runtime intended to host the breach browser.

[eliot-akira]

I read through this long and fascinating thread twice, as I'd love to see more momentum in this direction. Here is the last statement by @zcbenz, I believe is relevant still:

There is no technical obstacle on implementing this, the whole thing can just be an Electron app itself. But the problem is it is not easy to implement neither, taking all the details into account, I think _it requires a small team working full time on it_, I'm not sure whether it would ever happen.

(Added emphasis) - How I interpret this is, if the solution is ever going to be built, it will depend on community effort and/or business needs of GitHub and other organizations with a stake in it.

A concise overview by @yan-foto looks like a sensible plan of approach, with clear goals/requirements. There's a reference implementation/prototype called lepton by @iamale, proposing a "universal runtime version manager" targeting Electron. A comment by @jorangreef was intriguing, as he describes "an app in production with a semver auto-update mechanism", with differential update, deduplication (not sure if those two are the same) and compression. Many of the features he described seemed perfect for adapt/adopting.

These are definite steps of exploration and progress, and it gives me hope that production-worthy general solution(s) will be born from this process.

[sapioit]

It's nice to see there are still people with faith in humanity.

The only thing keeping this from becoming a product is the lack of immediate business opportunities opened by the making of this product (concept).

[jorangreef]

@eliot-akira yes, that's what it is. The code I have written does per-file content-based deduplication (variable-sized chunks) within a file to minimize downloads when only a few bytes within a file change. It also updates only those files which change between versions and updates to the application as a whole are atomic in the face of any errors. Either the new version will launch or the old version will remain in place.

I would like to open-source the code, and there are some things I would still need to code for the overall idea. It would be great if multiple Electron apps could share a single auto-updater supervisor process (with a local shared immutable cache for deduplication across multiple independent Electron apps).

The goal of this whole approach would be to minimize overall download bandwidth required to install and update Electron apps, while keeping Electron apps completely isolated from one another (multiple apps would have their own Electron binaries). A non-goal would be to minimize disk footprint.

The first Electron app install would need to be downloaded in full, but subsequent Electron app installs would benefit from the local shared immutable cache.

[sapioit]

Since it's come to auto-update, I have to bring in the versioning problem, and the fact that multiple versioning systems seem to be using different numbers of digits for versioning. I, for one, am using Explicit Versioning.

[notpushkin]

We have two types of things that need to be updated here:

• Electron itself, which uses Electron Versioning and
• Apps, for which we don't really need to care about SemVer or anything as they generally do not have an API (unless they utilize some kind of addon system maybe).

Although if we do need to support several versioning systems, maybe we could have some setting in package.json, like "breakingVersionComponents": 1 for semver and 2 for Explicit Versioning or Electron Versioning?

(Another option would be to add another separator in version, like 1.3:3.7 which would separate breaking and non-breaking parts. Semver then becomes x:y.z and Electron Versioning x.y:z. This would break every single tool out there, though, I believe.)

[jobukkit]

@jorangreef If every app still has its own Electron binary, this would not fix the problem of excess RAM usage when having multiple Electron apps open.

[eliot-akira]

I noticed an older comment by @joshuawarner32, which I'd overlooked. He mentions an already-working example, not a general-purpose solution yet, but the summary is:

- Deploy .asar.gz files for several different apps on a server
- Distribute a single build of atom-shell to users, which doesn't bundle any code for specific apps
- On app startup, download the appropriate .asar.gz file (if it's not there, or there's a more recent one on the server). Extract it and run the contained app.
- Our atom-shell build accepts a --app argument to specify which app to download/run

Along with @jorangreef's example (thank you for the further explanation - edit: oh, I didn't realize each app has its own Electron binary, which is a different model than I thought), I see that people are already building their own specific solutions to the problem of a shared runtime - which also provide features of an app launcher/manager of some kind. It would be great to see some convergence, to break out some of these features mentioned into general-purpose modules.

I wonder if there are existing, non-Electron-specific modules that could be leveraged, like the download mechanism: runtime server, versioning, differential updates. Seems a bit daunting to build it all from scratch, if others have "solved" some of these areas already. @iamale, thanks for your work on Lepton, in my eyes this is a concrete implementation in the direction of a solution.

[notpushkin]

Another idea: just publish apps in npm, and make a special npm wrapper that can install and run them. It would be really easy to deploy then and probably easy to use (dependino on quality of the wrapper though) as well. What do you think guys?

[YurySolovyov]

For oss projects, we could use npm as hosting, for closed source ones, we need to be able to point package manager to .asar file to download, and maybe some more metadata. The question is, what this kind manager will look like (and called)

Also, I wonder if we really need a separate command to install apps, maybe just
electron install atom would be enough?

[zeke]

Just saw this on Node Weekly:

Electrino is an experimental featherweight alternative to the popular and powerful Electron. It implements a minuscule portion of the APIs available in Electron, but the output app size is much smaller.

https://medium.com/@pauli/put-your-electron-app-on-a-diet-with-electrino-c7ffdf1d6297

https://github.com/pojala/electrino

[notpushkin]

@YurySolovyov Closed source projects should have no problem with being published in npm as well — there are a few Node.js ones out there already, i. e. enclose. (Technically, it isn't in npm, just a loader, but I anyway I can't see how sideloading .asar's would help as they still can be unpacked.)

[YurySolovyov]

I was just trying to think how can we re-use as much of existing ecosystem as possible.

[sapioit]

Not sure we want to...

[jorangreef]

@jorangreef If every app still has its own Electron binary, this would not fix the problem of excess RAM usage when having multiple Electron apps open.

@Jop-V No, it wouldn't. I don't want to conflate things and end up with a monster. The idea is to have a great installer/auto-updater that solves most of the problems in the comment that opened this issue: https://github.com/electron/electron/issues/673#issue-44580957

The problem is not so much having multiple binaries on disk, but the network impact on a user having to download and auto-update multiple copies of the same binary.

Solving memory usage is definitely worth doing though. I think there are a variety of ways to do that but they don't necessarily need to be bundled with a deduplicating installer/auto-updater.

[jorangreef]

If disk footprint is also an issue, that can be addressed through hardlinks on systems which support it, although that would start to infringe on the idea of isolation between apps (e.g. if one app decides to write to a hardlink shared with something else).

Alternatively, at least on macOS in the medium term, APFS will copy files by reference, with transparent deduplication under the hood, minimizing disk footprint. This would be better than hardlinks since it would keep isolation intact (copy-on-write).

[notpushkin]

@jorangreef What's the point of using separate Electron binaries for apps? What kind of isolation are you talking about? Sorry if that's a stupid question, but I just don't get it.

[jorangreef]

@iamale Some apps may target different official versions of Electron, or may distribute a pre-built Electron compiled from modified sources. This approach would work with them all. Even with major differences in binaries (as opposed to JS and asset changes), the deduplication would eliminate 70% or more of download requirements, so apps targeting different binaries would still benefit.

With this approach, it's even possible to ship any kind of package (any set of files and directories defined by a JSON manifest), not necessarily only Electron apps. It may as well be a Python app. It's agnostic to any choices the developer wants to make towards how they structure or package their app (files vs asar etc).

The main idea is that the ecosystem benefits from very small install/update deltas as more and more apps use the same installer/auto-updater. These days, every app is running their own supervisory auto-updater and it's unnecessary.

[notpushkin]

@jorangreef Thank you! I guess it's much like classic APT/RPM repositories, just more “smart” (no need to declare your dependencies or whatever).

[sapioit]

The question is, what this kind manager will look like (and called)

What about Minifitron (minified electron)?

I think the archive file (i.e. .asar) shall contain the contents of the folder where the binary of electron is, so we can add or replace the files that deviate from the standard. We shall have each version in a folder or archive and copy it to a new temporary folder for the time it is run.

And there should be an option to have the apps installed or minified, the instalation would leave them as a new instance of electron, while the minified would use md5 to delete the files that have not been changed by the unarchiving of the .asar (or what we would be using) and archive that back and replace the original file, as a updated state of the app we used.

[eliot-akira]

Just to throw in another idea: what if each Electron app did contain their own copies of runtimes, but upon launch, they could check if there is an already-running instance of Electron (and its version) - if it's compatible (within semver range, etc.), the app could communicate with it and use that same instance; if not compatible, then the app can start its own instance of Electron. That would seem to address the issue of reducing memory foot-print.

(Then again, by the time the app is launched, I guess there will already be multiple instances of the runtime anyway..so might not be logical..)

[jkurei]

@eliot-akira Had a very similar idea while reading this thread.

(Then again, by the time the app is launched, I guess there will already be multiple instances of the runtime anyway..so might not be logical..)

Well, it could load just enough to check this, and after it sees there is not an available "electron mate" to share runtime with, load the rest.

This would create a lot of security concerns though. An evil app could offer a modified version of the runtime to others, and access their data. May be it could use a shared memory segment, make it read-only, and "user" apps (that use the runtime of a previously launched app) could checksum the runtime, but developing these for all the targeted OSes might be a nightmare (I don't even know if this exists in Windows; I assume it must).

If only for curiosity's sake, is this non sensical? Would it be as much, or may be much more work than a separate shared runtime? Does it create other additional issues?

ADDED LATER: This might be a far shot, but there exists technologies to deduplicate identical pages on RAM; they're used mostly in virtualization environments and hypervisors. This is the first thing I thought about and ran to check if today's OSes use this, and then may be there would be a way to set electron in a way that this feature can be taken advantage of, but apparently not.

[jobukkit]

@jorangreef If a developer modified the Electron binary, shouldn't they just create a pull request?

[notpushkin]

@Jop-V They can, but they can maintain their own fork as well. They may even not publish sources if they don't want — MIT license puts no obligations on them.

[eliot-akira]

@jkurei To explore the idea further, I imagine a couple ways that may work:

1) A thin native wrapper around Electron or the app itself, that would be the "launcher". It checks for an already-running instance of Electron and its version: if compatible, hands over the rest of launch process to that instance and exit; if not, continue launching its own instance of Electron.

2) A JavaScript library that apps themselves could import into the main process, as a launcher. It could implement a protocol similar to above: broadcast the existence of an Electron instance, and a way for other apps (using the same library) to find and communicate with it - maybe via network port..?

[notpushkin]

@eliot-akira I think, by the way, that there is already some kind of primitive "protocol" you're talking about; it's that thing behind app.makeSingleInstance that somehow detects already running Electron processes and if they are the same app or not (probably by getting full executable path).

[eliot-akira]

@iamale I see, the method uses Chromium's native ProcessSingleton class, to register the current app directory (as a unique identifier?). Yes, I was picturing something like this, to be initialized before app.on('ready'). So, instead of just checking for existing instance of that particular app, it could check for any instance of Electron - then "hand over the launch process" if compatible.

For another angle, I found a module called node-ipc that can communicate between separate Node.js processes. Maybe that could be used to manage the launch process purely on JS layer.

[eliot-akira]

Here's a basic example of sharing a single Electron instance among multiple apps, using node-ipc for inter-process communication: https://github.com/eliot-akira/singletron-example.

The IPC hub is its own module, a thin wrapper around node-ipc. Its initialization is in the main process, main.js at the bottom.

When started, the app will first attempt to connect with an existing Electron instance. If there's none, it will start a "server" that listens for inter-process requests. Any subsequent instances of the app will connect with the first instance, receive its "config" (versions of Chrome, Node, Electron), and is able to exchange messages, for example, to request a new window for itself.

It's a naive implementation to start with, but what I like about the approach is that it's up to the app to handle how the server/clients should negotiate: checking compatibility, whether to open new windows, pass other config, etc.

[notpushkin]

@eliot-akira I think specifying a default server ID is a _really-really-bad_ idea at this point: if anybody starts using this for their own purposes, they'll likely just modify your example code for their own needs and break protocol (thus breaking other apps relying on this).

Apart from that, looks good and clean, thank you!

Re: security concerns: Electron apps are already running with user's privileges, and if we place them in one process tree, they'll indeed have (or be able to gain) access to other Electron instances. However, I think if we're going to mitigate that, everybody being able to supply a patched Electron instance certainly won't help.

[eliot-akira]

@iamale Thank you for the feedback. I exposed the option to set server ID, in case some apps wanted to implement their own protocol, rather than a common one for all Electron apps. In fact, there's no such common protocol yet - so, at this point it's not practical for that purpose, for unknown apps to connect and hand over launch process. It was meant to be a proof-of-concept, so I will update the doc to mention that. I'll keep exploring how to make it more practical.

[CiriousJoker]

I think it would be helpful for newly coming people to have some sort of summary again, similiar to the one someone did half a year ago.

Also, what does the end user do?

If I were the end user (and partly I am ofc), I'd want one single .exe/.app/.deb package that works kind of like the portableapps.net installer. That exe file should be small and download the latest upstreamed version of the software automatically together with the Runtime if it wasn't installed. In a simple setup process I should be able to set a "runtime location" (using the advanced options checkbox) manually together with checkboxes for creating system links (Desktop, start menu, etc.).

When I download the next software, the previous runtime (whatever that looks like, idc) should get reused if possible.

As a developer, I'd want to be able to create such a downloadable package with (at best) one click. This one click should upload my code to some repository (similiar to the play/web/app store) and create a .exe/.app/.deb package for me that I can share with the world.

I'm not that involved with Electron yet, as I'm just creating a small mod installer using Electron and Electron-Edge for C#.

I may be naive, but afaik, the only "real" reason, why things aren't compatible +-0.1 versions is that .node files have to be rebuilt for every version of Electron. Couldn't these .node files be built during setup using some sort of cloud solution?

Also, would it really be that big of a deal to create one big Electron LTS every half a year or so? I doubt that devs are actually using all the bleeding edge features of every new Electron release, so if we (kind of) force them to stick with one version for longer, that would really make things a lot easier.

Imo, the only problem is not having an LTS version of Electron ready for everyone to rely on.

With that lts version, building autoupdating, deduplicating, version management and all that other stuff should be a piece of cake.

The installer solution I proposed also has the added benefit, that the user doesn't really see the folder of the software, so we don't have to package it as .whatever and tell him to move it somewhere safe. Also, multiple "subexecutables" could be added during installation (for example for multiplayer games, multiple "profiles" etc.). These executables should just be called somelaunchername.electron, but in the Json format with information on how to launch the software (like what index.html to load and stuff like that). node_modules should be shared between executables (unless the dev NEEDS to use different versions of a module for each executable). This could be achieved by having three node_modules folders. One for each executable and one for both. In the package.json there would just be general information. Each .electron file should (imo) be able to overwrite the values in the package.json.

These .electron files could then be added to the start menu and other locations using either "runtime --appname" or a custom protocol like steam. Not sure which one is better, but the steam one has pissed me off quite a few times in the past.

Now if the dev really wants to ship a modified runtime, he can still package it the way it's currently done.

To conclude:
In my opinion, the next step is to create a stable lts version and to get developers on board by providing them with an easy packaging solution. The installation process could be improved later, currently it's enough to download a runtime and a launcher for these .electron files.

What do you guys think?

[develar]

@CiriousJoker electron-builder can create portable app and web-installer. As far I see, on windows shared DLL is a solution. Shared DLL also support reference counting, to automatically remove unused electron runtime. Currently, electron runtime on windows in the one big exe (except node.dll). If it can be splitted into dll, solution on electron-builder side is pretty easy to implement. It will also save memory, not only disk space.

@zcbenz @paulcbetts Have you considered using of shared dll? Is it possible to reduce size of exe and move all electron code into dll (so, keep only minimal code in the exe)? It is cool that since electron 1.7.5 shared C runtime is removed from node.dll and exe. Now, 30 MB of code is already in the DLL files (but exe is still big — 80 MB), so, I can start experimenting with it.

[CiriousJoker]

@develar imo, the real problem isnt how we can share dlls, but much rather how we can get a lts version of electron

[sedwards2009]

Does Chromium even do security fixes for older versions? I have a strong feeling that their policy is "upgrade to the latest version to be secure". Without security updates from Chromium, a LTS of Electron seems a long way off.

[CiriousJoker]

@sedwards2009 But are there really that many threats? Sure, I might be naive here, but we're not protecting the users from the open internet, but from (mostly) local apps. Running any .exe file is 10000 times more dangerous and writing .exe viruses is also 10000 times easier when even the installer for my legit electron app was called a virus by AVG.

So to me, "renewing" that security every 6 months or so is enough. Also, its way more unsecure to let every developer use their own packaged version of chromium, they could do way more evil with that anyway that we can't prevent (with a shared runtime or otherwise)

[YurySolovyov]

Yeah, security ends where app gets raw access to fs and networking.
You either trust app' vendor or stay away. <- that's for closed source apps.
For OSS ones, it is better, most of them at least have no bad intentions, but may contain bugs.

[CiriousJoker]

Okay, so how easy is it to get an lts version? Couldn't we just pick the current version and declare it an lts version?

The bigger issue will probably come up later, when we're trying to convince developers to build their apps for this specific version, but that's the last step anyway. For building a shared runtime we just need to pick a runtime first

[YurySolovyov]

I'd say this should be developer's decision - to figure out which runtime versions app should be working on.
Kind of like npm engines field.
Runtime's job here is to fetch needed dll or whatever and allow app to run.

[CiriousJoker]

But if we allow them to chose one of dozens of almost the same runtime which aren't compatible, we'd completely miss the point, wouldn't we? We're trying to avoid having multiple runtimes, not just deduplicating them. As someone mentioned somewhere at the start, deduplicating runtimes doesnt make much sense when there's a 1 in like 20 chance that the same runtime would be used again on the same system.

Or do you mean that we have most of the runtime shared and just switch out the different parts as needed? That would make sense, but I imagine it's hard to implement

[YurySolovyov]

I don't see how runtime can be shared, main size consumers are Chromium and Node, which are tightly integrated and I doubt they can be made to work with more than 1 version of each other.

I think node had something like that:

• some developers wanted latest v8, and were OK to tolerate breakage
• big companies wanted stability and long term support.

I'm not sure electron team has enough resources to support both, but I may be wrong.
Anyway, this does not conflict with versioning ranges, just need to indicate which versions are "LTS"

[CiriousJoker]

Well if every electron app uses the same runtime and just ships with the .asar package specifically built for that runtime, things would work out, wouldn't they?

[YurySolovyov]

Yeah, but you want newer Chrome and Node once in a while, right?

[CiriousJoker]

Ye, but I could live with having the same node/chrome for 6months np. Node has an LTS version anyway and chrome doesn't change that much from what I've noticed. Imo, the benefit of saving 100mb on every app install make up for the little speed boosts every now and then once a new version of Chromium / Electron is out. Also, once we got the developers on board, it shouldnt be too hard to reduce the time between new versions of the runtime. We could then support the latest version of the runtime and one or maybe two older ones.

Rebuilding the .node files every 6 months shouldn't be too hard as well I guess

[develar]

Electron-builder will support new option for all nsis targets — useSharedElectronRuntime. For now, all existing dlls will be, I think it is possible to reduce size of exe and move code to dll, but it can be done later, saving 30Mb is already a good goal.

1. Create signed assembly for all electron dll and publish to GitHub releases (bintray is not a option, because not reliable).
2. Nsis installer will download and install assembly if not yet installed. Security — assembly is signed. Electron-builder also supports installation per machine. And Windows provides additional level of security if assembly installed per machine, so, if need, you can enable it. Additionall option will be added to install assembly per machine in any case. Not yet clear — should it be by default true or false. I think false, to make sure that by default electron app can be installed without admin.
3. no need to worry about unused electron runtime — shared dll supports reference counting. On app uninstall, assembly will be removed and Windows will remove it if no more other apps use it.

[sedwards2009]

@CiriousJoker Re: security. I wasn't thinking of security problems were the application itself is malicious. I was thinking of electron apps which suck down and use untrusted content. i.e. any time an electron app shows a web page from the open web in a webview or similar.

Many apps don't expose themselves to the open web, and they would be fine running on an older electron. But the other extreme is something like Brave which as a web browser does nothing else than expose itself. :-)

[CiriousJoker]

@sedwards2009 ye, but if we start caring for everyone and their mum, we won't make any progress. There will always be reasons not do something. Not getting 100% of the devs on board is just a sacrifice we have to make. If they really need/want the security, they can have it manually.

[CxRes]

As for choosing an LTS, the most sensible choice imho is to bind ourselves to Nodejs LTS ie which ever version of electron corresponds to the node LTS is effectively electron LTS. This could bring some harmony and agreement so that we can move forward collectively and have some surety rather than leaving it as a free choice (which negates this community) or arguing (which leads us nowhere).

[CiriousJoker]

@CxRes But what version of Electron do we support? In my project, the main problem was to rebuild the .node files against the specific version of Electron (not Node).

We could just set the newest Electron version as the lts or use the one most widely used.

Am I missing something here? Imo, the problem is Electron, not Node

[CxRes]

Ok, let me try to explain this again with an example:

• Node LTS is currently on 6.11.1
• Hence we want use the highest version of electron that was officially built with highest version of node that satisfies the condition <=6.11.1 >=6.0.0
• Highest version of electron built using Node 6 is electron 1.4.15 which is built with Node 6.5.0
• So electron 1.4.15 is where we fork for electron LTS and we rebuild it with 6.11.1 (and so on till there are new Node v6 releases)

When Node LTS moves to v8 (LTS are even numbers only), we will together migrate to an as yet unreleased electron which will be built using Node v8 or remain on current LTS fork till such an electron version is released.

[YurySolovyov]

What about devs who want latest and greatest?

[develar]

@CxRes @CiriousJoker Let's avoid off topic here. No any technical reasons prevent using of different Electron versions. Fragmentation will be in any case (developer released app one year ago and forgot, other app from another developer released recently and uses another version).

[CxRes]

@YurySolovyov The entire idea of LTS is that it is not latest and greatest - its stable. And if we cannot agree to focus on one solution - we are doomed to suffer from the Pareto sub-optimality of not having any runtime because no one group has time and ability to rebuild every version and screen them for bugs.

Also see @CiriousJoker comment above:

ye, but if we start caring for everyone and their mum, we won't make any progress.

[CxRes]

@develar Ok. But as you point out this is not a technical problem, it is political (getting to a agree on a build that works for most people consistently such that the burden gets shared).

[CiriousJoker]

So what do we do now? Do we pick a lts version and get started or try to find a solution to make it work for everyone by trying to build a one-fits-all runtime with modular dlls (as @develar suggests if I understood correctly)

[develar]

@CiriousJoker Side-by-side assembly for particular electron version. e.g. ElectronUserland.ElectronRuntimeAssembly.1.6.11.0.

[CiriousJoker]

@develar I cannot find that electronruntineassembly thing on Google, could you provide a link or smth?

[CiriousJoker]

Imo, to speed things up, we should just pick a random Electron version as the lts version and get started. We can change the version anyway at any point later on.
@develar I still don't know what you mean with that side by side assembly, but it has to work crossplatform and I imagine, we can implement it later as well.

We (or actually you guys) have been talking about this for years, but if noone starts, nothing is gonna change and we'll sit here for another 3 years discussing ideas and finding ways to not get started.

[cat998]

I hope electron developers can solve the separation of running environment and application as soon as possible, which will greatly promote electron applications, and I will see a bright future.

[CiriousJoker]

They didn't solve it for what seems like years to me, so I'm kind of pessimistic about that. Sure, it would be awesome, but are they even actively working on it?

[develar]

@CiriousJoker As stated above, I am working on shared electron runtime support for windows in the electron-builder. I have filed https://github.com/electron-userland/electron-builder/issues/1942 to track progress. I hope it will ready to use in 1 month.

[CiriousJoker]

@develar My bad, must have overlooked it.. In that case, good luck, would be awesome if you could make it!

[jorangreef]

If anyone is interested in the subfile content-defined-chunking deduplication referred to earlier in the thread in https://github.com/electron/electron/issues/673#issuecomment-157980607, I have open sourced it: https://github.com/ronomon/deduplication

[pistacchio]

This long thread has been dormant for the last months, but I think the problem it addresses is still worth investigating. During the weekend I made a small, minimal one-page utility to mass-rename files. Packed with electron-builder, the final Mac .app file is 121mb ...

[martpie]

I was thinking about this problem lately. Then I had a look at my smartphone: the Facebook app is 600MB big. Chrome is 325MB big, Mesenger is 300MB big... So a 120MB app on desktop, I do not really care...

Size is not a problem today. RAM and power consumption is.

_edit: if you disagree, feel free to share why_

[eliot-akira]

For the Windows platform, @develar's efforts with electron-builder (electron-userland/electron-builder#1942) looks close to achieving shared runtime.

Also noticed the app webcatalog, which packages web apps as standalone Electron apps, solved the issue of using a shared runtime, to massively reduce app size: webcatalog/webcatalog/issues/171. It was done in their packager, by symlinking shared resources among apps.

[joshas]

Progressive Web Applications (PWAs) are partial solution to this problem. On Android you can add PWA to home screen and it will act like real apk. Chrome team is now working to enable PWAs to be installed on the desktop (http://www.androidpolice.com/2017/12/05/google-wants-progressive-web-apps-replace-chrome-apps/), Microsoft also shows interest in supporting PWAs (https://www.windowscentral.com/faq-progressive-web-apps-windows-10) and I hope Firefox and Safari will catch up soon. Missing APIs (e.g. native file system access) will be added if developers show interest.
Most of web based Electron applications can easily be converted to PWAs, but I'm not sure about ones like Atom, VSCode - native code probably will need to be rewritten in WebAssembly.

[sergeysova]

@develar has any progress about this idea?

[jtoy]

I'll use when released!

[octacian]

Having recently come across this, I just wanted to throw my opinion in. I haven't read the entire issue though, so I may repeat some things unknowingly.

I think that something like this is very important, as although the size of an Electron program may not seem like such a big deal now, you really begin to see it when distributing a very simple and small program - say, a custom calculator. In the case of this, something like 5-10MB would be reasonable. 60MB or more would just be ridiculous though. For a larger program, the size of Electron itself becomes more ignorable, but it still can be annoying.

Now, such a runtime mode would, however, require some type of LTS release system considering the current rate of releases. Developers could then be encouraged to depend on an LTS release, and a list of the most common versions in use could even be displayed in the apps directory, encouraging them to use the most common unless it is lacking in a required feature.

The runtime mode itself should, IMO, have a builtin system for installing other versions of the Electron library as well, so that should the user request to install an application that requires a different version, the runtime program can then download it automatically as a simple step within the installation process.

I think it should be noted as well that the RAM usage of Electron apps could also be improved with the help of this, although the changes required to put it into effect would be far more major. With a shared runtime, Electron apps using the same version could share some resources within the memory.

Anyways, I'm very new to the entire source of the Electron project anyway, so sorry if anything was unclear, or if I repeated something needlessly. I do really hope that this gets implemented, however, as it would really make Electron a much more promising platform for even more types of programs. :+1:

[notpushkin]

@octacian I'm starting to think that PWAs are a better option. Not universal, though – you can't do _everything_ using PWA – but certainly for things like custom calculators they should be more than enough.

Now we just have to wait for PWA support on desktop to emerge.

[yasinaydinnet]

I also think that with such a system, 1- it will be easier to ship to embedded/smaller systems, and 2- in the future electron-like operating system base can be created for mobile devices.

[jobukkit]

@yasinaydin There is already LG WebOS. (There used to be Firefox OS too, but that is discontinued)

[adalvigi]

@KeitIG My sir, your argument would be absolutely decent if the only issue with Electron were it's size. Having a slightly bigger program but simplifying complexity and possibility of failure for developers (we can't deny Electron is great in giving developers a standardized way of building apps without much room to screw up) would be awesome. Alas, using a separate version of the libraries for each app also means increased RAM usage, and distributed (and harder to fix) security issues. I had explained this better in my duplicate issue, but in a nutshell I really think a shared runtime library you can update on it's own and share between different Electron apps (which would be shipped in a "lite" version too, kind of like when some developers used to distribute Java files to use on your JVM and bundled .exes) would really be the deal breaker for Electron, and while I understand that it'd probably be a huge amount of work, it saddens me that it's not in the devs' plans yet. Let's hope they do it in the future.

[jacobq]

@dre-hh While I acknowledge the good intention of your proposal, it is, among other things, overly restrictive. For example, I have developed a commercial electron app for my employer, and it is used in highly regulated environments (no Internet connectivity, requirements for trace-ability / auditing, etc.). Our customers cannot allow us to do things like automatically update the software. It is a very different world than typical web apps for the masses. Evergreen browsers and automatic installation of OS security updates are, in my opinion, a great idea for most people in most cases, but my point is that they do not fit everywhere. Sometimes the risk of disruption caused by software change is far greater than the risk associated with running an out-of-date version of software, especially in controlled environments where other security mitigation techniques are in place. Hence taking an extreme measure such as blocking execution of applications using out-of-date versions of electron (or really, any library) is not an ideal solution, if a solution at all. (Besides, administrators already have many controls for restricting what programs can be executed, so they could implement this without any new OS features.)

Additionally, as with most security concerns, much of the problem stems from developers who are either unaware of the risks or have chosen, for whatever reason (e.g. lack of time), not to follow the security guidelines for the tools they are using. (e.g. Security, Native Capabilities, and Your Responsibility) We as developers all need to do our part in taking responsibility for the quality (which includes security) of our work.

Finally, I think it is a mistake to look to a commercial software entity, such as Microsoft, to address these concerns. This is partly because, as was just noted, we cannot expect to produce secure applications via developers who are not security conscious. More importantly, in my opinion at least, is that one of the beautiful things about Electron is its open-source and multi-platform nature, which are usually not high priorities for commercial software vendors. Well thought-out and successful progress will most likely require some level of consensus and effort from the whole developer community. Of course, contributions from commercial entities is, of course, appreciated.

[jobukkit]

@jacobq In case you didn't know, Microsoft is the owner of Electron.

[jacobq]

In case you didn't know, Microsoft is the owner of Electron.

@Jop-V Please add a citation for this; I don't think that is correct. The electronjs.com front page says:

Electron is an open source project maintained by GitHub and an active community of contributors.

Microsoft is a consumer of Electron (using it in products like Visual Studio Code) and (I presume) also a contributor, but they do not have ownership. The license file does not mention Microsoft nor does the Wikipedia article suggest that they have a position of ownership.

[styfle]

@jacobq Microsoft is going to acquire GitHub so if Electron is currently maintained by GitHub, it will soon be maintained by Microsoft as far as I can tell.

https://news.microsoft.com/2018/06/04/microsoft-to-acquire-github-for-7-5-billion/

https://blog.github.com/2018-06-04-github-microsoft/

[martpie]

Microsoft acquiring Github does not mean anything for Electron. Github is/will be still a separate company, with its own products and projects. I do not see Microsoft taking the ownership of Electron, as much as I do not see Microsoft taking the ownership of Atom.

I think this debate is not really helping the initial point of the issue though.

[gaui]

https://github.com/GoogleChromeLabs/carlo

[zeke]

Carlo looks pretty intriguing. I took it for a spin and it's very intuitive. Has anyone experimented with pkging it yet?

[danielo515]

Why all the proposals for reducing the bloat of end users hard drive and memory are never worked on? This is a fantastic idea which is 4 years old and is still just an idea. I can't understand why this kind of things are not on the top of the project priorities

[kapouer]

@danielo515 because instead of creating yet another industrious project, there is https://github.com/GoogleChromeLabs/carlo#q-can-a-node-app-using-carlo-be-packaged-as-a-desktop-app

[danielo515]

What are you talking about? This is already a enormous project. It is also the most widespread and used, so pointing people out to a different project for a feature that should be part of this one doesn't seem correct to me.
Also, Carlo it's a different approach, they use your installed Chrome, so it is not like a daemon version of Electron .

[martpie]

I can't understand why this kind of things are not on the top of the project priorities

Because making it a reality is hard (like "really" hard).

If you are not happy (which is totally understandable), feel free to fork Electron and submit a PR.

Don't get me wrong, if there was an easy solution to this problem, I am sure it would have been implemented already.

[jacobq]

@danielo515

... pointing people to a different project for a feature that should be part of this one doesn't seem correct to me.

I think the point is that each project has it's strengths and weaknesses, and Carlo is well suited for space-constrained cases. For my use cases, a 100MB app size is perfectly fine, so although I'd love to see this feature land, the other electron features are far more important to me.

[styfle]

@jacobq Yes, you're right—different use cases call for different tools.

If anyone else is interested, I created a list of similar tools for developing js apps on the desktop.

https://github.com/styfle/awesome-desktop-js

[kapouer]

I just don't use Electron because i can't afford updating dozens of 100MB apps - and you know how apps need to be updated (i have other opinions about it but sharing them won't help).
What i really like about the approach of carlo is the idea to use the installed browser.
Even better, surely carlo could use whatever browser is available, not only chromium, since puppeteer can manage other browsers too. On OSX you'd spawn Safari, on linux epiphany(webkitgtk) or firefox, etc...
So really to get an app you'd only have to install nodejs and your app.
Now the question is going to be: how good pkg can be at using system-installed libs, and native addons ?
For example if i had to package the app i'm currently developing, i'd need postgresql, sharp (and vips),, webkitgtk, and exiftool. I would like to be able to distribute the app only to linux debian, fedora, or ubuntu users. I would like to get away with the "crazy bundles" and instead capitalize on the hard work done by those distributions, for free.

[baconbrad]

@kapouer ... use the installed browser ...

While in theory a good idea it does result in increased difficulty in coding applications on the web side since you would now have to focus on support for multiple browsers and the browsers they fall back on to may be limited in what you wish to do as they don't have full support for things like you can find in Chromium. You also rely on the user to ensure their browser is up to date and that the multiple browser vendors don't break your application with an update.

Many users have got around having massive 50 Mb updates by switching to delta updates or having package managers only update the .asar file or resources folders. And only do a large Electron update when needed. This is what we do in our company and it has worked out great.

[anandhumanoj]

From a naive user's perspective, Using Electron app-shell as Shared Runtime would make it really easy to distribute and use. For Eg. With 5 electron apps in a single system, you have 5 instances of app-shell and it is around 300MB, right?

I strongly believe electron is the future of desktop UX and a this would be a big step ( look at .NET Runtime -- may be a bad comparison, but, it's cool right? ) to bring the web-tech more powerful.

[ashelleyPurdue]

Electron already has a runtime for development use, doesn't it? The "electron ." command, I mean. Why not reuse that for the user side?

To deal with multiple apps depending on different version of Electron, you could use the approach that .NET core uses. Simply have multiple versions of the runtime installed side-by-side, and have apps declare the minimum(or maximum) version they require.

[ProPuke]

I've made a proof-of-concept launcher: https://github.com/ProPuke/electron-shared

Single executable, under 1MB in size. Pass it an app directory or asar package and it will check your electron version requirements in your package.json devDependencies and download/run with the correct version.

Electron runtimes get stored in AppData\Local\Local\electron-shared (windows) .cache/electron-shared (linux) Library/Application Support/electron-shared (mac) so they can be shared between apps.

Without a path specified it will automatically run the app directory or app.asar if one exists; So you could distribute your app with just this file and app.asar. Or this could be associated with .asar files and just the asars could be distributed.

It also has a few commandline params, including --downloadOnly and --silent so it could be called from within installers during the setup process.

It doesn't handle automatic updating (it only downloads if a compatible version of Electron isn't already available on your machine), but this could be done periodically on launch, or by a background service (as long as the runtimes are placed in the right place it will find and use the latest when apps are launched).

Knowing when to delete old copies of the runtimes could be tricky. They might pile up over time. We'd either need to keep track of which installed apps used which versions (and thus which were no-longer necessary) or maybe just store a last-used date for each runtime version and auto-delete them if they haven't been used in a while.

Anyway, a proof of concept/proposal. Does something like this solve problems? Have I missed anything glaring and obvious?

[anandhumanoj]

I tried out @ProPuke 's electron-shared and I was able to run electron-quick-start demo. But I was not able to run any of the apps described in electronjs Website( 'cause most of the apps uses some shell script to start up the app and not just package.json, as I understand ). But I believe this _proof of concept/proposal_ could be a pointer to the next big upgrade.

[nwdxlgzs]

I have an idea. Because electron currently supports the command "electron.exe path". I think you can open it by calling multiple asars from the command line. At present, I'm just an idea. I hope someone can do it, because I've just touched on this software.
_____________中文翻译版本__________
我有一个想法。因为目前electron支持命令“electron.exe path”。我觉得可以通过使用命令行去调用多个asar实现打开。目前我只是一个想法，我希望有谁可以去做一下看看，因为我目前刚刚涉及这个软件。

[nwdxlgzs]

粗略的看了看全部对话（和其他的讨论，如Multiple "apps"），我总结一下——原生调用多个asar而不是靠多个二进制文件启动，这个已经有很久的历史了，但是未支持。但是electron的default_app就是一个利用命令行实现的（可能，我暂时不清楚）。我想通过electron调用cmd（好像可以）应该可以实现这个功能兼容吧。
Taking a cursory look at all the conversations (and other discussions, such as Multiple "apps"), I conclude that native calls to multiple Asar instead of multiple binaries start, which has a long history, but is not supported. But Electron's default_app is an implementation that takes advantage of the command line (and, perhaps, I don't know for a while). I want to call cmd through electron (as if I can) should be able to achieve this feature compatibility bar.

[sapioit]

In the worst case scenario, you would have to make a .bat file or the equivalent for non-windows OSes, and run it from Electron.

[sapioit]

Possibly useful links:

https://nodejs.org/api/child_process.html#child_process_child_process_spawn_command_args_options
https://www.freecodecamp.org/news/node-js-child-processes-everything-you-need-to-know-e69498fe970a/

9718

https://stackoverflow.com/questions/50491092/execute-a-cmd-line-with-electron-angular
https://github.com/martinjackson/electron-run-shell-example

[nwdxlgzs]

It seems that only when there is no app.asar or app folder is the command electron. exe path valid

[sapioit]

Oh, so you need to have an electron to run the exe path inside the electron used to set all that up. Still better than an electron installation for each and every app used. You would only need one electron installation for every version of electron your app needs. You would probably need to make a separate extension which is a renamed zip archive which contains both the app we're trying to run and a file with the recommended version (would run it right away), compatible versions (would ask if to run or install the recommended version, and also download the recommended Electron version in the background), or would just show a downloading progress bar, for the recommended version to be downloaded.

[sapioit]

Or I misunderstood something.

[nwdxlgzs]

This idea is probably the same: since bat is currently easy to say, take it for example, the content is electron.exe app1.asar, after execution, electron opens the content of app1.asar. That's probably the way it works, but when you write the next sentence, electron.exe app2.asar, in bat, you can only do it when the electron of app1.asar ends. And when bat is turned off, the electron is turned off. But I write electron.exe app3.asar|electron.exe-v or electron.exe app3.asar|start, and so on. When I close bat, electron can survive, but I can't continue to open other asars. I think this stand-alone program might be better than this one, because bat is still somewhat limited.

[sapioit]

Not if you use temporary folders.

You have the (archived or unarchived) versions in a electron_versions folder, from which you use either the recommended version or the newest compatible version you have downloaded by copying the version you're using to a temporary folder, and after you finish it, you delete the folder.

The path of the locally saved files would either be in the electron folder or next to the file you're using. If you're using it from an archive containing the things I mentioned within the last 24h, then you would have to move the saved data files back into the archive, or not move it at all before you delete that temporary electron instance folder.

[nwdxlgzs]

SSD？

[sapioit]

???

The reason for deleting the temporary electron folder is because the app could theoretically change the files of the electron installation they're runing on, and by design. I mean, the archive can contain the files and folders which need to be written or overwritten inside the temporary electron instalation.

[sapioit]

So after the installation is used, it can be treated as corrupted by the app it ran, so it cannot be recycled, and thus must be deleted. However, the version from which the by-then corrupted installation used as a base is still intact, in the electron_versions folder of the runtime-electron app we're trying to make.

Actually, if the archive with distinct extension, which I mentioned before, is read-only, the data it used could be moved to %appdata%, theoretically. So an app ran inside the runtime-mode-electron we're trying to make would actually be portable (self-contained), and thus copying that file would be enough to take the configuration of that app with you.

But we would also probably need to add an update function for that filetype, which would be another electron installation which would copy and overwrite the files from the new installation file to a copy of the old installation file, and return the new installation file with the data it had. The whole thing could be automatically done, if there is a download link in the files used by the runtime-mode-electron we're trying to make.

[sapioit]

Once that's ready, if you want to go the extra mile, you could even add a git client from which to update the apps, though that might require using github and bitbucket to host archives compiled files.

[nwdxlgzs]

you can use "curl,wget, aria2c" download file,and user can don't need install git，oh，you need use "gzip,7z"unpack zip。

[sentialx]

I'm making a tool electron-runtime which works similarly to electron-builder. It bundles only the app.asar file with a simple sh script (later this will be an UI to show the progress of Electron download). Currently it works only for macOS.

When running an app packaged with electron-runtime It basically downloads a corresponding major version of Electron, but the newest in case of minor and patch, since the breaking changes of Electron occur only in major version increments.

The electron-quick-start app is only 62 KB, screenshot:

If you're interested in it, leave a star, PR or an issue.

[sentialx]

I've updated the package a little bit, that it can build using electron-builder (Here's described how it works). So now you can build even Visual Studio Code with it! (but still only on macOS).

[sentialx]

I successfully released a first version of electron-global! (I changed it, since electron-runtime has been taken). You can build your app using my npm package with electron-builder. I've also added an UI with a progress bar to show the progress of an Electron download. I hope it gets more popular to make Electron better.

[nwdxlgzs]

I'm sorry, I use the Windows system, and I don't have the money to use the MacOS.But I believe your project will be very successful.

[sentialx]

@nwdxlgzs It supports Windows, macOS and Linux...

[nwdxlgzs]

I thought this project only supported MAC.
I'm going to take time to try this great project.

[sapioit]

Thanks for making it and for keeping us up to date!
I'm also using Windows.
I will take a look at the code when I get the change.

[eMPee584]

Well, how about integrating electron-global upstream and making it the new default 😏