Etcher: Select drive as source

[alexandrosm]

This is a three-purpose feature request.

Overtly, it's about having the ability to select a drive as a source. So, imagine having two sd drives connected, and selecting the one as a source, and the other as a destination. I think this is a perfectly legitimate Etcher feature. Interestingly, a system drive would be a legitimate thing to select as a source in this case.

Covertly, this is about allowing the selection of things "other than files from the filesystem" as sources. In the future we could add urls, torrents, and other things as sources. If we work out the interaction for having files and drives, we can extend it to other things as they become available.

Even more covertly, this is one of the features we'd need for a hardware version of Etcher. ;)

[jviotti]

I like this idea. In order to achieve it, we should first sort out how to determine how much of the drive to copy, which is exactly what we need for the backup feature as well (https://github.com/resin-io/etcher/issues/266).

I'll mark this issue as "pending design", so we can start figuring out how to properly present this paradigm shift to the user while we sort out the technical side in the backup ticket,

[jviotti]

/cc @konmouz

[WasabiFan]

Theoretically, if you always represent these data sources/destinations as abstract streams, would you just be able to drop-in a disk stream in place of a file one (or a Node.js buffer or whatever internal structure you use)? My thinking is essentially that you could have both the source and destination be completely generic internally and then just specialize the UI to facilitate selection. With something like this, it would presumably be easy for someone to implement a new source or destination, because you wouldn't need to adapt internal switching logic.

[lurch]

@WasabiFan Yeah, and that's kinda covered by the etcher-image-write and etcher-image-stream modules as mentioned in https://github.com/resin-io/etcher/blob/master/docs/ARCHITECTURE.md

I agree with @jviotti that this PR sounds very much like a logical follow-on from #266

[alexandrosm]

To me it sounds like a precursor. If you had a drive as a source and a file as a destination, voilà, backup

[konmouz]

We can change the UI from 'Image' and 'Drives' to 'Source' and 'Destination'. Probably when you select a drive as source, it should look unavailable on the destination side. Regarding backup, it is true that it will be covered via this feature, however, it might worth having a 'backup' CTA as a more direct approach (mainly for new users).

[alexandrosm]

agreed that there may be a need for a backup side-flow even if one could
put it together with these primitives.

having abstract source and destination/target selectors was one of the
earliest things we were talking about with @jviotti and I kinda let it drop
so we could focus on the core, interesting to see it coming back, means it
wasn't a terrible idea, probably.

--

Alexandros Marinos

Founder & CEO, Resin.io

[phone]

@alexandrosm

On Wed, Dec 7, 2016 at 9:03 AM, Konstantinos Mouzakis <
[email protected]> wrote:

We can change the UI from 'Image' and 'Drives' to 'Source' and
'Destination'. Probably when you select a drive as source, it should look
unavailable on the destination side. Regarding backup, it is true that it
will be covered via this feature, however, it might worth having a 'backup'
CTA as a more direct approach (mainly for new users).

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/950#issuecomment-265393015,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCLRf85xmvrsQ90PHSe-0spCZ4_o6ks5rFnZOgaJpZM4LFwk3
.

[lurch]

I guess when selecting the 'Destination' as a file though, you'd only want to be able to select a single file, rather than selecting multiple files as in the multi-write mode. And it probably doesn't make much sense to read from a file and write to another file (unless you wanted to specifically enable that as a verification / testing mode).
So I guess the allowed combinations would look something like:

• read-from-file -> write to one or more drives (typical disk-flashing mode)
• read-from-drive -> write to one or more drives (disk-cloning mode)
• read-from-drive -> write to a single file (backup mode)

Probably when you select a drive as source, it should look unavailable on the destination side.

Agreed. And when selecting a file as destination, you'd need to make sure it's not on the drive you're going to be reading from (which we can do, now that drivelist reports which mount-points each drive uses).

When reading-from-drive we'll need to unmount the drive first (so that we can access it in raw mode), so it might be a nice feature to re-mount the drive again at the same mount-point(s) after we've finished reading from it? (if it's possible to do that reliably)
Hmm, and perhaps that means we need another category of drives for drivelist to detect - drives that we "know" we can't unmount (which will include at a minimum the "boot drives"), and therefore will never be able to read from (in raw mode) ?

[jviotti]

I think we should really consider this paradigm change now, since it means the whole Etcher workflow would change, and better to at least consider how this would look like before adding more features, otherwise it will be a pain to introduce later on.

[konmouz]

read-from-file -> write to one or more drives (typical disk-flashing mode)
read-from-drive -> write to one or more drives (disk-cloning mode)
read-from-drive -> write to a single file (backup mode)

Just to add 'read-from-online source -> write to one or more drives' which I guess UI-wise would be slightly different from 'read-from-file -> write to one or more drives'.
Also, if it makes sense 'read-from-online source -> write to a single file'. Well, this sounds like download, so let me know if there is such a use case.

[lurch]

Not sure if 'read-from-online source -> write to a single file' "works" as a concept, but when doing 'read-from-online source -> write to one or more drives' it might be nice to have an option to cache (or save) the image file (and its metadata?) locally, so that if you want to flash an online-image multiple times, you can just use the saved-version instead of streaming it again.

[jviotti]

Also, if it makes sense 'read-from-online source -> write to a single file'. Well, this sounds like download, so let me know if there is such a use case.

I think we can leave that out for now.

[alexandrosm]

I am not sure why we are enumerating combinations if the two sides will be
independent?

--

Alexandros Marinos

Founder & CEO, Resin.io

[phone]

@alexandrosm

On Wed, Dec 7, 2016 at 2:33 PM, Juan Cruz Viotti [e-mail]
wrote:

Also, if it makes sense 'read-from-online source -> write to a single
file'. Well, this sounds like download, so let me know if there is such a
use case.

I think we can leave that out for now.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/950#issuecomment-265462019,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCBSAyHAN_yb7rqdFHyCHik5O3SABks5rFsPDgaJpZM4LFwk3
.

[konmouz]

I guess for purposes like if you select an A drive as source, probably A will be unavailable on the destination side. Am I wrong?

[alexandrosm]

sure, just want to make sure we are thinking about the two things as
independent and not tying ourselves to specific patterns.

maybe we should list the special cases instead (like the one you mentioned
above)?

--

Alexandros Marinos

Founder & CEO, Resin.io

[phone]

@alexandrosm

On Thu, Dec 8, 2016 at 10:33 AM, Konstantinos Mouzakis <
[email protected]> wrote:

I guess for purposes like if you select an A drive as source, probably A
will be unavailable on the destination side. Am I wrong?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/950#issuecomment-265707996,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCFLANjghbZV4hnMAcr4ws99MJmtdks5rF90DgaJpZM4LFwk3
.

[konmouz]

For v.01, just to double check:

• Can a destination be online?
• Can you edit the files of a connected device or drive? Which means that source and destination is the same. Probably this is configuration.

[lurch]

Sorry if I was causing confusion. Yeah the two sides are "independent", but there are some combinations that don't make sense, so I was just trying to whitelist the valid combinations, rather than blacklist the invalid combinations.

In the current version of Etcher, the only "source" is "local file" and the only "destination" is "local drive".

I guess our currently-planned "sources" are: "local file", "online stream", "local drive"; and our "destinations" are: "local file", "local drive(s)". (But obviously there's nothing precluding other options being added to either of these lists).

[alexandrosm]

I would replace "online stream" with simply "url" but otherwise yeah, it
all checks out. In cases like this I generally prefer blacklists than the
other way around, as they are true-er to the generative nature of
independent primitives.

--

Alexandros Marinos

Founder & CEO, Resin.io

[phone]

@alexandrosm

On Thu, Dec 8, 2016 at 11:21 AM, Andrew Scheller [e-mail]
wrote:

Sorry if I was causing confusion. Yeah the two sides are "independent",
but there are some combinations that don't make sense, so I was just trying
to whitelist the valid combinations, rather than blacklist the invalid
combinations.

In the current version of Etcher, the only "source" is "local file" and
the only "destination" is "local drive".
I guess our currently-planned "sources" are: "local file", "online
stream", "local drive"; and our "destinations" are: "local file", "local
drive(s)". (But obviously there's nothing precluding other options being
added to either of these lists).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/950#issuecomment-265717510,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCPR2dawQYysFs3oQpuPrfYMSHLtGks5rF-hQgaJpZM4LFwk3
.

[konmouz]

regrading "online stream", except of a url, would it be possible to have a dropdown/ filter search UI element to select an image from an online source? At least for popular images would be great. (e.g. type 'Ras' and get a list of different Raspbian Image Versions)

[lurch]

Fair enough. In that case the blacklist is:

• local file -> local file
• url -> local file

(and with the additional filtering that @konmouz mentioned of not being able to select the same local-drive as both source and destination; and if doing local-file -> local-drive or local-drive -> local-file we should ensure that the local-file isn't on a filesystem that's on the drive being written to or read from)

[lurch]

would it be possible to have a dropdown/ filter search UI element to select an image from an online source?

Yeah, something like that _is_ planned in the Etcher roadmap :) @alexandrosm might be able to find up the doc and send it to you on flowdock.

[jviotti]

Done in the latest SDK