By one-off, I mean changing some small thing about the image, like the hostname or a static ip address. One use case is for classrooms that may be using embedded devices and want to set them up without having to boot and log into each device (comes from here).
I'm guessing to make this work, the image file would have to have a FAT partition since that is pretty much the only partition type that works on all OSes without additional software. A simple solution might be to have the image file could have extended meta data mount this partition and open a configuration file in the OS default text editor when the image burn is finished instead of ejecting the drive. Or even better, a simple text editor built into Etcher so that Etcher can eject the drive when editing is finished.
For example, in our ev3dev OS for Raspberry Pi, users always need to edit config.txt to select which add-on board they are using since there are multiple add-on boards and they cannot be automatically detected.
dlech
All 78 comments
Hi @dlech ,
Coincidentally, this is a feature we've been internally discussing (in Gitter, so not really internally after all) as part of our "Extended Archives" feature (see the WIP manifest here: https://github.com/resin-io/etcher/pull/707).
We've implemented configuration support in our Resin CLI (https://github.com/resin-io/resin-cli) already, but we're looking to extract this functionality and make it generic in Etcher.
This is what we currently have
resin-image-fs: A very handy and reliable module to perform file operations in image's FAT partitions (like move, copy, replace, etc). It currently only supports FAT, but nothing prevents us from adding support for other file systems as long as its feasible to do so in a cross-platform fashion. We should also remove the "Resin" piece of the module, as it suggests the module is Resin.io-only.- A way to define "questions" to ask the user, for example:
options: [
message: 'Processor'
name: 'processorType'
type: 'list'
choices: [ 'Z7010', 'Z7020' ]
,
message: 'Coprocessor cores'
name: 'coprocessorCore'
type: 'list'
choices: [ '16', '64' ]
]
- The answers from the options presented to the user are passed to
resin-device-operations, a module used by the Resin CLI which accepts a set of configuration "commands" and executes them on an image. Check the following commands as an example:
[
command: 'copy'
from:
partition:
primary: 1
path: '/bitstreams/parallella_e16_headless_gpiose_7010.bit.bin'
to:
partition:
primary: 1
path: '/parallella.bit.bin'
when:
coprocessorCore: '16'
processorType: 'Z7010'
,
command: 'copy'
from:
partition:
primary: 1
path: '/bistreams/parallella_e16_headless_gpiose_7020.bit.bin'
to:
partition:
primary: 1
path: '/parallella.bit.bin'
when:
coprocessorCore: '16'
processorType: 'Z7020'
]
The actual commands make use of resin-image-fs.
This is what we're missing
- Polishing the way options and commands are declared. We want to ditch the concept of "commands" and use a more "high-level" way of declaring what a file goes somewhere, etc; since the current approach is very imperative.
- Add support for common "schema-like" languages, like JSON, YAML, INI, etc; and allow users to define that a certain value goes in a certain "key" of a certain file. We manage the rest.
- Maybe templating support?
In any case, this is all taking shape and we don't have anything clear at the moment, but if you're interested, we're discussing a lot of this stuff in Gitter, so I suggest you to keep an eye there.
Given that you opened this issue, lets make this the home of more organized discussion about this topic.
jviotti
on 26 Sep 2016
+1 Another usecase for such a feature would be to setup wifi. On many devices entering the wifi settings is hard. I don't want to complicate the feature request, but I wonder if it should be considered to be able to re-run the customization steps on a preflashed image?
ensonic
on 27 Sep 2016
@ensonic you'll be glad to hear that @jviotti is going to be spending the next two weeks focusing on this precise problem. Wifi config is certainly a big use case.
alexandrosm
on 27 Sep 2016
I dunno how suitable it is for Etcher, but you might want to have a look at http://www.pibakery.org/ ( https://github.com/davidferguson/pibakery - also based around NodeJS and Electron I believe) which was created to solve issues very similar to those being discussed in this thread :-)
https://www.raspberrypi.org/blog/pibakery/ ping @davidferguson
lurch
on 30 Sep 2016
Hi @lurch ,
We've indeed looked at PiBakery, which is in fact based on Etcher (it uses Etcher's backend modules, and our elevation system, among other stuff).
While PiBakery is very Raspberry Pi specific, we're creating a completely generic system that not only allows users to configure, but to re-configure as well (read config settings back from the device).
We are currently working on this system, which is the reason why I'm not very active anymore (I'll get back to normal speed once the foundations are done), and hopefully everything will make more sense afterwards (we'll publish specs, etc).
jviotti
on 30 Sep 2016
We've indeed looked at PiBakery, which is in fact based on Etcher (it uses Etcher's backend modules, and our elevation system, among other stuff).
Oooh, that's interesting! In that case I wonder why @davidferguson uses his own CommandLineDiskImager program (forked from Win32DiskImager), rather than using the image-writing code from Etcher?
we're creating a completely generic system that not only allows users to configure, but to re-configure as well (read config settings back from the device).
Sounds cool. If you've not already done so, you might want to have a look at Webmin and LuCI for ideas / inspiration / sample-code :-)
lurch
on 30 Sep 2016
In that case I wonder why @davidferguson uses his own CommandLineDiskImager program (forked from Win32DiskImager), rather than using the image-writing code from Etcher?
I use CommandLineDiskImager for Windows because I was getting some really weird errors when I tried to use resin-image-write on Windows (can't remember the exact error now, but it appeared in resin-image-write, etcher-cli and also the main Etcher app, and on multiple Windows computers) so I decided to use Etcher on Mac (and Linux) and a version of Win32DiskImager on Windows.
I'll try resin-image-write again on Windows again at some point, but the error was tricky to reproduce and seemed to appear at random times, so for the moment I'll probably stick with CommandLineDiskImager.
davidferguson
on 30 Sep 2016
@davidferguson you're probably referring to the issue narrated in this post -- https://resin.io/blog/the-perils-of-writing-disk-images-on-windows/ , which have now been resolved in the latest version of etcher-image-write. The resin-image-write module is deprecated and we generally recommend people use etcher-image-write instead, which should be a drop-in replacement.
alexandrosm
on 30 Sep 2016
@alexandrosm That's the error - thanks for linking that! Great news that the issue is fixed - I'll work on using etcher-image-write in the next release of PiBakery.
davidferguson
on 30 Sep 2016
In the spirit of opening up the development of Etcher further, I'll paste here a high-level document of the discussions we've had with @jviotti on the configuration pipeline. It's super abstract and ambitious, but we think we've actually cracked it. Juanchi has been on this for the last week or so, and by the end of this one he should have a version zero for another resin.io project, which should be easy to transfer to Etcher shortly thereafter.
Device (re)configuration for Etcher
Introduction
Users that write OS images with Etcher, often want to configure these images with Etcher as well, before they boot the devices that these images are intended for. In addition, they want to be able to reconfigure these images later, either because they may have made a mistake, or because the settings are no longer correct and the device does not offer a way to change its configuration, very common with headless IoT devices.
Addressing this in a generic fashion is an extremely challenging problem, as the configuration information is often stored on the device in an unpredictable way, split between several files, each of which is of different format. To make things worse, these configuration files often do not express the user’s intent directly, but as a composite of various settings, which in aggregate express the user’s initial high-level input. Solving the configuration problem requires Etcher to have the ability to take user-level configuration and convert it into machine-level configuration, which is complex but possible. However solving the re-configuration problem optimally requires being able to reverse one’s steps, and convert machine-level configuration into user-level configuration, which is considerably harder.
If one is able to dictate the format of the configuration files on the device, things are fairly simple, as we can push the complexity to the device runtime. That’s how we do it on resin-cli for resin-os devices. However solving the problem in a general fashion requires being able to express as many different configuration approaches as possible.
This document describes an approach that excludes a fairly small amount of potential approaches to configuration, while at the same time being very economical with the metadata required to encode how to mediate between the user and the machine.
The four stages
The cornerstone of the approach is the definition of four distinct stages that the information goes through.
The first stage is the “user interface” stage, which is the stage in which the user interacts with the information. This may take the form of a web form, or a command line wizard, but the key is that it optimises for communicating with the end user and empowering them to express their intent, while avoiding errors, whether those are by misunderstanding or accident.
The second stage is the “dry json” stage. In this stage, the information is expressed as a JSON structure that is optimised for avoidance of repetition and internal consistency. This stage very closely resembles the visual stage, but is stripped of user interface considerations and reduced to a format that is easy for machines to process and transmit.
The next stage is the “wet json” stage. In this stage, the data is still encoded as a JSON structure, but instead of optimising for clarity and consistency, it optimises for matching the structure with which it is going to be stored on the device.
The final stage is the “file” stage, during which the information is stored on the device itself, potentially in different files, and potentially with those files having different serialisation formats between them, and even several formats co-existing nested within the same file. This stage is optimised for consumption by a running operating system, and while users are sometimes told to edit these files directly, the process is often quite technical and error prone.
In order for our approach to be fully described, we must examine each pair of stages, and how information is converted from the one to the other.
Transformation pipeline
In order to define a fully two-way transformation, we need to define two-way transitions between each pair of stages. Our pipeline should look something like this:
user interface <--(transition)--> dry json <--(transition)--> wet json <--(transition)--> files
Stage transitions
From user interface to dry json
The first transition is pretty straightforward, if only because this problem has been solved well by various existing systems. One system that is a good example, regardless of whether we decide to use it ultimately, is http://schemaform.io/. It can generate HTML forms from JSON Schema files. Those forms can be used to extract information from a user, which they convert to JSON. In reverse, given the data in JSON form, they can use it to populate a form, therefore enabling full two-way transitions. We may not use this transition as-is as we want more datatypes than just what is available in json schema, but the main approach will not diverge much.
From dry to wet json
This transition is the hardest one, and the one that is the least obvious. A single-field entry in the dry json form can become several fields in the wet json form. Further, those several fields can, in fact, operate as templates on which other fields from the dry json structure must be mapped.
While it’s not hard to think how to go from dry to wet json with the use of a templating system such as handlebars, it’s much harder to think how to reverse the process without having explicit code with which to do the transformation. The extremely interesting finding here is a library called jsonexp, which can take patterns as inputs, and not only match them on an arbitrary JSON file but in fact, extract undefined subsegments of those patterns and return them to the caller. The exciting piece of information is that the “patterns” that jsonexp would need to convert wet json to dry json are the same that a templating engine such as handlebars would need to convert dry json to wet json, if one ignores small syntactic differences which are easily overcome. As such, these patterns can be used to concisely express the relationship between dry and wet json and be used by different tools to transition data in either direction.
Upon further examination, we’ve decided not to use jsonexp directly, as we would want to stay within the confines of json tooling, but we will be following a fairly similar approach.
From wet json to files
The final transition is that between wet json and configuration files. Since wet json stores information in the same way as the files themselves, all that is needed is to express which files are to act as stores of data, and how the data is to be serialised (selected from a set of understood configuration file formats). The information intended for this stage should be expressed in a way that allows for the transition to be done in the reverse direction as well.
Additional notes
A note on manual file alterations
For this entire system to work, we must account for the case where users alter the configuration files on a device, and the resulting files are outside the range of the patterns. In that case, the system must partially break the abstraction and fall back to allowing the user to manually edit the unmatchable segment. Ideally, the user will be able to restore the segment to a default value, and also unaffected segments of the configuration file (or other configuration files) will still surface in structured inputs.
As a corollary functionality, we must consider the case when the user wants to manually edit the target files, perhaps to create a configuration which is not available by manipulating the high-level configuration. This too must be made possible by including a “custom value” alternative on the appropriate input.
A note on reconfiguration
For Etcher to be able to reconfigure a device, it must have access to the transformation description files. As such, those should be written to a partition of the flash drive as defined by the extended archive, so that they can be re-discovered by Etcher when it discovers the drive in the future.
A note on raw files
There exist situations in which we want to transfer a raw file from the UI all the way to the disk. For instance, if a picture of a logo, or some binary executable such as a kernel module or .dtb is needed. These assets can either be kept as they are from the visual to the file stage, or converted to ASCII data which is encoded in the transitional dry and wet json structures.
A note on “filesets”
When a variable number of files have to be created, which correspond to the instances of a multi-member entity (think network connections a device should try), the concept of mapping to a single file may break down. It is possible however to think of a single “fileset” entity, each member of which is serialised in a given way (e.g. yaml). We can then maintain the mechanisms that write to a given entity, which in this case is a fileset, but this is reflected on disk as a collection of files, achieving our goal. Given the right selection rule (e.g. all the files in a directory, or all filenames matching a certain pattern), the process can be reversed and the fileset generated based on the raw files.
alexandrosm
on 3 Oct 2016
After a conversation with @alexandrosm regarding how (and if) image configuration can fit to v01, I have some exploration. The idea is to fit image configuration within 'Image Info modal'. This window can appear by user's request (when clicking the image name or the 'i' icon on the main UI window), or automatically when a new image is selected (only when necessary).
Pinging @jviotti & @taahirisaacs, to initiate this discussion.
*Note: The modal is scrollable and current content is a rough estimation (would change according to the selected image).
Current 'Image Information Modal':
Potential Update:
konmouz
on 12 Dec 2016
I like where this is going. I think we should definitely show this by default, and even have some sane way UX-wise to actually enforce it, since there might be images that make no sense to leave un-configured.
Re-using the image details for this is a clever idea, however I wonder how obvious it is to users that clicking an informative bubble on the first step will trigger a configuration dialog. Maybe we can have another bubble with say a "cog" icon to open a different configuration modal?
Another idea: could we rename the "Flash!" button label to something like "Configure & Flash!" if we detect the image can be configured? In that case, "Looks Good" in the configuration modal would become "Flash".
I'd also help to define the different type of controls that could take place in the modal. By seeing the current sketch, I have several questions from a technical point of view:
- The "Persistent Logs" feature contains a danger icon. How do we present that in the JSON schema?
- Several things are presented in a grid during the "Ethernet" configuration phase. How do we determine if we should show in a grid, one below the other, etc? Do we use certain heuristics?
jviotti
on 12 Dec 2016
The possible controls here are (let me know if I'm missing any):
- Number input
- Text input
- Password input
- Text enumeration
- Number enumeration
- Boolean input (a checkbox)
jviotti
on 12 Dec 2016
Re-using the image details for this is a clever idea, however I wonder how obvious it is to users that clicking an informative bubble on the first step will trigger a configuration dialog. Maybe we can have another bubble with say a "cog" icon to open a different configuration modal?
This is my main worry. Putting all of this configuration in a "details" pane just feels wrong... it would be much more natural to have an obvious way of accessing this configuration. My initial thinking is along the lines of a cog icon as you said, but that feels too subtle. Maybe it should pop up automatically when an image is selected.
It would also be nice if there were a way to see a quick summary of the chosen options; maybe everything that is non-default would be displayed in the image details along with the path.
WasabiFan
on 12 Dec 2016
I also think it should pop up automatically for suitable images, and i
think @konmouz kinda said that too, but not too clearly.
I agree with others that overloading the info is probably not awesome, and
the resulting modal looks packed. A separate modal is probably the way to
go.
--
Alexandros Marinos
Founder & CEO, Resin.io
+1 206-637-5498
@alexandrosm
On Mon, Dec 12, 2016 at 8:40 PM, Wasabi Fan notifications@github.com
wrote:
Re-using the image details for this is a clever idea, however I wonder how
obvious it is to users that clicking an informative bubble on the first
step will trigger a configuration dialog. Maybe we can have another bubble
with say a "cog" icon to open a different configuration modal?This is my main worry. Putting all of this configuration in a "details"
pane just feels wrong... it would be much more natural to have an obvious
way of accessing this configuration. My initial thinking is along the lines
of a cog icon as you said, but that feels too subtle. Maybe it should just
pop up automatically when an image is selected.It would also be nice if there were a way to see a quick summary of the
chosen options; maybe everything that is non-default would be displayed in
the image details along with the path.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/718#issuecomment-266546139,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCNqRbMARSikMjOvu_LO-6rdrc8jCks5rHbEmgaJpZM4KFQHD
.
alexandrosm
on 12 Dec 2016
I am trying to reduce the amount of total modals and group information when possible. I'll explore it a bit more and I'll come back with a new UI.
konmouz
on 12 Dec 2016
I think we should definitely show this by default
How on earth would that work? IMHO this should only be available if the image-metadata explicitly asks for it, because it doesn't make sense to try and "guess" what settings an arbitrary image might need, what format they should be stored in, which files / partition they need to be saved on, etc.
Going back to my NOOBS experience: In NOOBS there's a mini "configuration step" while the OS is getting installed, where the OS gets an opportunity to configure itself based on what partitions it's _actually_ getting installed onto. Since NOOBS can't know how different OSes need configuring, it looks for a partition_setup.sh script (see e.g. this one ), and if found it simply passes it a list of partitions as command-line arguments. It's then up to the script to mount whichever partitions it needs to write to, do whatever configuration is necessary, and unmount the partitions again.
When I saw @konmouz 's design pictures I assumed the image-metadata would simply specify whether it wants the "ethernet config" and "wifi config" configuration tabs visible (and we could define other standard configuration tabs). But from what @jviotti is saying with his discussion of widgets and layouts, it sounds like he's imagining each extended archive will be able to specify completely customisable configuration UIs? (which is obviously another whole level of complexity)
If we _are_ allowing custom-interfaces, then perhaps things like an "IP address input" control would also be useful? (or maybe just having a validation-regex option on the "Text input" (which the publisher could set to ^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$, or even the regex from http://www.regextester.com/22 ), would be sufficient?)
Are the tickboxes next to the textboxes in @konmouz 's pictures so that we can tell the difference between NULL (i.e. don't store this setting) and "" (i.e. store this setting, but set it to an empty string) ?
I assume we'll be using reconfix-configurations stored in the image-metadata to convert the configuration captured by Etcher into the OS-specific config files? And then using LKL (as discussed in Athens) to actually write those config files to the SD card? Or for this first version are we just going to be writing a single JSON config file to a nominated FAT partition on the card, and leave it up to the OS to make use of that config file however it chooses?
I guess (as an advanced option) we could even have a button to save the configuration as a 'default config' back into the image metadata itself (but just in the metadata, leaving the raw-image untouched), so that if the user is repeatedly writing images with the same config values (e.g. wifi login details) to multiple cards, they don't have to keep re-entering them each time they start Etcher.
Just having re-read through @alexandrosm 's "Device (re)configuration for Etcher" comment, one part left out of that which has just occurred to me is that if we're writing configuration files directly, we probably also need to be careful about the ownerships and permissions that new files get created with. (should reconfix just be given numeric uids and gids, or should reconfix be given the "more friendly" usernames and groupnames and attempt to read /etc/passwd and /etc/group/ from the image to lookup the appropriate uids and gids?)
Something else I've just thought of - obviously to ensure that we can verify the image got flashed correctly (by comparing checksums), we'll only want to "save" the config onto the SD card _after_ we've flashed and verified it. But as https://resin.io/blog/the-perils-of-writing-disk-images-on-windows/ explains, on Windows we actually write the MBR-chunk of the image to the SD card last, because apparently Windows doesn't allow raw-access to 'partitioned' space. So in order to be able to write the config to the card, does that mean we'd actually have to read back the MBR-chunk from the card, erase the MBR-chunk, write the settings, and then write back the MBR chunk? :-S (and I wonder if that _might_ lead to a catch-22 situation, with LKL needing to be able to read the MBR?)
lurch
on 13 Dec 2016
How on earth would that work? IMHO this should only be available if the image-metadata explicitly asks for it, because it doesn't make sense to try and "guess" what settings an arbitrary image might need, what format they should be stored in, which files / partition they need to be saved on, etc.
Of course. This would only apply if the image has Etcher-readable metadata and specifies things to configure.
When I saw @konmouz 's design pictures I assumed the image-metadata would simply specify whether it wants the "ethernet config" and "wifi config" configuration tabs visible (and we could define other standard configuration tabs). But from what @jviotti is saying with his discussion of widgets and layouts, it sounds like he's imagining each extended archive will be able to specify completely customisable configuration UIs? (which is obviously another whole level of complexity)
I think this feature is pretty much useless if you can't configure the controls that are shown. That will definitely be difficult and design-intensive, as with the rest of this idea.
WasabiFan
on 13 Dec 2016
But from what @jviotti is saying with his discussion of widgets and layouts, it sounds like he's imagining each extended archive will be able to specify completely customisable configuration UIs? (which is obviously another whole level of complexity)
Exactly, what is shown is completely generated from the Reconfix schemas, and its indeed a big design challenge.
If we are allowing custom-interfaces, then perhaps things like an "IP address input" control would also be useful? (or maybe just having a validation-regex option on the "Text input" (which the publisher could set to ^\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}$, or even the regex from http://www.regextester.com/22 ), would be sufficient?)
That is a great idea: to have a collection of more complex domain-specific inputs like IP addresses. I'll create an issue in Reconfix to track it.
I assume we'll be using reconfix-configurations stored in the image-metadata to convert the configuration captured by Etcher into the OS-specific config files? And then using LKL (as discussed in Athens) to actually write those config files to the SD card? Or for this first version are we just going to be writing a single JSON config file to a nominated FAT partition on the card, and leave it up to the OS to make use of that config file however it chooses?
Reconfix will do the whole thing. LKL is only needed when writing to partitions other than FAT32, but I think we're fine with that, and therefore we'll delay the introduction of LKL here.
Just having re-read through @alexandrosm 's "Device (re)configuration for Etcher" comment, one part left out of that which has just occurred to me is that if we're writing configuration files directly, we probably also need to be careful about the ownerships and permissions that new files get created with.
This is a very good point. I'll create an issue on Reconfix for it.
Something else I've just thought of - obviously to ensure that we can verify the image got flashed correctly (by comparing checksums), we'll only want to "save" the config onto the SD card after we've flashed and verified it.
Exactly, configuration will happen after the flash/validation process.
But as https://resin.io/blog/the-perils-of-writing-disk-images-on-windows/ explains, on Windows we actually write the MBR-chunk of the image to the SD card last, because apparently Windows doesn't allow raw-access to 'partitioned' space. So in order to be able to write the config to the card, does that mean we'd actually have to read back the MBR-chunk from the card, erase the MBR-chunk, write the settings, and then write back the MBR chunk? :-S
The problem described in the blog post only applies when freely writing bites using raw I/O on the drive. Reconfix, on the other side, will mount the filesystems and interact with them instead.
jviotti
on 13 Dec 2016
need to be careful about the ownerships and permissions that new files get created with
But that's obviously not an issue if we're just writing to a FAT partition ;)
Reconfix, on the other side, will mount the filesystems and interact with them instead.
Ah, you mentioned https://github.com/resin-io/resin-image-fs in an earlier comment, so I'd assumed you were going to be using that, rather than mounting the FAT partition in the OS directly.
Several things are presented in a grid during the "Ethernet" configuration phase. How do we determine if we should show in a grid, one below the other, etc?
For the design images that @konmouz has posted, perhaps we could group controls together, into "meta-controls"? So there's a Priority meta-control (textbox and tickbox), a Static IP meta-control (textbox and tickbox), a Hostname meta-control (just a textbox) and a Persistent Logs meta-control (label and tickbox).
And then the Ethernet tab is made up of a default section (no header) and an advanced section (with an Advanced Settings header).
The default section is composed of the Priority and Static IP meta-controls, and a single label control (saying that Ethernet is the default). The advanced section is composed of the Hostname and Persistent Logs meta-controls. And then maybe the meta-controls in this example all specify that they use a half-width, and the label control specifies that it needs a full width (or maybe these sizes get calculated automatically), and the controls get "flowed" into each section, so that in the default section the Priority and Static IP meta-controls get flowed next to each other, and the label control gets flowed underneath.
So, in a crude (incomplete) JSON form, perhaps something like:
{
"config": {
"label": "Network Setup Configuration",
"tabs": [
{
"label": "Ethernet",
"sections": [
{
"controls": [
{
"label": "Priority",
"type": "meta-control",
"width": 0.5,
"controls": [
{
"type": "textbox",
"default": "0"
},
{
"type": "tickbox",
"default": false
}
]
},
{
"label": "Static IP",
"type": "meta-control",
"width": 0.5,
"controls": [
{
"type": "textbox",
"default": "192.168.1.111"
},
{
"type": "tickbox",
"default": true
}
]
},
{
"type": "label",
"width": 1.0,
"label-text": "*By default all devices are configured to use Ethernet"
}
]
},
{
"label": "Advanced Settings",
"controls": [
{
"label": "Hostname",
"type": "meta-control",
"width": 0.5,
"controls": [
{
"type": "textbox",
"default": "resin"
}
]
},
{
"label": "Persistent Logs",
"type": "meta-control",
"width": 0.5,
"controls": [
{
"type": "label",
"label-icon": "warning",
"label-text": "This can wear out Flash Media"
},
{
"type": "tickbox",
"default": false
}
]
}
]
}
]
},
{
"label": "WiFi"
},
{
"label": "Cellular"
}
]
}
}
But that's obviously not an issue if we're just writing to a FAT partition ;)
Ah, good catch!
Ah, you mentioned https://github.com/resin-io/resin-image-fs in an earlier comment, so I'd assumed you were going to be using that, rather than mounting the FAT partition in the OS directly.
Yeah, we have resin-image-fs for things like FAT, but we'll probably be switching to performing an actual mount as recommended by the LKL talk we had in Athens.
For the design images that @konmouz has posted, perhaps we could group controls together, into "meta-controls"?
Very nice feedback. What you're describing is basically the high-level "visuals" stage of Reconfix, for which we don't have much yet. Maybe we can create a couple of real-world examples based on your ideas to test the design against?
jviotti
on 13 Dec 2016
@jviotti @lurch, for my screens I used a configuration sketch from @shaunmulligan. The more examples (print screens, text, anything) you can provide me with, the better. Then I can try to figure out a way to group objects into UI elements and fields. Also I need to understand how long and complicated this configuration can be, which can seriously affect the UI. @lurch's example/suggestion is already very helpful!
konmouz
on 14 Dec 2016
I need to understand how long and complicated this configuration can be
If we're letting the publishers define their own customisable configuration screens inside the image-metadata, then I guess there's no limit to how long and complicated they can be! Maybe having the tabs (Ethernet, WiFi, Cellular, etc.) across the top scrollable left<->right (of which there can be an "unlimited" number), with a configuration pane on each tab that's scrollable up<->down (which can have "unlimited" length), will be enough to contain "any" configuration UI?
lurch
on 14 Dec 2016
top scrollable left<->right
That's tricky since horizontal scrolling is not very effective. It might be better to have a LHS vertical navigation (scrollable if need be to host Ethernet, WiFi, Cellular, etc.) and the rest would contain sub-options of each option.
konmouz
on 14 Dec 2016
Some examples I know of from other modules:
{
message: 'Network SSID'
type: 'input'
name: 'networkSsid'
default: data.networkSsid
}
{
message: 'Network Key'
type: 'input'
name: 'networkKey'
default: data.networkKey
}
{
message: 'Do you want to set advanced settings?'
type: 'confirm'
name: 'advancedSettings'
default: false
}
{
message: 'Device Hostname'
type: 'input'
name: 'hostname'
default: data.hostname,
when: (answers) ->
answers.advancedSettings
}
{
message: 'Do you want to enable persistent logging?'
type: 'confirm'
name: 'persistentLogging'
default: data.persistentLogging
when: (answers) ->
answers.advancedSettings
}
{
message: 'Processor'
name: 'processorType'
type: 'list'
choices: [ 'Z7010', 'Z7020' ]
},
{
message: 'Coprocessor cores'
name: 'coprocessorCore'
type: 'list'
choices: [ '16', '64' ]
}
{
message: 'Network Type'
name: 'network'
type: 'list'
choices: [ 'ethernet', 'wifi' ]
}
{
message: 'Wifi Ssid'
name: 'wifiSsid'
type: 'input'
when:
network: 'wifi'
}
{
message: 'Wifi Key'
name: 'wifiKey'
type: 'input'
when:
network: 'wifi'
}
...and this is what the current resin.io dashboard offers you when downloading an image:
lurch
on 14 Dec 2016
Its worth noting that the screens @lurch posted are being automatically generated already :)
jviotti
on 14 Dec 2016
thanks guys, this is a good starting point.
konmouz
on 14 Dec 2016
Design Update: I prepared two quick flows with adobe XD (be careful no all the UI buttons work). I am trying to reduce/ group the CTAs around the img to keep the UI light and clean (Info, change, configure). Give it a go and let me know your thoughts (ignore image configuration modal window content for now - I am still working on it).
Flow 1: https://xd.adobe.com/view/bbdfb1e8-f1b8-4de2-a467-fc17cf3cb5ae/
Flow 2: https://xd.adobe.com/view/4006fa10-79dd-48c2-aee8-f9541b17e041/
konmouz
on 19 Dec 2016
Look nice, but for "Flow 1" I don't like the fact that the "Change Image" button is only 'hidden inside' the image-info modal. And for "Flow 2" I'm not sure it's very obvious that clicking on the small "X" will open the file-selection dialog?
My gut feeling is that "Change Image", "Image Info" and "Configure Image" are all buttons 'important enough' to warrant being on the main dialog (even if they're just icon-buttons).
Possibly _too_ radical of a departure from the current UI, but if we detect that the chosen image _is_ configurable (via its metadata), what if we changed the Etcher GUI to a four-step process, with a new "CONFIGURE IMAGE" stage appearing between the current "SELECT IMAGE" and "SELECT DRIVE" stages?
(and I guess the image meta-data could specify whether configuration is optional (in which case we could immediately enable the "Select drive" or "Flash!" buttons), or whether configuration is compulsory, (in which case we could keep the "Select drive" and "Flash!" buttons disabled until a 'valid' configuration has been entered))
And I guess this "emphasising" of the Configuration step would also make it more obvious that Etcher (along with the extended image metadata) offers more functionality than just your common-or-garden disk-image-writing software :-)
If the image has no metadata, or if its metadata doesn't contain any configuration info, then we could either have the GUI automatically adapt to its current three-stage layout, or we could simply leave the "CONFIGURE IMAGE" stage greyed-out (i.e. how the "FLASH IMAGE" stage appears when you first start Etcher).
lurch
on 19 Dec 2016
My gut feeling is that "Change Image", "Image Info" and "Configure Image" are all buttons 'important enough' to warrant being on the main dialog (even if they're just icon-buttons).
I think these are too many options, even with icons. Anyway the image name itself is a strong CTA that would be nice to include one of the functions. It is expected to click the image name to either get details or change it.
Possibly too radical of a departure from the current UI, but if we detect that the chosen image is configurable (via its metadata), what if we changed the Etcher GUI to a four-step process, with a new "CONFIGURE IMAGE" stage appearing between the current "SELECT IMAGE" and "SELECT DRIVE" stages?
This is an interesting suggestion to explore, but since anyway we are planning a heavy UI redesign to accommodate new functionality (not for v1), I would try to keep v1 as simple as possible.
konmouz
on 19 Dec 2016
Very cool stuff!
- Can we rename "Image Configuration" to something else, in a more imperative way as we do for other CTAs, like: "Configure Image"?
- I like displaying the image size next to the filename. How will this work for extremely long filenames though?
- If an image allows configuration, then the call to action for the select image step should clearly be "Image Configuration", however what do we do when the image doesn't allow configuration? We can show something else (like "Change"), but switching the label depending on the image would be a bit strange/confusing to me (for example: why did image X allow me to configure but not image Y? Did I do something wrong? How do I enable configuration?). Maybe we should show no bottom label at all if the image can't be configured?
Flow 1:
- I personally like the concept of the "cross" icon to remove the current image, but I worry that we're putting too much stuff horizontally on the same line (e.g: the filename, the size, the un-select icon, etc) given that we have no control over how long the filename could be. I guess its fine the way it is if we have a clear way forward for protecting ourselves agains long filenames.
Flow 2:
- I don't like how the "Image Configuration" label changes to "Change" once the user clicks the informative icon. I think it should stay the same.
Overall I believe we're almost there. The flow is very cool, so just a matter of polishing it a bit more :)
jviotti
on 19 Dec 2016
Can we rename "Image Configuration" to something else, in a more imperative way as we do for other CTAs, like: "Configure Image"?
Agreed
I like displaying the image size next to the filename. How will this work for extremely long filenames though?
We can set the classic solution of setting a limit and then use the triple-dot punctuation 'resinim...' or 'resinim... .img'
If an image allows configuration, then the call to action for the select image step should clearly be "Image Configuration", however what do we do when the image doesn't allow configuration? We can show something else (like "Change"), but switching the label depending on the image would be a bit strange/confusing to me (for example: why did image X allow me to configure but not image Y? Did I do something wrong? How do I enable configuration?). Maybe we should show no bottom label at all if the image can't be configured?
TBH I don't like showing and hiding buttons since it is highly confusing for the user. I think we can just grey out the 'configure image' and the tooltip can display 'this image is not configurable'.
I don't like how the "Image Configuration" label changes to "Change" once the user clicks the informative icon. I think it should stay the same.
Sorry my bad, I copied the old UI. I will update the image.
konmouz
on 19 Dec 2016
konmouz
on 20 Dec 2016
This is looking great :+1: So just to make it clear: can we add the flow where an image is not configurable, just to see how that would look?
jviotti
on 20 Dec 2016
@jviotti I've updated the link.
As discussed, we can remove the 'configure image' completely when an image is not configurable. The benefit with current solution is that we can let user know why the CTA is inactive (using a tooltip). If the CTA is not there, user has to make the assumption that this feature is not available with the selected image.
konmouz
on 20 Dec 2016
@konmouz this is looking awesome. One thing I worry about is the scrollable part of the configuration, but don't have solid ideas on alternatives.
shaunmulligan
on 20 Dec 2016
@shaunmulligan I am with you on this. Current solution is more like a placeholder and I am trying to come up with a better and modular solution to fit the needs of individual images.
konmouz
on 20 Dec 2016
@konmouz
As discussed, we can remove the 'configure image' completely when an image is not configurable. The benefit with current solution is that we can let user know why the CTA is inactive (using a tooltip). If the CTA is not there, user has to make the assumption that this feature is not available with the selected image.
I agree, it makes a lot of sense. I love the tooltip and the current flow. I wonder if the label should be completely greyed out if the image is not configurable -- currently, it turns to another type of blue, which might still give the impression that is clickable.
jviotti
on 20 Dec 2016
As a user, I feel a bit trapped when the only two options are "revert" and "looks good". I generally expect the secondary action to be passive (e.g. "cancel"), not active ("revert").
WasabiFan
on 20 Dec 2016
It may be nice to have the "configure image" give an explanation of the
sort "The current image cannot be configured due to absence of
configuration metadata. Only extended archive images are configurable.
(learn more)." That can act as a way for extended archives to be
discoverable while not being super intrusive. thoughts?
--
Alexandros Marinos
Founder & CEO, Resin.io
+1 206-637-5498
@alexandrosm
On Tue, Dec 20, 2016 at 6:02 PM, Wasabi Fan notifications@github.com
wrote:
As a user, I feel a bit trapped when the only two options are "revert" and
"looks good". I generally expect the secondary action to be passive (e.g.
"cancel"), not active ("revert").—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/718#issuecomment-268313178,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCLsQLAp6ziTcx7sr337C9HnzxL5zks5rKBhSgaJpZM4KFQHD
.
alexandrosm
on 20 Dec 2016
@konmouz Slight 'bug' in your current Flow in that when you click on the drive to display the drive-details modal, you can't click on the 'X' to close it. Also, when you select the non-configurable image, and click the small 'x' to deselect the drive, you can't then click on the small 'x' to deselect the image.
I like the way you've got the image-details modal and drive-details modal looking so similar :+1: (but FYI the image-details modal _may_ end up displaying more information when we add more info to the extended-archive metadata)
In the current Flow you need to 'close' an image before you can 'open' a new one (and the same for drives), whereas with the current GUI it's only a single button-click to open the file-dialog. OTOH, clicking on an 'x' to signify "open new image" is probably confusing from a UX point of view. What if you instead showed a folder-icon next to the image-name to indicate "open different image"; and a small disk icon next to the drive-name to indicate "select different disk"? (but then that also means that the "Select Image" and "Select Drive" buttons would _only_ be visible when you first started the GUI). Sorry, I'm waffling...
@alexandrosm that's obviously far too long for a tool-tip, and displaying a pop-up window just to explain that a feature isn't available does feel kinda awkward IMHO.
I agree with @WasabiFan that the "revert" button is potentially confusing - is it like "cancel" where all changes since the dialog were opened are discarded (but still keeping changes from previous dialog usages), or is it like "reset to defaults" where all changes are discarded and the image is reset to its virgin state?
Also, maybe it's because I'm old and non-hipster, but I'd prefer "OK" to "Looks Good".
lurch
on 21 Dec 2016
@jviotti, the button gets a transparency when inactive, grey should be ok as well.
@alexandrosm, I agree that this is a bit too long. what about the current text + learn more?
@lurch, thanks for the bugs, I'll have a look. Regarding the icons, in the initial flow the 'x' button redirects to the selection window directly (drive or image) which can be a bit confusing since it acts more like a 'replace' button. This is why I added this extra step, but obviously it is a bit longer. The icons you suggested or an icon for 'replace' can have multiple meanings for the user, while the 'x' one is much more clear and this is why I decided to stick to it.
konmouz
on 21 Dec 2016
what's the current text?
--
Alexandros Marinos
Founder & CEO, Resin.io
+1 206-637-5498
@alexandrosm
On Wed, Dec 21, 2016 at 9:26 AM, Konstantinos Mouzakis <
[email protected]> wrote:
@jviotti https://github.com/jviotti, the button gets a transparency
when inactive, grey should be ok as well.
@alexandrosm https://github.com/alexandrosm, I agree that this is a bit
too long. what about the current text + learn more?
@lurch https://github.com/lurch, thanks for the bugs, I'll have a look.
Regarding the icons, in the initial flow the 'x' button redirects to the
selection window directly (drive or image) which can be a bit confusing
since it acts more like a 'replace' button. This is why I added this extra
step, but obviously it is a bit longer. The icons you suggested or an icon
for 'replace' can have multiple meanings for the user, while the 'x' one is
much more clear and this is why I decided to stick to it.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/718#issuecomment-268476017,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCAUP17ezhRbS1_Ab8koKS5mLCqZqks5rKPDLgaJpZM4KFQHD
.
alexandrosm
on 21 Dec 2016
@alexandrosm, 'This image is not configurable'
konmouz
on 21 Dec 2016
but how would a learn more link work with a tooltip?
--
Alexandros Marinos
Founder & CEO, Resin.io
+1 206-637-5498
@alexandrosm
On Wed, Dec 21, 2016 at 10:03 AM, Konstantinos Mouzakis <
[email protected]> wrote:
@alexandrosm https://github.com/alexandrosm, 'This image is not
configurable'—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/718#issuecomment-268483656,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCEpWHYpEzULIlx8niT_NhubeT35eks5rKPmMgaJpZM4KFQHD
.
alexandrosm
on 21 Dec 2016
I guess @Shou may have the answer. But do we really need the 'learn more'? Where this will redirect?
konmouz
on 21 Dec 2016
The idea is to inform people about the existence of the format. If it can't
be discovered, it won't be used.
--
Alexandros Marinos
Founder & CEO, Resin.io
+1 206-637-5498
@alexandrosm
On Wed, Dec 21, 2016 at 10:26 AM, Konstantinos Mouzakis <
[email protected]> wrote:
I guess @Shou https://github.com/Shou may have the answer. But do we
really need the 'learn more'? Where this will redirect?—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/718#issuecomment-268488695,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCBv0cNXfTfl6t0u5gMAmIa1r5bkJks5rKP7ggaJpZM4KFQHD
.
alexandrosm
on 21 Dec 2016
@jviotti @lurch, regarding configuration options, can we put a limit on UI elements/tags (e.g. use up to 3 tabs)?
konmouz
on 21 Dec 2016
We could potentially keep displaying the tooltip on hover, then the link is clickable. Though if we want discoverability, I think it may make more sense to slap the message/link directly below, or somewhere else overt. Or perhaps we can trigger the tooltip to show up by default if it's non-configurable, but instead downwards so it doesn't overlap anything; it could either be dismissed by hovering over it when the cursor leaves (the normal tooltip behaviour), or a timeout.
Shou
on 21 Dec 2016
The idea is to inform people about the existence of the format. If it can't be discovered, it won't be used.
@alexandrosm Thinking about it, it's actually the image _publishers_ who we want to inform about the existence of the extended-archive format, not really the end-user Etcher consumers ;-)
So I think as long as we have good, clear, easily-navigable (and easily findable) documentation on https://etcher.io we should be okay? Publishers will (hopefully) naturally wonder "How can I make my image configurable?"
the 'x' one is much more clear and this is why I decided to stick to it.
Yeah, clarity should be preferred over minimising-clicks :+1:
regarding configuration options, can we put a limit on UI elements/tags (e.g. use up to 3 tabs)?
@konmouz Yeah, I guess for this initial version we could put that restriction in place, and then see if any image-publishers complain ;-) I guess if the image-publisher uses really-long tab titles then we could truncate them with the usual ellipses ... format?
lurch
on 21 Dec 2016
@jviotti @lurch, regarding configuration options, can we put a limit on UI elements/tags (e.g. use up to 3 tabs)?
I don't think we should. There are legitimate cases for complex configuration option combinations. I feel tabs are the wrong UI control here, given that it will clearly not scale to more complex set of options. Maybe we should include all the available sections as an horizontal stack instead, in order to get rid of the limitation?
@alexandrosm Thinking about it, it's actually the image publishers who we want to inform about the existence of the extended-archive format, not really the end-user Etcher consumers ;-)
So I think as long as we have good, clear, easily-navigable (and easily findable) documentation on https://etcher.io we should be okay? Publishers will (hopefully) naturally wonder "How can I make my image configurable?"
Yeah, but publishers are most likely users as well, and if it the feature is discoverable from the UI, its easier for both publishers and other users to see it (and users might then be the ones bugging publishers for it), so I feel we should try to make it discoverable both on the website and on the app itself.
jviotti
on 21 Dec 2016
My naive notion is that if users can see it, they will ask publishers, and
if enough users ask, the publishers will get curious.
--
Alexandros Marinos
Founder & CEO, Resin.io
+1 206-637-5498
@alexandrosm
On Wed, Dec 21, 2016 at 6:15 PM, Juan Cruz Viotti notifications@github.com
wrote:
@jviotti https://github.com/jviotti @lurch https://github.com/lurch,
regarding configuration options, can we put a limit on UI elements/tags
(e.g. use up to 3 tabs)?I don't think we should. There are legitimate cases for complex
configuration option combinations. I feel tabs are the wrong UI control
here, given that it will clearly not scale to more complex set of options.
Maybe we should include all the available sections as an horizontal stack
instead, in order to get rid of the limitation?@alexandrosm https://github.com/alexandrosm Thinking about it, it's
actually the image publishers who we want to inform about the existence of
the extended-archive format, not really the end-user Etcher consumers ;-)
So I think as long as we have good, clear, easily-navigable (and easily
findable) documentation on https://etcher.io we should be okay?
Publishers will (hopefully) naturally wonder "How can I make my image
configurable?"Yeah, but publishers are most likely users as well, and if it the feature
is discoverable from the UI, its easier for both publishers and other users
to see it (and users might then be the ones bugging publishers for it), so
I feel we should try to make it discoverable both on the website and on
the app itself.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/718#issuecomment-268598853,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCMDi2RgPVmot2CIEqu_PbhMw_orNks5rKWzTgaJpZM4KFQHD
.
alexandrosm
on 22 Dec 2016
I feel tabs are the wrong UI control here, given that it will clearly not scale to more complex set of options. Maybe we should include all the available sections as an horizontal stack instead, in order to get rid of the limitation?
Could you explain what you mean by 'horizontal stack' ? Do you mean sections stacked one under the other, creating a single huge vertically-scrollable view that users will find it really hard to navigate, or something else?
Looking at what some of the other applications on my Ubuntu 14.04* laptop do, there's a mixture of styles:
Single-pane only
Tabs only
Left-listbox only (kinda like tabs, but displayed on the left inside of across the top)
Both left-listbox and tabs
Left-treeview only (with expandable/collapsible branches)
Obviously some of them are better suited for 'simpler' configurations, and others are better suited for 'complex' configurations.
Which means (and I'm really hesitant to suggest this) that _maybe_ the Etcher "configuration-UI language" needs both a 'simple' version and a 'complex' version? :confused:
* Yes, I realise Linux apps traditionally don't have the best UX design, but it's all I have to hand right now.
lurch
on 22 Dec 2016
my mind naturally went to an accordion (similar to, but NOT
https://jqueryui.com/accordion/ ) for the settings.
alexandrosm
on 22 Dec 2016
Currently I am working on a different approaches/ solutions, including vertical scrolling, accordions, LHS navigation and tabs. Will upload soon to discuss.
konmouz
on 22 Dec 2016
Overall notes: For the settings options I created 6 elements that can cover all needs (let me know if I missed something). 1) input field (text), 2) classic dropdown with preloaded option, 3) confirmation field (a tick/switch option), 4) combo input field (text and switch to enable/disable), 5) combo dropdown (dropdown and switch to enable/disable), 6) notes field (discreet text to inform user - this can be replaced with an 'i' icon next to the field where you can hover over to get details). These elements can be used in one or two columns.
Accordion menu for main navigation. Ideally, 2 rules should apply here: 1) only one accordion unfolded at a time, 2) all options should be visible in the modal window without scrolling (scroll only inside the unfolded accordion - should have a limit of 3-4 accordion titles to provide some space to the selected accordion). Limited tabs (no more than 4-5, otherwise the UX will be challenging in this tiny space) is a nice way to avoid vertical stacks and therefore scrolling. We can also provide secondary titles inside an accordion or tab.
This is the most scalable and less UX-friendly solution. Plain vertical stack with scrolling. We can use only two levels of titles to organise information.
LHS navigation for the top level categories and RHS window for relevant content. Each category can have subcategories or/and limited tabs to organise information.
konmouz
on 22 Dec 2016
@konmouz we might also need the ability/option to drop in or paste a whole file as well.
shaunmulligan
on 22 Dec 2016
I absolutely love the LHS navigation + RHS content concept. Its both user friendly and scales really well :+1:
jviotti
on 23 Dec 2016
Looking _really_ nice @konmouz :-)
It occurs to me that (from the code point of view) we could probably have it so that if only one LHS-navigation is specified, we don't bother showing the LHS navigation bar; and if only one tab-pane is specified, we don't bother showing the tab-strip. And I think if we do it right, that allows the image-publisher to (be able to) specify both the 'simple' and 'complex' UIs that I mentioned earlier, while keeping things uniform from the code-side and without requiring inconsistent syntax for different UI-layout styles.
E.g. if the image-publisher specifies:
Category Title 01
Category Title 02
Tab 01
Subcategory Title 01
various controls
Subcategory Title 02
various controls
Category Title 03
Category Title 04
Category Title 05
then we'd get something looking like @konmouz 's last image.
But if the image-publisher instead specifies:
Category Title 01
Tab 01
Subcategory 01
various controls
Subcategory 02
various controls
then we'd get something like @konmouz 's second image.
But if the image-publisher instead specifies:
Category Title 01
Tab 01
Subcategory 01
various controls
Tab 02
Tab 03
then we'd get something like @konmouz 's first image (ignoring the accordion stuff).
Or for a simple specification the image-publisher could just do:
Category Title 01
Tab 01
Subcategory 01
various controls
and we'd just get a very simple UI with no navigation / multiple pages / etc. (obviously the titles used in these examples are purely illustrative, and the numbers and specific header-names wouldn't be required)
I.e. I guess what I'm proposing is that the "specification" hierarchy would always be (1-or-more Categories) -> (1-or-more Tabs) -> (1-or-more Sections) -> (1-or-more controls), with the "display rules" that if there's only one Category, then the LHS-navigation won't get shown, and if there's only one Tab, then the tab-navigation won't get shown.
What do people think, could something like that work? (if we wanted to get fancy, and if things were 'labelled' at what level they're at, then I guess we could optionally allow the Category and Tab levels to be 'skipped' (rather than forcing at least one to be specified), making things even easier for the image-publishers)
I'm unsure whether we need both Tabs and Sections, and in what order they should be nested. And we'll obviously need to come up with better names for these different 'levels'.
@konmouz I mentioned this briefly in a previous comment, but I think there needs to be an ability for a single control to be able to spread over the full dialog-width (either by the image-publisher specifying that, and/or by us calculating the size of the control, e.g. a combo-box with very long item names), rather than always assuming that it'll always be two "half-width" controls side by side?
@shaunmulligan Are you proposing a text-editor control, like NOOBS has? https://github.com/raspberrypi/noobs#easy-config-file-editor
(obviously the reconfix stuff should greatly reduce the need for resorting to editing files in plain-text mode!)
lurch
on 23 Dec 2016
A few things strike me here --
First, the project of autogeneratting forms from structured data strikes me
as something we should solve (one way or another) separately from Etcher
and Reconfix as it's a problem we need solved in a bunch of places.
At the same time, in the interest of clean semantics, I'd want to avoid
inserting terms that imply specific ui implementations like "tab" into the
format, rather keeping things on more freely interpretable terms like
"group/set/category" and other words that when paired with "theory" make up
some field of abstract algebra :P.
In particular, the reconfix/etcher configuration flow should be runnable
both from the UI and the CLI, and as such, we should probably be designing
any representation in a way it can serve both at the same time.
@jviotti, does this feel like it's worth breaking off into a separate
project or does it make sense to continue the work here for now?
--
Alexandros Marinos
Founder & CEO, Resin.io
+1 206-637-5498
@alexandrosm
On Fri, Dec 23, 2016 at 3:26 AM, Andrew Scheller notifications@github.com
wrote:
Looking really nice @konmouz https://github.com/konmouz :-)
It occurs to me that (from the code point of view) we could probably have
it so that if only one LHS-navigation is specified, we don't bother showing
the LHS navigation bar; and if only one tab-pane is specified, we don't
bother showing the tab-strip. And I think if we do it right, that
allows the image-publisher to (be able to) specify both the 'simple' and
'complex' UIs that I mentioned earlier, while keeping things uniform from
the code-side and without requiring inconsistent syntax for different
UI-layout styles.E.g. if the image-publisher specifies:
Category Title 01
Category Title 02
Tab 01
Subcategory Title 01
various controls
Subcategory Title 02
various controls
Category Title 03
Category Title 04
Category Title 05then we'd get something looking like @konmouz https://github.com/konmouz
's last image.But if the image-publisher instead specifies:
Category Title 01
Tab 01
Subcategory 01
various controls
Subcategory 02
various controlsthen we'd get something like @konmouz https://github.com/konmouz 's
second image.But if the image-publisher instead specifies:
Category Title 01
Tab 01
Subcategory 01
various controls
Tab 02
Tab 03then we'd get something like @konmouz https://github.com/konmouz 's
first image (ignoring the accordion stuff).Or for a simple specification the image-publisher could just do:
Category Title 01
Tab 01
Subcategory 01
various controlsand we'd just get a very simple UI with no navigation / multiple pages /
etc. (obviously the titles used in these examples are purely illustrative,
and the numbers and specific header-names wouldn't be required)I.e. I guess what I'm proposing is that the "specification" hierarchy
would always be (1-or-more Categories) -> (1-or-more Tabs) -> (1-or-more
Sections) -> (1-or-more controls), with the "display rules" that if there's
only one Category, then the LHS-navigation won't get shown, and if there's
only one Tab, then the tab-navigation won't get shown.
What do people think, could something like that work? (if we wanted to get
fancy, and if things were 'labelled' at what level they're at, then I guess
we could optionally allow the Category and Tab levels to be 'skipped'
(rather than forcing at least one to be specified), making things even
easier for the image-publishers)I'm unsure whether we need both Tabs and Sections, and in what order they
should be nested. And we'll obviously need to come up with better names for
these different 'levels'.@konmouz https://github.com/konmouz I mentioned this briefly in a
previous comment, but I think there needs to be an ability for a single
control to be able to spread over the full dialog-width (either by the
image-publisher specifying that, and/or by us calculating the size of the
control, e.g. a combo-box with very long item names), rather than always
assuming that it'll always be two "half-width" controls side by side?@shaunmulligan https://github.com/shaunmulligan Are you proposing a
text-editor control, like NOOBS has? https://github.com/
raspberrypi/noobs#easy-config-file-editor
(obviously the reconfix stuff should greatly reduce the need for resorting
to editing files in plain-text mode!)—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/718#issuecomment-268935327,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCN-CEOvpwjmyutrlKD_jb246Dsrwks5rKz9dgaJpZM4KFQHD
.
alexandrosm
on 23 Dec 2016
It occurs to me that (from the code point of view) we could probably have it so that if only one LHS-navigation is specified, we don't bother showing the LHS navigation bar; and if only one tab-pane is specified, we don't bother showing the tab-strip. And I think if we do it right, that allows the image-publisher to (be able to) specify both the 'simple' and 'complex' UIs that I mentioned earlier, while keeping things uniform from the code-side and without requiring inconsistent syntax for different UI-layout styles.
If we can do so, it would be amazing.
I'm unsure whether we need both Tabs and Sections, and in what order they should be nested. And we'll obviously need to come up with better names for these different 'levels'.
In case we provide tabs and in order to keep the categories organised, tabs should be allowed once per Category and always top level. Meaning that tabs can work as subcategories and subcategory can be inside a tab or at the same level. Otherwise, we can completely remove tabs.
e.g.:
Category 01
Tab 01
Subcategory 01
Tab 02
Subcategory 01
Subcategory 02
Tab 03
Subcategory 01
Subcategory 02
At the same time, in the interest of clean semantics, I'd want to avoid
inserting terms that imply specific ui implementations like "tab" into the
format, rather keeping things on more freely interpretable terms like
"group/set/category" and other words that when paired with "theory" make up
some field of abstract algebra :P.
I see the point but also think that publishers might want to have a control over the UI. On the other hand, grouping info as you suggested (and bearing in mind the broader context you mentioned) gives us the opportunity to create a fluid/adaptive UI that would be altered in relevance with the specific context (e.g. Etcher) and with publisher's content.
konmouz
on 23 Dec 2016
Let's try as hard as we can to make the automatic ui sane enough that
publishers don't have a strong need to alter it. We need to balance
publisher requests with the broader roadmap, that also works in publishers'
favour :)
--
Alexandros Marinos
Founder & CEO, Resin.io
+1 206-637-5498
@alexandrosm
On Fri, Dec 23, 2016 at 10:10 AM, Konstantinos Mouzakis <
[email protected]> wrote:
It occurs to me that (from the code point of view) we could probably have
it so that if only one LHS-navigation is specified, we don't bother showing
the LHS navigation bar; and if only one tab-pane is specified, we don't
bother showing the tab-strip. And I think if we do it right, that allows
the image-publisher to (be able to) specify both the 'simple' and 'complex'
UIs that I mentioned earlier, while keeping things uniform from the
code-side and without requiring inconsistent syntax for different UI-layout
styles.If we can do so, it would be amazing.
I'm unsure whether we need both Tabs and Sections, and in what order they
should be nested. And we'll obviously need to come up with better names for
these different 'levels'.In case we provide tabs and in order to keep the categories organised,
tabs should be allowed once per Category and always top level. Meaning that
tabs can work as subcategories and subcategory can be inside a tab or at
the same level. Otherwise, we can completely remove tabs.
e.g.:
Category 01 Tab 01 Subcategory 01 Tab 02 Subcategory 01 Subcategory 02 Tab
03 Subcategory 01 Subcategory 02At the same time, in the interest of clean semantics, I'd want to avoid
inserting terms that imply specific ui implementations like "tab" into the
format, rather keeping things on more freely interpretable terms like
"group/set/category" and other words that when paired with "theory" make up
some field of abstract algebra :P.I see the point but also think that publishers might want to have a
control over the UI. On the other hand, grouping info as you suggested (and
bearing in mind the broader context you mentioned) gives us the opportunity
to create a fluid/adaptive UI that would be altered in relevance with the
specific context (e.g. Etcher) and with publisher's content.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/718#issuecomment-268967573,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCD-nNptSkY6SWe60TdqySXMM23WTks5rK54CgaJpZM4KFQHD
.
alexandrosm
on 23 Dec 2016
Here is a real-life example of the kind of file that needs to be generated: connman-service.conf.
I am trying to visualize how the UI would work for this...
- First, it allows for multiple networks to be configured, so I am imaging a need for a
+button to add new configurations. Or would Etcher just be limited to a fixed number? - Second, there will need to be a way to select either Wired or Wi-Fi. The remaining UI will depend on this selection. For example, if Wired is selected, the Wi-Fi fields like security type and passphrase should be disabled or hidden.
- Similarly, if DHCP is selected, fields for IP address, netmask and gateway should be disabled or hidden.
dlech
on 24 Dec 2016
More real-life example. As a publisher, I would want to do something like this...
It has some complex rules... For example, For Raspberry Pi 2 Model B, there is no built-in Bluetooth or Wi-Fi, so they are disabled. But, Raspberry Pi 3 has them.
However, the BrickPi addon board uses the same serial port as Bluetooth. Bluetooth can work by using a different UART, but at the cost of reduced performance. So if it is selected, I want to show a warning. This could be a popover with an explanation or a link to a website.
PiStorms, on the other hand, doesn't have this problem, so we don't want to show the warning for it.
And here is the Network pane, but don't pay too much attention to it because it is not exactly how I would like it yet.
dlech
on 24 Dec 2016
And here is a stab at how you might define this is json. I think it works quite well.
{
"categories": [
{
"label": "Hardware",
"groups": [
{
"label": "Raspberry Pi",
"options": [
{
"id": "model",
"type": "combobox",
"label": "Model",
"items": [
{
"id": "model_rpi2",
"text": "Raspberry Pi 2 Model B"
},
{
"id": "model_rpi3",
"text": "Raspberry Pi 3 Model B"
}
]
},
{
"type": "note",
"text": "Raspberry Pi Models Zero and 1 are not supported by this image"
},
{
"id": "rpi3_bluetooth",
"type": "checkbox",
"label": "Built-in Bluetooth",
"badge": {
"type": "warning",
"visible": "eval(model.selected == 'model_rpi3' && rpi3_bluetooth.checked && addon_board.selected == 'board_brickpi')",
"popover": "The built-in Bluetooth will operate using the 'miniuart' on the Raspberry Pi, which could result in degraded perfomance."
},
"enabled": "eval(model.selected == 'model_rpi3')"
},
{
"id": "rpi3_wifi",
"type": "checkbox",
"label": "Built-in Wi-Fi",
"enabled": "eval(model.selected == 'model_rpi3')"
}
]
},
{
"label": "Addon Board",
"options": [
{
"id": "addon_board",
"type": "combobox",
"label": "Model",
"items": [
{
"id": "board_brickpi",
"text": "Dexter Industries BrickPi"
},
{
"id": "board_pistorms",
"text": "Mindsensors.com PiStorms"
}
]
}
]
}
],
},
{
"label": "Network",
"groups": [
{
"label": "General",
"options": [
{
"id": "hostname",
"type": "textbox",
"label": "Hostname",
"default": "ev3dev",
"validate": "regex('[a-zA-Z0-9_-]{1,15}')"
}
]
},
{
"label": "Wi-Fi",
"options": [
{
"id": "wifi_security",
"type": "combobox",
"label": "Security",
"items": [
{
"id": "wifi_security_open",
"text": "Open"
},
{
"id": "wifi_security_wep",
"text": "WEP"
},
{
"id": "wifi_security_wpa",
"text": "WPA/WPA2"
}
]
},
{
"id": "wifi_ssid",
"type": "textbox",
"label": "SSID"
},
{
"id": "wifi_passphrase",
"type": "textbox",
"password": true,
"label": "Passphrase",
"enabled": "eval(wifi_security.selected != 'wifi_security_open')"
},
{
"id": "wifi_ssid_hidden",
"type": "checkbox",
"label": "Hiddden"
}
]
}
]
}
],
}
@alexandrosm
First, the project of autogeneratting forms from structured data strikes me
as something we should solve (one way or another) separately from Etcher
and Reconfix as it's a problem we need solved in a bunch of places.
My gut feeling is that the best place for this would be as an extension to reconfix, rather than a generic / standalone "autogenerating forms from structured data" project, since I suspect many of the "data-type" issues, will be stuff that reconfix needs to handle anyway? e.g. differences between options that are free-form text, and options that are a single selection from a limited list of possible choices (where a combo-box would traditionally be used). [I should probably look in more detail at the SchemaForm project you mentioned earlier]
Let's try as hard as we can to make the automatic ui sane enough that
publishers don't have a strong need to alter it.
That's a noble goal, but I suspect there'll have to be at _least_ some kind of UI/UX "hints" in the data, e.g. to know which data should be grouped together to display on one "page" at a time. (i.e. network settings and display settings should clearly be displayed on separate "pages" (panes/tabs/modals/whatever) but from the data itself it may not be possible to automatically deduce this grouping. And then there's other issues like knowing which fields should be displayed in the preferences UI using a plain-text entry control, and which should be displayed using a password-text entry control. And what display-order makes most sense for the controls.
@dlech Hmm, complicated examples indeed. That's really useful, thankyou.
Your JSON approach is interesting, but IMHO eval has no place whatsoever in JSON - we should try to come up with some more declarative (and programming-language-neutral) scheme of defining the "complex rules" surrounding visibility, enabled-state, validation, etc. (I can imagine that changing the value of one control, might cause the validation-logic of another control to change).
I'm too tired right now to work out if it's actually any better :wink: , but I wonder if using something in the style of a spreadsheet-formula is better? i.e.
"visible": "AND(AND(EQUALS(model.selected, 'model_rpi3'), rpi3_bluetooth.checked), EQUALS(addon_board.selected, 'board_brickpi')))"
Or perhaps something like an SQL WHERE-clause?
"visible": "model.selected = 'model_rpi3' AND rpi3_bluetooth.checked AND addon_board.selected = 'board_brickpi'"
Of course this example is only testing values on the same "page" but I guess (thinking back to the VirtualBox example) there's cases where settings on one "page" might also affect what settings (controls & their validation) get displayed on a different "page" (i.e. settings-values might need to have the ability to be 'globally scoped' so that they can be accessed from any other "page").
lurch
on 24 Dec 2016
Your JSON approach is interesting, but IMHO eval has no place whatsoever in JSON
Indeed. I did not mean that javascript eval should be used, but just consider it pseudo code for some declarative language as you suggest, preferably an already existing one with a javascript library (that protects against the execution of arbitrary code) so that we don't have to invent a new one.
dlech
on 24 Dec 2016
@jviotti, does this feel like it's worth breaking off into a separate
project or does it make sense to continue the work here for now?
There are some cool stuff going on here, and the problem applies directly to this issue (apart from being useful in other projects as well), so I guess we can continue diving into it here, and extract the discussion somewhere else when actually implementing it (it will probably be a separate repo)
My gut feeling is that the best place for this would be as an extension to reconfix, rather than a generic / standalone "autogenerating forms from structured data" project, since I suspect many of the "data-type" issues, will be stuff that reconfix needs to handle anyway? e.g. differences between options that are free-form text, and options that are a single selection from a limited list of possible choices (where a combo-box would traditionally be used). [I should probably look in more detail at the SchemaForm project you mentioned earlier]
Maybe. The way you display the forms is completely decoupled from Reconfix. Having them together has some advantages, but it's also tempting to keep them separate in order to keep the repo focused on the core of Reconfix. I didn't make my mind about it yet :)
jviotti
on 24 Dec 2016
Fantastic stuff @dlech!
We've already used reconfix to configure connman on resinOS 1.x, and for
network manager on resinOS 2.x. It's funny you mentioned connman because
it's been specifically a thing that we used to calibrate the design of
reconfix.
On the "warning when eval()" issue, a solution that immediately pops to
mind is pattern matching on the Dry JSON structure. So it would either be a
JSON fragment that the engine would attempt to match to the state entered
by the user (and display the warning if true), or, even better, some kind
of JSON query/schema that could allow a range of potential JSON fragments
to be described as triggering the warning. In that way, it can be cleanly
layered on top of the Dry JSON, be used in the UI layer(s), while at the
same time isolating it from deeper layers of the data.
You'll also be glad to know that the current reconfix configuration format
allows for differing types of configuration options depending on a
top-level option, though I'd want @jviotti to confirm your pattern fits
within it.
@lurch, when I say "make the automatic ui sane", I don't mean "don't have
groups at all". What I mean is "don't have things named Tab, Dropdown,
Button, etc", in a way that makes it impossible for the CLI to utilise
natively. Basically don't make the semantic description of the structure
prescriptive as to the interaction that will fill it in, in a way that
prevents unforeseen (or foreseen) use. Maybe someone would want to use this
same representation to get settings via a voice interface. They shouldn't
be forced to decide how to describe a "dropdown". This line of thinking
becomes more specific when we get to things like the dropdown vs radio
buttons. My approach is to have a construct for "select one of the set" and
in the visual case use a radio or dropdown depending on the number of
options, with some sane threshold.
--
Alexandros Marinos
Founder & CEO, Resin.io
+1 206-637-5498
@alexandrosm
On Sat, Dec 24, 2016 at 5:58 AM, Juan Cruz Viotti notifications@github.com
wrote:
@jviotti https://github.com/jviotti, does this feel like it's worth
breaking off into a separate
project or does it make sense to continue the work here for now?There are some cool stuff going on here, and the problem applies directly
to this issue (apart from being useful in other projects as well), so I
guess we can continue diving into it here, and extract the discussion
somewhere else when actually implementing it (it will probably be a
separate repo)My gut feeling is that the best place for this would be as an extension to
reconfix, rather than a generic / standalone "autogenerating forms from
structured data" project, since I suspect many of the "data-type" issues,
will be stuff that reconfix needs to handle anyway? e.g. differences
between options that are free-form text, and options that are a single
selection from a limited list of possible choices (where a combo-box would
traditionally be used). [I should probably look in more detail at the
SchemaForm project you mentioned earlier]Maybe. The way you display the forms is completely decoupled from
Reconfix. Having them together has some advantages, but it's also tempting
to keep them separate in order to keep the repo focused on the core of
Reconfix. I didn't make my mind about it yet :)—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/718#issuecomment-269070796,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCOvid7FFy4FAups8mIQMB_46vESlks5rLLScgaJpZM4KFQHD
.
alexandrosm
on 24 Dec 2016
@alexandrosm Sure, using "semantic descriptions" is kinda what I meant when I earlier said we need to find "better names". By having things grouped hierarchically (and with a limited 'depth') then the (G)UI layered on top can be switched to use tabs, or accordions, or LHS-menu navigation, or something else, and it should "all just work".
With your example of dropdowns vs. radios (and I'm sure there'll be other examples too), I wonder if we could have the automatic GUI providing a default, with the publisher able to add hints / overrides if the automatic system doesn't quite do what they want? (with the proviso that not all "(G)UI implementations" of this scheme will necessarily support the overrides)
lurch
on 24 Dec 2016
I suspected we're generally in violent agreement, good to see you feel the
same way :D. As to the hint being provided, I'd prefer to wait it out and
see if people really do need that kind of knob, even though I can think of
a way where such a threshold can be expressed in a non-ui-type-biased way.
--
Alexandros Marinos
Founder & CEO, Resin.io
+1 206-637-5498
@alexandrosm
On Sat, Dec 24, 2016 at 1:49 PM, Andrew Scheller notifications@github.com
wrote:
@alexandrosm https://github.com/alexandrosm Sure, using "semantic
descriptions" is kinda what I meant when I earlier said we need to find
"better names". By having things grouped hierarchically (and with a limited
'depth') then the (G)UI layered on top can be switched to use tabs, or
accordions, or LHS-menu navigation, or something else, and it should "all
just work".With your example of dropdowns vs. radios (and I'm sure there'll be other
examples too), I wonder if we could have the automatic GUI providing a
default, with the publisher able to add hints / overrides if the automatic
system doesn't quite do what they want? (with the proviso that not all
"(G)UI implementations" of this scheme will necessarily support the
overrides)—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/718#issuecomment-269085149,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCGt6jKjzg-0dMGdcmqXk19wlnLltks5rLSLlgaJpZM4KFQHD
.
alexandrosm
on 24 Dec 2016
I like @dlech 's schema and I think it fits our model very well, however we need to think how to present custom operators such as eval and regex. We can keep it they way it is, or make use of something like https://github.com/jviotti/queryl.
jviotti
on 26 Dec 2016
It worth stating again that the visuals stage is completely decoupled from Reconfix, so we can put something up and keep iterating without any constraints, which is a great position to be in.
jviotti
on 26 Dec 2016
Any update on this?
ensonic
on 5 Jul 2017
We have a small team converting the proof of concept into a real module. Keep an eye on this repo: https://github.com/resin-io/reconfix/
jviotti
on 5 Jul 2017
I feel like this thread turned into feature creep insanity. Why not just allow the user to provide a "post-etching" script? Etcher would provide disk partition info to the script via environment variable.
Simple. Not specific to any hardware/OS/project.
As an iterative update to that feature, grep the provided script for a line that includes ".*expected environment variables:". If found, after the colon will be considered a JSON map of {"variable_name": "default value"} pairs that will be displayed in the UI with text fields.
Again, simple. Not specific to assumptions about wifi setup, etc. Etcher is a globally useful tool for imaging SD cards. It shouldn't become PiBakery.
RichardBronosky
on 12 Nov 2018
I found this issue thread via another by @KM4YRI (via @lurch) that asked about a GUI WiFi config during the flash process for Etcher.
Just wondering, what's the latest state of this, and is it possible to add anything to a custom Raspbian OS image to flag to Etcher to ask for WiFi credentials in the GUI as yet?
vindard
on 13 Sep 2020
Related issues
TheEpicNoobZilla
·
4Comments
mwiegant
·
5Comments
markcorbinuk
·
5Comments
AreDubya
·
3Comments
jviotti
·
5Comments
Most helpful comment
+1 Another usecase for such a feature would be to setup wifi. On many devices entering the wifi settings is hard. I don't want to complicate the feature request, but I wonder if it should be considered to be able to re-run the customization steps on a preflashed image?