Mu: Feature Request: Manually Select COM port

[the-stanely]

This request is for Windows 10.

Please provide a means for the user to select desired COM port. It would be useful for users with a number of active COM ports.

Also, multiple instances of Mu are possible, so it would be useful when working on multiple IoTs that need to communicate with each other.

Thank you.

[dybber]

I completely agree. Without the ability to override the automatically selected device, it's also problematic to communicate with certain devices. For instance, the currently very popular ESP32-based "M5StickC" uses a generic Vendor and Product ID, so it is not possible to detect it. The user has to be able choose to connect to unrecognized devices such as the M5StickC.

I've just created a few mockups of how it could work. We could add a dropdown in the statusbar, where the user can choose between devices:

For Mac and Linux it would be a bit more complex, for example:

Would this make sense? I could try to work on it, if we can agree on how it should work. Perhaps @ntoll has some input here?

We would still have to decide on a few other things as well:

• Should the selected device follow the open buffer? What about the REPL, should it also support multiple open connections? Alternatively, only support one open connection and make the setting global - thus when you change device, the connection will close, and a new connection would be opened to the selected device. This alternative would be the simplest to do.
• Should the dropdown menu only be shown when in one of the MicroPython modes? Or should it be shown as soon as there are any USB-serial devices? I think it would make sense to make it part of the MicroPython modes, rather than a global thing

[dybber]

Further discussion along the same lines, is in Issue #851 and pull-request #855. Just mentioning @carlosperate, as he could probably also chip in here

[the-stanely]

From a user's perspective, the pull-down menu in the status bar seems like an excellent option. My perception of Mode is that it selects the target, and a COM menu would select the channel. That arrangement makes a lot of sense. Targets that don't use a COM port for communication probably don't need to see a port selection, or it could be grayed out, or it could be there and do nothing of value until the user switched to a target on a port.

Mu could still attempt to pre-select the target/COM port it thought was right, and the user could change it if needed.

From my perspective, it would be fine for REPL to open a dedicated connection to the target. If the user wants to work simultaneously with another target, they could open another instance of Mu and select their new target/port there.

Working with multiple devices/ports with one instance of Mu would probably get complicated. It would need some sort of tabbed environment. I would be fine having two or more Mu windows open. In fact, I would prefer it.

[dybber]

Hi @the-stanely

I've worked on this, and the dropdown is no longer just a mockup, but actually allows you to select between different MicroPython devices. However, I don't run Windows, and it would nice if you could help test it, as I've only tested it on Mac for now.

The branch is available over here: https://github.com/dybber/mu/tree/feature/port_selection

I still need to do some polishing, e.g. a prompt to change mode when the user changes device, so it's not completely ready to merge into Mu just yet, but it would be nice with some input before I continue.

[the-stanely]

I'm excited to try it out! Can you just give me a quick overview of what I need to install it? I didn't see an installation exe, like with my other Mu versions. I'm running 64-bit Win10 and am set up to develop on at least 3 ESP32s simultaneously .

[dybber]

Hi @the-stanely

As I'm on a Mac, I'm not able to actually make an .exe installer, unfortunately (it was previously, but something in the build system have changed). If you want to try it you would have to do that yourself. The general way to go is to set up the development environment as described here: https://mu.readthedocs.io/en/latest/setup.html

Then you should be able to create the .exe by running make win64 in your terminal, standing in the mu-directory (same folder as the Makefile).

Martin

[the-stanely]

OMG those are some nasty instructions, plus they don't work. I seriously doubt many Windows users would be able to get a repo going with those.

Anyway, it was like pulling teeth but finally got the "port_selection" version running. Will let you know what I see.

[the-stanely]

Looks like it's working, thank you! I have 4 COM ports, 3 USBs and a 9-pin RS-232. There are 3 ESP32s connected to the 3 USB ports. For each one, concurrently in their own sessions, I was able to:

• REPL
• view Files
• RUN

• transfer file to ESP32

  Each of my ESP32 modules are different. They are:

• Heltec WiFi Kit w/0.96" OLED Display

• LilyGo TTGO w/1.14" TFT Color Display
• Devkit (older style) with Nokia 5110 attached

Although they are different boards from different manufacturers, they all show up in the port list as, CP210x. I don't know what that means because I didn't see that in any of their descriptions/specs when I was selecting them for evaluation. It would be nice if the port list could also include the port number/ID so I could more easily tell which is which.

I just got a couple different ESP32s that I'll test tomorrow. I'll also do some more extensive tests with all of them.

But this is very cool and I'm really excited. I'm working simultaneously with several ESP32s all at once. I hope others can use this feature as well.

[the-stanely]

No go with the 'make win64'. Got some stuff I don't understand...

========================================================== FAILURES ===========================================================
_____________________________ test_micropython_mode_find_device_darwin_remove_extraneous_devices ______________________________

    def test_micropython_mode_find_device_darwin_remove_extraneous_devices():
        """
        Check that if on OS X, only one version of the same device is shown,
        as OS X shows every device on two different ports.
        """
        editor = mock.MagicMock()
        view = mock.MagicMock()
        mm = MicroPythonMode(editor, view)
        mock_port = mock.MagicMock()
        mock_port.portName = mock.MagicMock(return_value="tty.usbserial-XXX")
        mock_port.productIdentifier = mock.MagicMock(return_value=0x0204)
        mock_port.vendorIdentifier = mock.MagicMock(return_value=0x0D28)
        mock_port.serialNumber = mock.MagicMock(return_value="123456")
        mock_port2 = mock.MagicMock()
        mock_port2.portName = mock.MagicMock(return_value="cu.usbserial-XXX")
        mock_port2.productIdentifier = mock.MagicMock(return_value=0x0204)
        mock_port2.vendorIdentifier = mock.MagicMock(return_value=0x0D28)
        mock_port2.serialNumber = mock.MagicMock(return_value="123456")
        device = Device(
            mock_port2.vendorIdentifier(),
            mock_port2.productIdentifier(),
            "/dev/" + mock_port2.portName(),
            mock_port2.serialNumber(),
            "micro:bit",
            None,
            None,
        )
        with mock.patch("sys.platform", "darwin"), mock.patch(
            "mu.modes.base.QSerialPortInfo.availablePorts",
            return_value=[mock_port, mock_port2],
        ):
>           assert mm.find_devices() == [device]
E           assert [] == [<mu.logic.De...021511FEFF28>]
E             Right contains one more item: <mu.logic.Device object at 0x0000021511FEFF28>
E             Use -v to get the full diff

tests\modes\test_base.py:321: AssertionError
------------------------------------------------------ Captured log call ------------------------------------------------------
WARNING  mu.modes.base:base.py:432 Could not find device.
=

[ZanderBrown]

Well based on the test title and that your on Windows thats a dodgy test

[dybber]

Hi @the-stanely, great that you are excited and many things are working. Here are some responses:

• Currently the port ID is shown as a tooltip in the dropdown. 99% of users would only ever connect one device at a time, and wouldn't need this information, expert users can do mouse over or look in the log-file in the admin-window.

• The name shown is based on VendorID and ProductID, as shown in this list: https://github.com/dybber/mu/blob/feature/port_selection/mu/modes/esp.py#L44 The problem with some of these devices is that the manufacturer doesn't use their own VendorID/ProductID, but ships their products with very generic VendorID/ProductID of those manufacturing the USB-serial controller. (E.g. Heltec should have their own VendorID, like Adafruit has their own VendorID, Heltec should not use the VendorID of the CP2102 USB-controller they are using)

What we can try to do is recognize such devices based on "manufacturer string". That's what I've done to recognize devices manufactured by https://m5stack.com/
It might give us problems, as I'm unsure if that part of the code works across platforms (Mac/Windows/Linux), but let us at least try. I've just pushed a commit that prints the device info including "manufacturer string" to the log window on connect/disconnect. Could you look at what manufacturer string is printed for each of the devices?

• I forgot that 'make win64' which creates an installer requires all tests to pass and full coverage perhaps as well(?). Anyway, the concrete test case should actually run the same on Mac and Windows, but I will have to look closer at it to understand if there's something I'm missing, I will try to fix that and report back!

• Yes setting up the development environment just to test a branch is inconvenient. I haven't been involved setting up the build system, but that's another topic entirely how to automatically generate .exe files (e.g. for every pull-request)

[dybber]

I might have fixed the test case that failed as well, please let me know if it is still failing. (I forgot to patch os.name to "posix")

[the-stanely]

@dybber... Oh, MOUSEOVER! Never occurred to me. This gives all the info I need. Very cool! Also found same info in the log, though that's not as easy as the mouseover.

Tried the build again with 'make win64'. It didn't create an exe. Different failures this time, but I think it's getting closer. Here are mostly the errors...

Installing collected packages: pycodestyle, entrypoints, pyflakes, mccabe, flake8, pyserial, six, python-dateutil, tornado, decorator, ipython-genutils, traitlets, pywin32, jupyter-core, pyzmq, jupyter-client, backcall, wcwidth, prompt-toolkit, parso, jedi, pygments, colorama, pickleshare, ipython, ipykernel, qtconsole, numpy, pygame, pgzero, appdirs, semver, nudatus, itsdangerous, Werkzeug, MarkupSafe, Jinja2, click, Flask, PyQt5-sip, PyQt5, QScintilla, PyQtChart, regex, typed-ast, toml, pathspec, attrs, black, mu-editor
    Running setup.py install for backcall ... done
  Attempting uninstall: mu-editor
    Found existing installation: mu-editor 1.1.0a2
    Not uninstalling mu-editor at c:\users\stanely\mu_editor\mu-port_sel, outside environment C:\Users\stanely\AppData\Local\Temp\mu-pynsist-kx7phqin\mu-packaging-venv
    Can't uninstall 'mu-editor'. No files were found to uninstall.
    Running setup.py install for mu-editor ... done
Successfully installed Flask-1.0.2 Jinja2-2.11.1 MarkupSafe-1.1.1 PyQt5-5.12.1 PyQt5-sip-4.19.19 PyQtChart-5.12.0 QScintilla-2.11.1 Werkzeug-1.0.0 appdirs-1.4.3 attrs-19.3.0 backcall-0.1.0 black-19.10b0 click-7.1.1 colorama-0.4.3 decorator-4.4.2 entrypoints-0.3 flake8-3.7.9 ipykernel-5.1.4 ipython-7.13.0 ipython-genutils-0.2.0 itsdangerous-1.1.0 jedi-0.16.0 jupyter-client-6.0.0 jupyter-core-4.6.3 mccabe-0.6.1 mu-editor-1.1.0a2 nudatus-0.0.4 numpy-1.18.1 parso-0.6.2 pathspec-0.7.0 pgzero-1.2 pickleshare-0.7.5 prompt-toolkit-3.0.4 pycodestyle-2.5.0 pyflakes-2.1.1 pygame-1.9.6 pygments-2.6.1 pyserial-3.4 python-dateutil-2.8.1 pywin32-227 pyzmq-19.0.0 qtconsole-4.4.3 regex-2020.2.20 semver-2.9.1 six-1.14.0 toml-0.10.0 tornado-6.0.4 traitlets-4.3.3 typed-ast-1.4.1 wcwidth-0.1.8
Creating pynsist configuration file C:\Users\STAN(E~1\AppData\Local\Temp\mu-pynsist-kx7phqin\pynsist.cfg
Getting frozen requirements.
Checking for wheel availability at PyPI.
...
- backcall==0.1.0 missing
...
- nudatus==0.0.4 missing
...
- PyQtChart==5.12.0 Traceback (most recent call last):
  File "win_installer.py", line 300, in <module>
    run(bitness, repo_root)
  File "win_installer.py", line 264, in run
    installer_exe = create_pynsist_cfg(venv_python, repo_root, pynsist_cfg)
  File "win_installer.py", line 196, in create_pynsist_cfg
    wheels = pypi_wheels_in(requirements)
  File "win_installer.py", line 155, in pypi_wheels_in
    if any(r.package_type == "wheel" for r in releases):
TypeError: 'NoneType' object is not iterable

[the-stanely]

Hold up, I'm trying the build again but it will take a while. I think there was a problem with my virtual environment.

So one of the reasons I'm having difficulty setting up the development environment is because try as I might, GitHub won't give me the 'port_selection' repo. It always gives me the mu repo it was forked from. The only way I could do it was by downloading the zip. Admittedly, I don't understand GitHub that well.

[the-stanely]

Yes, there was a problem with my environment, but after I fixed it, still no go. It started out with these two errors:

Error: [Errno 2] No such file or directory: 'C:\\Users\\Stanely\\mu_editor\\mu-port_sel\\.venv\\python.exe'

Error: [Errno 2] No such file or directory: 'C:\\Users\\Stanely\\mu_editor\\mu-port_sel\\.venv\\pythonw.exe'

So it looks like an error int the build script. I found those files in '.venv\Scripts' and copied them into '.venv'. Then I got this:

'''
Building 64-bit Windows installer
Temporary working directory at C:\Users\STANELY\AppData\Local\Temp\mu-pynsist-lz14r5f2
Creating the packaging virtual environment.
Error: Command '['C:\Users\STANELY\AppData\Local\Temp\mu-pynsist-lz14r5f2\mu-packaging-venv\Scripts\python.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 3221226505.
Updating pip in the virtual environment C:\Users\STANELY\AppData\Local\Temp\mu-pynsist-lz14r5f2\mu-packaging-venv\Scriptspython.exe
Fatal Python error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x0000365c (most recent call first):
Installing mu with C:\Users\STANELY\AppData\Local\Temp\mu-pynsist-lz14r5f2\mu-packaging-venv\Scriptspython.exe
Fatal Python error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00009ae8 (most recent call first):
Creating pynsist configuration file C:\Users\STANELY\AppData\Local\Temp\mu-pynsist-lz14r5f2pynsist.cfg
Getting frozen requirements.
Fatal Python error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00009abc (most recent call first):
Traceback (most recent call last):
File "win_installer.py", line 300, in
run(bitness, repo_root)
File "win_installer.py", line 264, in run
installer_exe = create_pynsist_cfg(venv_python, repo_root, pynsist_cfg)
File "win_installer.py", line 193, in create_pynsist_cfg
for line in pip_freeze(python, encoding=encoding)
File "win_installer.py", line 124, in pip_freeze
output = subprocess.check_output([python, "-m", "pip", "freeze", "--all"])
File "C:\Anaconda3\lib\subprocess.py", line 395, in check_output
**kwargs).stdout
File "C:\Anaconda3\lib\subprocess.py", line 487, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['C:\Users\STANELY\AppData\Local\Temp\mu-pynsist-lz14r5f2\mu-packaging-venv\Scripts\python.exe', '-m', 'pip', 'freeze', '--all']' returned non-zero exit status 3221226505.
'''

I don't know what to do about that.

[dybber]

Hi @the-stanely

Here's how grab a specific branch with git :

• First clone the repository: git clone https://github.com/dybber/mu.git
• Change directory to the newly created: cd mu
• Then change to the port_selection branch: git checkout feature/port_selection

If you then want to update (e.g. if commit another patch to Github) you just call git pull, standing within the mu directory, and git would pull those changes down to your local machine.

As I'm not on Windows, I'm unfortunately not able to help with the build error you are getting, I'm sorry.

My guess is that the same error would happen without my changes, if you want to test that you could go back to the main-branch of mu. With git you would do something like git checkout master or git checkout origin/master to swap back to the main mu-branch without my changes. Then try to run "make win64" again.

[the-stanely]

Hi @dybber,

Thanks, I was able to get the 'port_selection' branch.

[the-stanely]

Tested this port selection feature with another manufacturer's version of the ESP32 -- DevKitC. It also identified itself as CP210x. Works great!

[dybber]

Thanks, good that it works. I would love to get some feedback on the user-interface.

For those who haven't tried it, here's how it looks like if you just connect one device:

When multiple devices are connected:

Some questions:

• Most users, will probably only connect one device at a time, should it be shown different for one device (i.e. no dropdown) than with multiple devices? Any ideas of how best to do it? If in the regular Python mode, I think it should still be shown that a micro:bit is connected, so it should still be an indicator separate from the mode-indicator.
• It seems a bit weird both to have "BBC micro:bit" written in the dropdown and as the mode-indicator. Should the mode-indicator perhaps say "BCC micro:bit mode", rather than just "BBC micro:bit"? Can we do it any simpler?
• When a user changes between devices (e.g. from micro:bit to an ESP-device), I guess it should show a prompt to change mode to the appropriate mode (if they are of different type)... any other such prompts necessary?

[dybber]

Perhaps relevant for @ntoll to chip in regarding the UI

[ZanderBrown]

I would expect this to be narrowing down the mode

microbit mode -> _this_ microbit

In which case I wouldn't expect the dropdown when only one device (for the current mode) is found

Of course you would still get

[micro:bit 2] [microbit mode]

With multiple microbits

[dybber]

I guess it would still be nice if it was shown that a micro:bit was connected, when not in the micro:bit mode, or what?

An alternative could be using icons:

(I'm using Github emojis here, but we should perhaps find some better symbols)

In micro:bit mode:

• When no micro:bit is connected it shows: :x: [micro:bit mode]
• When 1 micro:bit is connected it shows: :electric_plug: [micro:bit mode]
• When 2+ devices are connected [dropdown] [micro:bit mode]

Not in micro:bit mode:

• When no micro:bit is connected: [other mode name]
• When 1 micro:bit is connected it shows: :electric_plug: [other mode name]
• When 2+ devices are connected [dropdown] [other mode name]

[dybber]

But then how do you see that a device is an ESP-device, and you are currently in the micro:bit mode? (That the device is incompatible with the current mode)

[ZanderBrown]

You don't, Mu should have already offered to switch (and hopefully you know it's ESP vs microbit anyway)

[dybber]

I've just pushed some updates. How does this look?

In Python mode, no change:

In micro:bit mode, without any connected device:

In micro:bit mode, with a connected device:

With multiple devices connected:

[the-stanely]

Hi @dybber ,

I looked at it in ESP32 and Python 3 modes. I don't have any other devices.

1) It looks good for the ESP32. Selecting a different port changes the selection, but port isn't changed until after I close & re-open REPL. I'm not sure, but I thought the prior version changed ports as soon as the selection was made.

2) The REPL panel does not follow the edit panel's theme.

Those were the only two unexpected things I saw.

[the-stanely]

There were some errors thrown and a crash. Two apparent separate things...

DirectWrite: CreateFontFaceFromHDC() failed (Indicates an error in an input file such as a font file.) for QFontDef(Family="Fixedsys", stylename=Regular, pointsize=14.25, pixelsize=15, styleHint=5, weight=50, stretch=100, hintingPreference=0) LOGFONT("Fixedsys", lfWidth=0, lfHeight=-15) dpi=96
DirectWrite: CreateFontFaceFromHDC() failed (Indicates an error in an input file such as a font file.) for QFontDef(Family="Modern", stylename=Regular, pointsize=14.25, pixelsize=19, styleHint=5, weight=50, stretch=100, hintingPreference=0) LOGFONT("Modern", lfWidth=0, lfHeight=-19) dpi=96
DirectWrite: CreateFontFaceFromHDC() failed (Indicates an error in an input file such as a font file.) for QFontDef(Family="MS Serif", stylename=Regular, pointsize=14.25, pixelsize=19, styleHint=5, weight=50, stretch=100, hintingPreference=0) LOGFONT("MS Serif", lfWidth=0, lfHeight=-19) dpi=96

The above happened when I was switching ports and running code in ESP32 mode. And then there was this crash...

Traceback (most recent call last):
  File "C:\Users\Stan (Ely) K\mu_editor\mu\mu\modes\base.py", line 178, in <lambda>
    remaining_task = lambda commands=remainder: self.execute(commands)
  File "C:\Users\Stan (Ely) K\mu_editor\mu\mu\modes\base.py", line 176, in execute
    self.write(command)
  File "C:\Users\Stan (Ely) K\mu_editor\mu\mu\modes\base.py", line 162, in write
    self.serial.write(data)
AttributeError: 'NoneType' object has no attribute 'write'

This happened when I switched from Python 3 to ESP32 and then ran code and switched ports.

[ZanderBrown]

The REPL panel does not follow the edit panel's theme.

That's a separate, and known, issue

[dybber]

@the-stanely: With my most recent commit, the REPL/Plotter/Files panes reconnects to the new device on device change.

[dybber]

Could you check if the error you mentioned happing when switching from Python 3 to ESP32 and running code in the REPL/switching ports still happens?

[the-stanely]

I can confirm that selecting a different COM device changes the port selection immediately. It is no longer necessary to restart REPL to make that happen.

Switching from Python 3 to ESP32 did not cause the exception crash.

Thank you. Where should I ask about how to fix the make executable error?

[dybber]

Great! I've just pushed test-cases for the rest of the code, I think that might be the road block for making the executable (everything needs to be tested & pass before the Makefile allows to generate executables)

I will make a pull request now, and request further feedback/input through that channel.

[dybber]

The PR have now been merged into Mu, and will be part of next Mu release. We are working towards Mu 1.1.

I'm thus closing this issue.

[the-stanely]

Thank you!