Circuitpython: merge nrf52840 usbboot PCA10056 and feather_nrf52840

[ladyada]

hiya @hathach we started getting some modules so we can have some beta feather nrf52840 soon. since the pinout is different than the PCA (dev kit) please copy/duplicate the current nrf52840 board so we have both PCA10056 and also a feather_nrf52840 (you can keep same pinout for now on that and ill let you know once the pin definition is set for it!)

[arturo182]

I thought we already had separate pin files for the DK and the feather:
https://github.com/adafruit/circuitpython/blob/master/ports/nrf/boards/pca10056/pins.csv
https://github.com/adafruit/circuitpython/blob/master/ports/nrf/boards/feather52840/pins.csv
?

[ladyada]

oh yeah! what i meant was to make the usbboot keep both up to date if possible. i was following the instructions and using a DK :)

[hathach]

To prevent confusion, maybe we should adding a macro inside the feather52840/mpconfigboard.h to switch the pin map, the board name is still feather52840. Once we got the production board, we will remove the pin map of pca10056 (inside feather52840/mpconfigboard.h)

[arturo182]

Sounds like the least invasive way to do it, I like it :)

[dhalbert]

I would say make it two boards so that we can compile for either even when the feather shows up, just by doing BOARD=. We did something similar when we had two revs of the Metro M4 for a while with different pin mappings.

[ladyada]

yah - i think we will have people using both the pca DK and the feather52840 so having two board definitions is good.

for example, i plan on putting together the feather PCB this week but will still be using the DK for testing shields since it is arduino-shape and has JLink built in.

so having both is good :)

here's the rev B Feather52840 pinout:

[hathach]

so we will have feather52840 and feather10056 !!! Please pick an name for feather code on dk. I am really bad at naming ;(

[dhalbert]

you can call it pca10056: that's the Nordic Semiconductor name for this board

[hathach]

@dhalbert ah thanks, I was thinking to leave the current pca10056 board as it is since I am not sure if it is used by others.

[ladyada]

haha - i think the only people using it are here in this thread so it is no problem :)

how about we keep the names but just make sure that the pinout for pca10056 board is only for pca10056 and then feather52 will match the schematic above? sorry it is so confusing but now is a good time to fix it! :)

[hathach]

Ah yeah, that is much clearer. Will do it asap.

[ladyada]

thank you :)

[hathach]

@ladyada I making bootloader hex for feather52840 (accoding to your schematic). Could you tell me which pin to force DFU and which pin for FactoryReset

[ladyada]

OK please use P0.15 for DFU (connected to user switch)

i dont have factory reset connected yet so please use P0.19 for now

[hathach]

just push the update, pca10056 will behave like feather52840 from now. I.e it needs bootloader + sd and can upload via dfu-flash.

I also update the flash (with jlink) target to update bootloader setting (mark app as valid and disable CRC checksum) so that we could flash circuitpython with jlink.

The dfu-related target is also moved from board.mk to common's Makefile since several files uses them now (details here https://github.com/adafruit/circuitpython/blob/nrf52840_usbboot/ports/nrf/Makefile#L285 )

I don't have the prototype, so I will just use the pca10056 target from now on :)

[arturo182]

I know this is on a different branch, so not really a problem, but for now I'm using the pca10056 board files for my own board and I don't want to use the bootloader on it just yet, so I think once we merge we should revert to not force the users to have a bootloader on non-feathers. I think we could maybe make it configurable with make flags like we do with softdevice, so when we build we could just specify make BOOTLOADER=0 or BOOTLOADER=1, we could do that with the feathers too, just for them the default value would be 1.

[hathach]

@arturo182 you could just flash it with flash target as you did currently, you don't have to flash the bootloader If you don't use it. Though I think pca10056 & feather52840 should share the same linker for fat filesystem

[arturo182]

Oh, I thought if the bootloader is present then the firmware is flashed to a different location in the memory, was that not the case? I need to read through the commits in that branch at some point ;)

[hathach]

Ah, The bootloader is placed at the end of the flash, the application address is the same in both cases. However, the bootloader did perform a checksum to decide if the application (circuitpython) is valid. This memwr will set it to 0 ( disable crc ). So yes, you could simply make flash it now, I use it as well, to just speed up the testing here :D

https://github.com/adafruit/circuitpython/blob/nrf52840_usbboot/ports/nrf/Makefile#L259

[arturo182]

Thanks for the explanation, I will try it today then!

For a moment I was confused how does the SD know to execute the bootloader and not the application, but I checked the bootloader's source and I can see when you flash the hex, it also writes the bootloader address to the UICR which is used by the SD to jump before executing the bootloader, feels like magic ;)

[arturo182]

@hathach Hmm, I thought this bootloader was meant to work like the CPX where you normally get a MSC for dropping .py files and then when you reset with double press/similar you get uf2 drive, but I flashed the bootloader and I always get the uf2 drive, I even tried to put P0.11 and P0.12 low/high to simulate buttons not being pressed. Am I doing something wrong or is this not implemented yet?

[hathach]

@arturo182 ah, yes, nice catch, the bootloader address is written to ucir for its detection.

Ah yes it is not implemented or rather implemented in the circuitpython. The bootloader only expose uf2 drive, you need to flash the circuitpython afterwards, by copy uf2 created by uf2 target
https://github.com/adafruit/circuitpython/blob/nrf52840_usbboot/ports/nrf/Makefile#L314

After complete, bootloader will reset and circuitpython will run, you should see REPL via jlink cdc (since native usb cdc is still work in progress). To display msc while in circuitpython, we need to port usb stack to circuitpyhon, which I am working on. Though since I didn't work much with cp, I am still new to it and getting it a bit slower. Once the usb stack is ported to cp, you should has something similar to CPX.

PS: We cannot do double reset trick to enter bootloader with nrf52, Nordic somehow spends lots of effort to clear all the registers unnecessarily when reset. You need to force it by pressing button1 if using pca10056.

[hathach]

Thing to do next would be

• usb cdc to replace uart in circuitpython
• usb msc for storing py files in circuitpython

Though, I am still doing a bit more testing with bootloader in OTA and DFU Serial mode to make sure it is reliable enough.

[arturo182]

Ah I see, so it's the CP app that creates and exposes the MSC for the py files, not the bootloader, good to know. As for the USB stack, couldn't we use the one that ships with the nRF SDK? I have used it before to do composite HID and CDC in another project, it's pretty nice to use. Otherwise, couldn't we create an abstraction layer so that both nrf and atsamd ports could use the same stack? That would make it easier to add new ports in the future and would remove duplicated code. I also worked a bit with the USB middleware in STM32 so maybe I could be of some help.

[hathach]

Ah thank you vẻy much, for usb stack, currently we are using my own usb stack called tinyusb in the bootloader, it got both cdc + msc working already as we see.

https://github.com/hathach/tinyusb

It is not perfect, but work with other chip such as lpc families from NXP as well ( though some update code is needed).

My issue is with the circuitpython rather than usb stack. It is a bit comprehensive for my brain, and hard to debug ( I will try to create an segger studio to deal with it later on). Yeah, I did the initial port, but I forgot most things since then :( .

[tannewt]

@arturo182 I'd suggest you make a new board for yours. It'll be easiest that way.

@hathach I use GDB + JLink and recommend it. (I haven't used it on nRF yet though.) To switch the serial connection to USB you'll need to change this file: https://github.com/adafruit/circuitpython/blob/master/ports/nrf/supervisor/serial.c The SAMD version is here with a helper to buffer from usb.

USB mass storage has two parts. 1) CircuitPython reading the filesystem and 2) USB reading and writing blocks. The corresponding code for the SAMD port is here and here. disk_read and disk_write are MicroPython abstractions that use the APIs from 1 to read and write.

I realize its a lot to dive into and am happy to help and answer questions. Thanks for helping!

[hathach]

@tannewt ah thank you very much, those pointer is helpful.

[microbuilder]

Otherwise, couldn't we create an abstraction layer so that both nrf and atsamd ports could use the same stack? That would make it easier to add new ports in the future and would remove duplicated code. I also worked a bit with the USB middleware in STM32 so maybe I could be of some help.

Hi @arturo182 ... the goal here using tinyusb is actually to get away from vendor specific USB stacks so that we can use ONE liberally licensed and vendor-neutral stack across various MCU targets and peripheral blocks. It's a slower route in the short term, but will make things easier to maintain longer term since we'll have full control over the stack, bug fixes and future direction of the project without having to rely on the good nature of one specific vendor, and people who understand all of the various stacks involved.

[hathach]

@arturo182 usb cdc replace jlink for Serial REPL with 1745e20391a23bf65ff17a0515b803e97fa6970e . I also saw I did a really bad job implementing the uart driver for nrf52 back then. Are you working on nrfx uart driver like those with spi or i2c ? If not, I will try to do it as well, I think we will slowly remove those in hal/hal_* since it is micropython specific hal layer right ?

[arturo182]

@hathach Yes, I'm rewriting the UART stuff in nRFx, I'll let you know when that is done, shouldn't take too long now :) And yes, I have been removing the drivers in hal/* and the target is to get rid of all of them.

[hathach]

Ah thanks, indeed, the first time I work with cp, I don't really know which files are for mp and which are for cp. Things are a bit clearer now :D. I will move on with usb, will merge uart from yours later on. Thanks

[hathach]

@ladyada @arturo182 for the internal flash fs, how many KB do you think is enough, I feel 540KB for application (cp in this case) is too much while 256KB for fs is too small.

28KB is for raw flashing api (e.g bonding ) so that it has the same API with nrf52832 in Arduino library.

[arturo182]

We could probably add 128KB to the FS so then the app would be 412KB and the FS 384KB.

[dhalbert]

The SAMD21 boards without SPI flash have only a 64kB file system, but even that is sufficient for simple programs. Since this is temporary until we have implemented SPI flash, I think 128kB is fine for experimentation. The main use of the the filesystem is for .mpy library files

[arturo182]

I see, I just quickly glanced here https://github.com/adafruit/circuitpython/blob/master/ports/atmel-samd/boards/samd51x20.ld#L8 and saw that those boards have 512KB on internal flash FS.

[dhalbert]

Since CPy is <256KB on the M0 boards, we just split the 1MB of the SAMD51J20 in half when we started development, since we had so much room. But I don't think we ever intended to ship SAMD51 boards that didn't have SPI flash. At some point we may make very tiny SAMD51 boards (eg. Gemma or Trinket size) without SPI flash, but the added cost of SPI flash is very low, and it doesn't take much board real estate, so this is not necessarily likely.

For nrf, the SoftDevice is a big addition above and beyond CPy, and I assume we will need some additional code inside CPy for lower-level Bluetooth support. We may also want to freeze some Python modules into the CircuitPython firmware so that the board is usable even without anything in the filesystem. I am not sure what these will turn out to be but they might support Bluetooth access to the filesystem, basic Bluetooth operations, etc.

Feel free to adjust the internal filesystem size as necessary to accommodate a larger firmware image size or the need for library space. It's all just until we get the SPI flash working anyway.

[ladyada]

yep there may be a trinket or gemma m4 using the BGA vesion of the SAMD51, using the internal filesys - but no solid plans at this time :)

[hathach]

Ok, then we leave it as it is now (256KB for fat fs). I am reading QSPI specs now, and trying to get it work with the 64 Mb flash on the pca10056. Hopefully it is not too different with the qspi flash we use for production.

PS: Anyone know a good tutorial for QSPI, I am new to it :D

[hathach]

Btw, talking about the ble driver. It seems to be the main lacking between cp and arduino. Can we add c++ build to circuitpython, it is easier if we could just use the current ble library from Arduino.

[dhalbert]

@hathach Not quite sure what you mean by the BLE driver. Which Arduino library do you mean (I found several), and other there other examples?

Adding C++ code should not be hard, we just have to make sure the linkage is C and maybe write a wrapper.

[arturo182]

I think @hathach meant https://github.com/adafruit/Adafruit_nRF52_Arduino/tree/master/libraries/Bluefruit52Lib I had a glance at it and not sure if that's the way we would want to go with BLE in CP on nRF, even though that code uses C++, it's not very OOP and might not translate well to Python.

[hathach]

Ah thanks, yeah, that Arduino repo is I am referring to. Indeed, it is not really good well written in oop sense. The gap, gatt central is all considered singleton and most of the code is written with an spining head of mine lol. Though it does some basic work for user convenience. I am not so sure how it could be linked and used with cp either. If it is too troublesome, we may need to redo it for cp.

[arturo182]

That is definitely something I was planning to look into after I was done with the nrfx driver stuff (soon). I wanted to refactor the existing ble driver and work on expanding it and making it more accessible. I know @tannewt was saying that we would like to not create a new API for BLE but instead take inspiration/copy an existing one so users can easily take their BLE code for CP and run it on a RPi or similar. I know the bluepy API that the current driver is trying to emulate (although it's not a 100% faithful port) is far from finished and it uses camel case (see for example https://github.com/IanHarvey/bluepy/blob/master/docs/peripheral.rst) which would make it collide stylistically with the rest of the CP API that uses snake case. So it is still quite undecided and I think we should in the near future have a meeting/discussion on how to proceed. I want to start writing some proper BLE code for my own board in not a long time and I want the API to be in place :)

[hathach]

Ah right, it makes sense to implement a popular ble API in py world ( I don't know any since I am new to python ). So most of the script out there can be reused with minimal modifications.

From my exp with the Arduino lib, we may need to add freeRTOS as well, it is not required but it will make things easier.

[tannewt]

@hathach I recommend getting usb msc and usb cdc using internal flash checked into master before moving onto QSPI. Having usb working makes it easier for others to work on other stuff at the same time.

@arturo182 Lets start with bluepy and evolve it as we see fit.

[hathach]

@tannewt ah ok, I will do a clean up and we can merge it tomorrow.

[tannewt]

Great! Thank you for your work @hathach !

[arturo182]

@hathach Please add me to review, I might have a few comments :)

[hathach]

@tannewt no problem. There is only errands here and there. More people use it, easier to catch bug early. To be honest, I didn't use circuitpython much to fully test it.

@arturo182 yeah sure, feel free to do so, I am very new to cp.

[arturo182]

Should we close this one?

[tannewt]

Yup!