Zephyr: Command-line Zephyr meta-tool

[carlescufi]

This issue covers a meta-tool that would run from the command-line in order to build, flash, run, simulate, integrate, etc Zephyr-based images.

Motivation

West is to be added to the Zephyr project to fulfill two fundamental requirements:

• The ability to work with multiple Git repositories
• The ability to provide a user-friendly command-line interface to the Zephyr
  build system and debug mechanisms

Fundamental design constraints

• Building, flashing and debugging must remain compatible with raw CMake
• One must always be able to drop back to raw command-line (not using the meta-tool directly, although the tool must be installed) and continue working
• The tools will be written in Python, and split into libraries when possible/adequate

Basic scope

• Multiple repository management, including fetching and updating repositories that have known integration with Zephyr, such as MCUboot, OpenThread and others.
• Build, flash, debug, run
• Bootloader integration. Including image signing, image flashing (with offset) and building the bootloader itself
• Advanced emulator support, including potentially networking setup with QEMU

Extended scope

• Multi-core applications (#3066)
• Console
• Firmware update
• Device Management

Scope of the CMake build system (with the help of the tool):

• Building
• Flashing (raw at address 0x0)
• Debugging
• Basic emulator usage

---

Features and Commands to be supported:

• [x] bootstrap
• [x] init
• [x] sync (update repositories from remotes named in manifest) @ulfalizer
• [x] branch
• [x] diff
• [x] status
• [x] build (TBD, no multi-image support yet) @mbolivar

[carlescufi]

Name poll.
Reminder: Zephyr is a western wind.

• washoe explanation
• vayu explanation
• breeze
• wind
• vind
• west
• zeph
• zippy
• zool (Zephyr Tool)
• zmt (Zephyr Meta Tool)
• zp
• zt
• zctl (Zephyr control, eventually we could have zmgr later for management )

[mbolivar]

"zippy"?

$ grep ^z.*p.* /usr/share/dict/american-english 
zap
zap's
zapped
zapper
zapper's
zappers
zapping
zaps
zephyr
zephyr's
zephyrs
zeppelin
zeppelin's
zeppelins
zip
zip's
zipped
zipper
zipper's
zippered
zippering
zippers
zippier
zippiest
zipping
zippy
zips

[jhedberg]

• zool (for Zephyr tool)
• zmt (Zephyr meta-tool)
• zp
• zt (Zephyr Tool)

I think we'd want to keep it short (say, max 4 chars), neither of the shortest two character proposals above were found at least in my Linux distro.

[carlescufi]

I like the idea of a 4-letter name, so I will add a few to the list, including Johan's

[pfalcon]

I think we'd want to keep it short (say, max 4 chars)

I'd think we want to keep it clear for people and non-conflicting with the previous generations of tools which smartly wanted to make it short. Existing zephyr-env.sh takes a good lead in that.

Thus, zephyr-tool seems like an obvious choice.

[henrikbrixandersen]

ztool?

[carlescufi]

@mbolivar @nashif

Here is the roadmap I propose for this tool:

1. Pick a name
2. Start development of the tool in scripts/, documented but not as the default mechanism to build and debug.
3. Add features in this order:
4. Build, including specifying conf file and board
5. Flash, including the different options like masserase or not
6. Debug
7. Decouple the tool at the same time we introduce multiple repositories, by having a repo for the tool itself. This includes adding support for fetching the repositories themselves. At the same time make the tool the default way of building and flashing, since users will already have to deal with it for repo management. Instead of pypi, I think that maybe having it in a repo could be better, so that the repo itself is updated in the manifest. Need to solve bootstrapping though.

[mbolivar]

@carlescufi I'm fine with that if we can come to a resolution quickly on the name.

If the bikeshedding takes more than a week or two, though, I'd like to merge #6832 through the following path:

• call the tool zprog.py instead of zephyr_run.py, as that's hopefully more clear but won't be a name clash with the meta tool
• keep it _undocumented_ for now to have it as an experimental implementation detail

I have also been investigating ways to support on Windows the Unix feature that adds ZEPHYR_BASE/scripts to PATH via zephyr_env.sh.

One solution is to add the following to zephyr_env.cmd:

set PYTHONPATH="%PYTHONPATH%;%ZEPHYR_BASE%\scripts"
DOSKEY zprog.py=py -3 -m zprog

This will let cmd users type zprog.py at the command line and have the Windows Python launcher (available since 3.3) find the right Python 3 and invoke zprog.py as a module.

Using zprog.py here is just an example. We could do this only for the meta tool as another option.

[mbolivar]

@nashif and any other interested parties, any comments on the two above proposals?

[mbolivar]

We've been discussing a bit and have come up with the name west.

This name is:

• short
• hopefully easy to pronounce by a large proportion of the community
• ergonomic to type (at least in my opinion on QWERTY and AZERTY)
• not taken on pip
• evocative of Zephyr as the western wind
• not an offensive term in any language we're aware of (checked informally as well as with http://wordsafety.com/)

The name was proposed to the TSC today and received no objections.

If anybody would like to post a strong objection to the name (with a rationale), please do here!

[mbolivar]

Initial west post is up. Old PR which had introduced zephyr_run.py is now obsolete. Future development will proceed through integration into west.

[carlescufi]

Linking this to #6770

[raveslave]

re:

Bootloader integration. Including image signing, image flashing (with offset) and building the bootloader itself

Good to see this generic tool, things that would be great to see featurewise:

• Flashing over serial and usb (incl. re-flashing the bootloader itself)
• Uploading code directly to (where applicable)
  
  
  
  • RAM, external SPI/NAND/eMMC attached to the CPU
  
  
• Standard way signal the application to jump to the bootloader and vice versa
• Read/Write of "device meta data"
  
  
  
  • i.e. hardware revision & unique serial-no, boot & appl sw version tags, etc.
  
  
• File system interaction (read/write/tree)
• Pluggable serial-protocol on how to talk to the device
  
  
  
  • i.e. when some exotic USB class might be the only way in
  
  
• Consider some way to add project specific commands?
  
  
  
  • RTC clock(s) read/write
  
  
  • raw access to i2c/gpio's and such (usually great for factory testing)