Mbed-cli: mbed update not updating (with a mercurial repository)

[RobMeades]

Apologies if this is a stupid question but I've just begun using mbed cli with a Mercurial repository and I can't get mbed update to do any updates. mbed update instead behaves like mbed deploy, fetching the version of the library specified in the .lib file.

What am I doing wrong? This is with mbed cli version 1.1.1.

[bridadan]

@RobMeades Can you tell me what exactly you're trying to accomplish? Are you trying to update a specific library inside a project (mbed-os for instance)?

[RobMeades]

I'm cloning something like this:

https://developer.mbed.org/teams/ublox/code/example-ublox-at-cellular-interface/

...then running mbed update in there and expecting to get the latest version of the library that it brings in via ublox-at-cellular-interface.lib. So, for instance, the latest revision of ublox-at-cellular-interface is #f6706249d076, so I would expect that version to arrive, but instead I get #c89190eda594, which is the revision referred to in the ublox-at-cellular-interface.lib file.

[bridadan]

@RobMeades You need to be in the ublox-at-cellular-interface directory when you run mbed update. Otherwise mbed-cli will try and update the project, at which point it will just make sure all of the libraries are at the correct pinned versions (hence the mbed deploy behavior you're seeing).

[RobMeades]

But then how do I update the top level project? Won't that downgrade the libraries again? Is there no way to ask mbed cli just to recurse through a build setting everything to latest (which I what the help text for mbed upgrade implies it does)? Or maybe a way to code the contents of the .lib file to mean "at least this version"? At the moment the only way around this would seem to have frequent check-ins entitled "update library link".

[bridadan]

@RobMeades Yeah this can get a little confusing. There isn't a way to currently recurse through all the folders and update each lib to the latest. Due to the nature of how libraries can change significantly between releases (in an incompatible way), I think this capability isn't present on purpose.

I would suggest doing the following:

cd <project directory>
mbed update                # Get the project to the latest committed version on the current branch

Then for each library, do the following:

cd <library directory>
mbed update                # Get the library to the latest committed version on the current branch

When you're happy with all the library versions, you can commit the *.lib file changes to each library or project.

[RobMeades]

So what is the difference between mbed deploy and mbed update? The confusion is that in all other systems I've come across having up at the front (update, upgrade) implies that it moves something "up". deploy says that it will "Find and add missing libraries", update will "Update to branch, tag, revision or "latest". If update doesn't update to latest it is quite confusing to say "latest" in there. I think I'm gonna add a feature request to give us mbed upgrade or mbed update -f or some such, 'cos it's a complete pain at the moment if you are developing a set of libraries and have to keep jumping directory just to keep things up to date.

[bridadan]

@RobMeades I understand what you're saying, it is kind of a pain to keep jumping directories. Its a bit tough though because blindly hitting "update everything" can sometimes lead to unintended consequences and broken code. That being said, I'm not 100% against adding the feature. Just needs a big warning on it 😄

To help further differentiate mbed deploy and mbed update, mbed update is actually a "superset" of mbed deploy.

mbed deploy simply looks at every .lib file it sees and makes sure the equivalent cloned repository matches the commit hash that's stored in the .lib file. If it can't find the cloned repo, it clones. If there is an orphan cloned repository with no matching .lib, it will remove the cloned repo.

mbed update will first do a git pull essentially for the current working repository (could be your entire project or library inside the project), then it will do an mbed deploy. Does that make sense? The thing that gets tricky is when you're doing an mbed update inside your own project (not a library within). It will not update any of the libraries unless there are upstream changes to the .lib files comitted in your project.

If that above paragraph still doesn't really make sense, I may need to do a write up/diagram on this. Believe me when I say you are not the first person to have trouble with this (myself included 😛)

[RobMeades]

OK, I think I've got it, but it is really subtle and, I think, not something that a user is likely to understand. In general, with mbed, there seems to be an effort to prevent the user messing up (viz the separate levels of config control for target, library and application) but the result of that is a loss of flexibility (e.g. I would like to specify different macro[] entries for different targets in mbed_app.json, but that's not one of the things that you are permitted to do, at least as far as I can work out) which makes it less attractive to more confident developers.

Anyway, I will make a separate enhancement request and see how that goes.

[RobMeades]

See enhancement request #511.

[bridadan]

@RobMeades You've hit the nail on the head, its always a constant struggle between flexibility and ease of use. If you'd like to have more control of your program, you can always export to something like a Makefile so you can tweak everything you like. We do our best to keep the support for the exporters working in the default configuration, but once you go off that we obviously can't make any guarantees.

e.g. I would like to specify different macro[] entries for different targets in mbed_app.json, but that's not one of the things that you are permitted to do, at least as far as I can work out

I don't think this is the case actually. You can indeed set different macros for different targets! Please see the documentation section here: https://docs.mbed.com/docs/mbed-os-handbook/en/latest/advanced/config_system/#overriding-cumulative-target-attributes

And for an example mbed_app.json:

{
    "target_overrides": {
        "K64F": {
            "target.macros_remove": ["BAD_MACRO"],
            "target.macros_add": ["GOOD_MACRO", "MACRO_WITH_VAL=\"Added!\""]
        }
    }
}

If you're looking to override a macro value, it gets a little tricky. At that point it'd probably be best to add that as config option. You can always override the default macro name if you need to.

[RobMeades]

Ooo, macros_remove/macros_add, I can use that, I must RTFM a bit more. Of course, if you could just put marcos anywhere and it would just sort things out that would be even more natural :-).

[theotherjimmy]

@RobMeades config can feel a bit more ergonomic for that sort of thing.

[RobMeades]

True, but I have two targets, one of which has TRNG and one doesn't, and, when building mbedClient, it's a real pain to have to have to edit mbed_app.json and cut/paste:

       "MBEDTLS_NO_DEFAULT_ENTROPY_SOURCES",
       "MBEDTLS_TEST_NULL_ENTROPY"

...into config all the time, especially when bl!%dy json doesn't support commenting things out so I really do have to delete the lines.

[theotherjimmy]

@RobMeades Is that a feature request for yaml config :smile: ?

[bridadan]

Heh, that case is a bit interesting since we highly discourage the use of mbed Client without hardware entropy since it is inherently insecure. Obviously you don't have much choice in this case, but it shouldn't be used in production deployments.

[RobMeades]

@theotherjimmy : I wouldn't dare :-).
@bridadan : yes, I read the warning in the header file when it pops out every single time I compile any mbed client source file :-).