Atmosphere: Boot2 system module load order / dependencies

[chaoskagami]

Feature Request

Overview:

• Some method by which load order of boot2 system modules (with boot2.flag present) can be controlled to prevent undesirable behavior would be nice.

  Smaller Details:

• This is sort of a dependency resolution problem, I suppose. Some way to specify that a module should only load after another module (if boot2.flag is present) would be enough, since there's nothing too complex to deal with yet. Either way, this would probably require a refactor of how EmbeddedBoot2::Main in pm does this. Right now, modules will load in the order readdir returns which is probably directory entry order on-disk.

  Why would this feature be useful?

  • An example: you use hid-mitm and sys-ftpd. sys-ftpd uses the hid service. Without any method to control the order, the order in which they will load is unpredictable. If hid-mitm loads first, sys-ftpd will (probably) proxy via that, wheras if sys-ftpd loads first, it will (probably) not. As far as I'm concerned, the fact that modules can use services that each other provide means there should be some form of dependency management.

[SciresM]

Have you considered simply ordering the folders alphabetically? 00_, 01_?

[SciresM]

Also, it doesn't seem to me that load order actually helps you reliably? There's not exactly determinism guaranteeing that the modules' main()s run in order.

[SciresM]

Oh, hm, first suggestion is non-viable as it's by title id, right.

[SciresM]

I also don't think I understand the example -- the two modules you listed must be able to run whether or not the other is present, there's no true dependency. What behavior are you actually meaning?

[friedkeenan]

I think what they mean is that ideally hid-mitm should mitm hid for as many processes as possible, but if sys-ftpd (which uses hid for pausing/unpausing) runs first, its calls to hid won't be mitm'ed, and so rebinding keys and third party controllers won't work for it.

[chaoskagami]

Have you considered simply ordering the folders alphabetically? 00_, 01_?

That depends. I don't have the FS code handy (I'm still attempting to suck less at RE, heh), but does it sort entires alphanumerically, or does it just return in-order as stored on disk? If the latter, then it's not ever going to return a predictable order, since the order is going to be dependent on what was last-written, and we all know how Horizon fucks with metadata. I also don't think it's a good idea to trust homebrew authors to order title IDs correctly amongst themselves; that requires a certain level of community organization that's never possible.

Oh, hm, first suggestion is non-viable as it's by title id, right.

Yeah. This is a readdir call run in /atmosphere/titles, all of which must be named by title ID, as far as I'm aware.

I also don't think I understand the example -- the two modules you listed must be able to run whether or not the other is present, there's no true dependency. What behavior are you actually meaning?

Right, there's no true dependency. There is a behavioral difference depending on which starts first (or more specifically, in what order service handles are acquired and mitm'd.)

Rather than true dependencies, It'd be more accurate to say I'm asking for something like how OpenRC (a Linux service manager) handles the after <servicename> specification; it doesn't need <servicename> to run, but if both are to run they must be started in the correct order.

You're also free to close this if you don't want to bother. I'm just not sure what the best way to handle this is, or I might have implemented something myself. I've mostly opened this as food for thought, more than anything. I expect given how useful mitm modules are, we're going to see a lot more of them and this might become important.

[SciresM]

Well, it's not really so much that I'm not willing as that I literally cannot imagine a mechanism for implementing this.

[SciresM]

@chaoskagami could you describe a mechanism by which this might be implemented? Processes don't declare what services they will register ahead of time...

[misson20000]

add a file to each title listing which services need to be deferred for mitm, just like with fsp-srv?

[misson20000]

As in more specifically, as soon as SD comes up, pm searches for these service deferrals and tells SM about them, then launches all the titles in whatever order. That way hid-mitm can say it needs to mitm hid, so SM defers all hid gets after SD bringup until hid-mitm comes up.

[SciresM]

@misson20000 I played with that kind of declaration, it's something you have to be very careful with due to the possibility of circular dependencies. If we think it's preferred I'm not entirely averse.

[chaoskagami]

@misson20000 - Much better solution than my naive boot2 ordering nonsense, since that won't race and guarantees strict ordering of service handle acquisition. I suppose the bug report title isn't really accurate anymore, though. Heh.

[ammaraskar]

Seconding this request, I wanted to make a sysmodule that MitMs usb <-> hid, ended up modifying boot2 main for it.

Here's a possible dumb solution that doesn't involve a dependency graph or something similar. Offload the work to the module author: they simply specify what module they want to be loaded before and after. For example, in my case I wanted to be loaded after usb and before tma. And if it's the case that tma is not loaded after usb (e.g an update changes the boot order) then just refuse to boot the module.

[misson20000]

@ammaraskar The problem is that anything doesn't defer service accesses until after mitm is registered will be prone to race conditions. Your sysmodule may be loaded after USB and before TMA, but you're at the mercy of the scheduler to give you a chance to register mitm before something asks for your service.

[ammaraskar]

aah, yeah that's why I have a WaitForMitm("usb:hs") call in there as well. Would a combination of where you want to be loaded + any optional MitMs to wait for suffice?

[SciresM]

Oh, huh, I should do this in the pm rewrite.