Darling: Questions rergarding Docker integration

[jamesbrink]

So first off I want to thank everyone hacking on this project! Awesome work.

I know the topic of Docker has been discussed and I realize it's redundant to some extent.

I do have a working Docker image here: https://github.com/utensils/docker-darling

Very hacky more or less experimental but I've been able to use it in pipelines with some success. Being a docker image just makes it easier for my particular workflows.

My real question is how complicated would it be to decouple the existing container logic and make this work with docker in a more native fashion. I am thinking more of a fork as I know it's not in alignment with the projects goals. I realize this would need to be a plugin or extension to docker possiblely due to the kernel module (or maybe binfmt_misc with just the module installed could do the trick.

It's something I've long wondered about but I've never done a deep dive on the code to really try to figure this out.

I wanted to get some high level input before attempting something like this on my own as I'm a not terribly profecient with c/c++ at the moment. I'd have no problem taking it on if it's a viable solution

Bonus question, why is there no version tagging? Sometimes I struggle with some builds naturally and have to use trial error to find a working build.

[jamesbrink]

This was meant to be more of a discussion but I'm on mobile and not sure if I can even apply that label

[bugaevc]

My real question is how complicated would it be to decouple the existing container logic and make this work with docker in a more native fashion.

It might be possible, but there are complications:

• we store our chroot contents in PREFIX/libexec/darling as regular files; you're going to have to either convince Docker to use that, or manually make an image from that, or even change our buildsystem to produce an image the way Docker wants it instead
• you cannot directly spawn a process inside the container, it has to be a child of launchd, because it needs to inherit the bootstrap port & stuff; that's what we use shellspawn for. That being said I was looking into making this unnecessary, because you can actually set the bootstrap port post factum, but there are further complications with XPC, and I haven't fully figured those out yet
• there's going to be something like a reverse shellspawn that'll spawn processes on the host at the request of the container; does Docker allow that?
• we rely on being able to access host's ELF libraries
• we're going to switch to another chrooting scheme where we actually keep the host's root as our root and do other tricks to pretend we're chrooting (vchroot), so again Docker would have to support us using the host filesystem like that

[jamesbrink]

Wow thank you @bugaevc this is a very detailed response with some great info. I'm going to dig into each one of threse bullet points you laid out and try to understand each and how this might work.

When you mention things like a reverse shell spawn, I believe items like this could be handled within the container.

What I'm going to likely do is strip down my existing docker image to something more minimal and see if I can make it less hacky and messy. I think from a docker standpoint the overlay mount was one of the more redundant and problematic areas I ran into initially

[jamesbrink]

@bugaevc do you happen to know off hand if there is a fast track method to building the kernel module? I was working on cleaning up both the Docker image I was working on (got it down to a reasonable 700mb uncompressed) as well as the arch linux package.

I also maintain the Arch AUR package for darling, so the kernel module can be installed like so:

yay -S darling-mach-dkms

The problem is that I essentially use the default build instructions, so it will build darling, and then darling lkm. Just curious if you knew off the top of your head a quicker method of getting that done.

Is this an acceptable way to build lkm on it's own

mkdir -p build
cd build
cmake ..
make rtsig_h # also i feel this should be built by lkm target ?
make lkm -j$(getconf _NPROCESSORS_ONLN)

[bugaevc]

Yes, LKM should be buildable with make lkm w/o the rest of Darling.

make rtsig_h # also i feel this should be built by lkm target ?

Sounds like it should be a dependency of lkm, yes.

[ciekawy]

having docker image would be awesome! especially with the Xcode toolchain :)

[0x5e]

if darling can be used in docker just like wine does, that will be really nice :)

[LubosD]

I've just created a repository that builds a Docker image: https://github.com/darlinghq/darling-docker

Unlike yours, it doesn't do the actual compiling - instead, it fetches a released DEB. But it properly starts the container with launchd and all the daemons. I'm pushing the first version to Docker Hub right now.

[daraul]

The million dollar question: will it run xcode, @LubosD?