Raspiblitz: Collaboration with cryptoanarchy deb repo?

[Kixunil]

Hi, I'd like to explore the possibility of collaboration with cryptoanarchy deb repository, which approaches the topic of making a Bitcoin full node with accompanying software using Debian packages.

The advantages of Debian packaging

• Automatic updates
• Automatic signature verification
• Dependency resolution solved by APT
• Clearer boundaries around individual components (that means not 100% clear, but I have some ideas to make them even clearer than now)
• Easy installation and integration of new components (as opposed to manually connecting things together or ad-hoc scripts). Just write apt-get install X and X gets installed and configured together with its dependencies automatically.
• Easy, clean uninstallation
• Integrated configuration using debconf and other tooling.
• debconf makes it possible to avoid irrelevant configuration options (e.g. port numbers) without missing the important questions (e.g. "Your disk space is low, select another location for Bitcoin timechain data")
• Various linting tools making sure the packages are less likely to break
• Detection of conflicting files (unless the package specifies Replaces, obviously)
• Time-tested approach to software deployment
• Active development of the system outside of Bitcoin circle

The disadvantages of Debian packages

• Additional overhead beyond steps necessary to make things function - e.g. mandatory documentation. I created my own tools to help with this.
• May take longer to understand for new contributors unfamiliar with Debian packaging
• Necessary compliance with somewhat strict rules if the packages should end up in official repositories - this is not mandatory and I currently sidestep this problem by having my own deb repository. It'd be nice to have the packages as official if possible, but it's not my top priority. (Note: the rules are very sensible, they just shoot for a higher quality than I'm currently willing to deliver.)

Current status of my project

Cryptoanarchy deb repo builder is currently experimental and so I wouldn't advise to integrate it into raspiblitz at this moment. This issue is meant to open the discussion about sharing our resources in order to create a better system, hopefully sooner.

Things that might be relevant/interesting for raspiblitz, in no particular order:

• Electrs fails to cross-compile, didn't try normal compilation on RPi yet. Compiles fine on x64
• The project adheres to Linux Filesystem Hierarchy Standard and seeks to adhere to other relevant standards (if not, it's considered a bug, unless it has to break the standard for some very good technical reason)
• The project already separates all relevant processes under different uids.
• The project uses btc-rpc-proxy to make permissions even more granular. This is a notable exception to the above because there's zero security benefit running it under a different user than bitcoin-mainnet
• The project strictly separates mainnet from testnet/signet, although currently by virtue of non-mainnet instances not existing (:D), the plan is to support it natively, under their respective users.
• All packages use debconf as the primary source of configuration - this auto-generates configuration UIs from provided templates, although many questions have sensible defaults.
• The packages connect to each other intelligently (assuming the user doesn't manually bypass debconf). Example: btc-rpc-proxy is automatically configured to use correct RPC port of bitcoind and it's automatically, correctly reconfigured when it's changed using dpkg-reconfiure. Making this work was a bit tedious, so there are plans to make it even cleaner.
• Appropriate services are restarted automatically when upgraded (though there is a not too harmful bug with unrelated services being restarted)
• bitcoind can be reconfigured to use a different directory than /var/lib/bitcoin-mainnet
• systemd is used for managing all services
• No usage of Docker - it'd go against the goals of making integrated system (Docker seems to strive to make disintegrated systems and doesn't play with deb packaging well)
• btcpayserver uses dbconfig (not to be confused with debconf) to automatically, correctly configure the database
• because of lack of Docker, btcpayserver uses what they call "manual deployment" which isn't supported much. I plan to open a discussion about it with btcpay devs to figure out how to improve the situation.
• btcpayserver isn't currently automatically integrated with nginx - this is the task being worked on right now
• There is a special configuration for cases where a package depends on other package being configured with some specific settings. The most notable example is electrs requiring the absence of pruning. In this case, there's a special package bitcoin-fullchain-mainnet that's only responsible for configuration - has no code, nor services. The installation of this package turns off pruning. electrs then depends on this package. (Actually, during writing of this I realized, I forgot to add this dependency correctly. :D Will be easy fix soon.)
• All packages intentionally leave user data (bitcoin timechain, LN wallet, btcpay data...) and system users after uninstallation. I wonder how to make this cleaner without risking accidental data loss.

Interesting planned features

• Cleaner, even more declarative packages - easier auditing, might allow adding new semi-trusted repositories that can't compromise whole system during installation
• Deterministic builds
• More Bitcoin/non-Bitcoin packages: RTL, Wasabi RPC server, Open Bazaar...
• Much cleaner ways of connecting various services
• Tor support
• TLS support
• Multiple lightning wallets/btcpayservers behind different Tor addresses - for privacy
• SSO
• Systematic approach to backups - a well-defined interface for packages to register their backupable data and a way to suspend services during backups, if needed (think database transactions).
• Automated tests

What I need to know

We have discussed this briefly at LNConf, so as far as I remember, this is interesting for you. I'd like to know some things to understand how should I continue.

• What features are missing in my project that would be necessary for it to be used within raspiblitz?
• What kind of additional support is expected from me? (editing some existing files? more stable apt repository? ...)
• What kind of support can I get? (what I'm interested in is help with packaging new things, testing, bug reports, brainstorming)

Disclaimer/clarification

Raspberry Pi is currently not my target platform. I have some specific requirements about automation and UX. As much as I like to contribute to Open Source projects, I can't afford to do this kind of contribution without getting anything back. I'm seeking balanced cooperation, where I provide a framework for doing the things described above and support for that framework and get back help with testing and packaging new things. I figured it's better to be open and honest about this. Hope we can find some common ground.

[rootzoll]

I can see that idea and if a trusted community builds around that repo it can be a source to make installs of apps easier and get interesting for a project like RaspiBlitz. @Kixunil keep us updated.

[Kixunil]

What would be the best approach in your opinion? To wait until every piece of Raspiblitz is packaged or to have mixed version first? Mixed version seems to be easier to me, but maybe there's a very good reason to avoid it?

[rootzoll]

Is it possible to start with apps like wasabi/btcpayserver/qpay while keeping bitcoin/lnd like it is for the time beeing? Also can you maybe give on the next Lightning Hackday a short overview how you plan to build up a community review over that repo?

[Kixunil]

Since bitcoin and lnd are dependencies of all others doing it in the opposite direction is not great, but it is technically possible. We could create special packages that contain nothing and provide the necessary features (configuration mainly) and have Provides field. Ideally we would have them as a separate component of the resulting deb repo to avoid accidentally messing up normal installations.

I think the overview is short enough to write it here. :)

• First, I want to have something working to show how user-friendly the packages are
• If people want to trustlessly pick up from that, they could review the build scripts and build it themselves
• I already "cheat" with bitcoind and lnd by downloading already deterministically built binaries (this was originally to cut build time, but it looks like I will keep it due to security benefits)
• According to docs Debian packaging itself is deterministic, it's possible I introduced some quirks, I will check at some point and fix if needed
• I plan to sign the whole thing with codechain
• Publish signed statement about the hashes of all packages
• Anyone who runs the builder should get the same hashes, so they can sign the same statement - multiple signatures -> more confidence
• In far future I might be able to provide financial incentive for reviewing
• There are some issues around debcrafter, that I'd like to solve before all this (mainly to make debcrafter more modular, cleaner and easier to review; possibly also moving the whole building to it, skipping makefiles, which are annoying to work with)

I don't plan to attend Lightning Hackday in person at this moment (this could change, but unlikely, especially due to Corona, I'm surprised it wasn't canceled yet), but I would be interested in remote participation if possible.

[rootzoll]

The Lightning Hackday will be online/remote - so there will be a chance.

Maybe it makes sense to start first a new kind of device to use your repo for - is there a open router or a NAS based on debian for which the repo can be used to easily build a node plugin?

Once there is a good trusted codechain governance we could think about changing the lnd & bitcoind installs.

[Kixunil]

I don't know about existing Debian-based routers/NAS. I did some basic tests on Odroid H2 with freshly installed Debian 10 and it looks good. I will definitely test more. I'd like to finish RTL (working on it right now) before serious testing.