Jormungandr: Consider using libp2p

[michaeljfazio]

Is your feature request related to a problem/context ? Please describe if applicable.
A clear and concise description of what the problem and/or context is.

The current iteration of the V1 test network is suffering from significant issues relating mostly to:

1. Establishing an effective p2p topology; and,
2. Stability of network code (timeouts, peer garbage collection, etc)

Furthermore, Cardano has yet to establish a formalised p2p wire protocol or message set that can be implemented by heterogeneous nodes (Rust, Haskell, etc).

Describe the solution you'd like
A clear and concise description of what you want to happen. possible alternative solutions

I would like to propose a move towards the use of libp2p.

Reasons:

1. The libp2p project has adopted some very sensible abstractions of p2p networking.
2. Using libp2p could significantly speed up development by allowing us to focus more on core features.
3. Though it is still very much in an early development stage, the Rust implementation of libp2p provides a solid foundation on which we could build (rust documentation).
4. The libp2p project is very actively developed and and seems likely to continue that way.
5. The move towards a standardised p2p framework opens up a great deal of possibilities and may even attract more attention toward the Cardano project.
6. The gossipsub routing layer looks very promising!
7. The protocol negotiation and upgrading process that is fundamental in libp2p will allow for better incremental development.
   ...
   Many more reasons I'm sure...

Additional context
Add any other context about the feature request here.

Some downsides.

1. Lack of Haskell implementation. However, Haskell is one of the languages that is slated for implementation. The Cardano project could contribute significantly here. Furthermore, I can see a "microservice" style approach evolving where Jormungandr could be the "router" upon which formally verified Haskell code could run?

[vincenthz]

sorry to rain on this parade, but this has already been considered many months ago (then libp2p wasn't as advanced also), and while it's probably something that make more sense retrospectively, we don't have the resources or time to actually handle such a move. I think we have also view some of the modules in libp2p with envy, as it's slowly becoming a set of great modular utils.

As to the other implementation, they already wholesale rejected libp2p, preferring their own full custom approach. So even if we moved to it here, there wouldn't be any consensus on the p2p that allow communication (pretending that would be the only problem).

[michaeljfazio]

No worries @vincenthz i get where you’re coming from (hands being tied and all).

I hope there were good reasons for a wholesale rejection 🙃

Just so it’s absolutely clear however. Using libp2p would not have necessarily meant abandoning PolderCast. We could write a Network Behaviour module for PolderCast in libp2p so as to reuse a lot of the nice abstractions without reimplementing the entire wheel.

I’d only try and convince “they” to accept the most basic of wire protocol features such as protocol negotiation for streamlined upgrades in the future.

👍

[ilap]

As to the other implementation, they already wholesale rejected libp2p, preferring their own full custom approach.

That's sad, very sad, as imo libp2p would solve a lot of networking issues e.g. NAT/FW etc. and I love the circuit relays idea combined /w multiaddr and the other cool features too.
I am wondering whether they even dug deep into it or just neglected it completely.

[vincenthz]

@michaeljfazio this is not about poldercast, this is more about the amount of time to use and convert to a foreign framework. In hindsight, it's quite possible we would have tried harder to use libp2p (maybe partially) if we knew what we now know.

We're really close otherwise to have something that hold the water, once we finish the few remaining blunt edges and the network is upgraded, it should be way smoother for everyone (at least on the p2p side)

@ilap agreed, they have lots of cool things in libp2p. hopefully, at least for jormungandr, in the future we'll get to reuse some of their lowest modular stuff at least.