Lwt: Retiring lwt_camlp4 and lwt_log from the repo

[aantron]

These are deprecated, in favor of the PPX and Logs_lwt, respectively. Once these are factored out into their own opam packages (#370, #484) in 4.0.0, we don't intend to ever maintain them again. At that point, the code will be only an eyesore in the Lwt repo.

So, I want to delete it from the repo. I propose to git-subtree each of these modules out into their own repos, say aantron/lwt_camlp4 and aantron/lwt_log, where they can remain archived forever. If somebody wants to take over them and do some more work, we can then easily hand them over, with a nice and complete history already there in each repo.

[aantron]

Actually, I'd eventually like to also move at least lwt_ssl and lwt_glib from the main repo, without deprecating them. The same goes for lwt_react, but the desire there is a bit weaker. @rgrinberg, I remember you had an objection to this because there would be lots of separate issue trackers. However, these packages really have independent life cycles at this point, I think, and they see practically no issue traffic.

Other than the separate issue trackers and code living in separate repos, there shouldn't be any effect on users. We would change the dev-repo fields in opam for the latest releases. I don't think we would have to even re-release the packages.

Any thoughts?

[rgrinberg]

For repos that you plan on deleting I think this is fine. But for the other repos my opinion is that while I see little risk in doing this, I also see absolutely no gain. All of these libraries still have you as the maintainer, so if there's more maintenance overhead with more repositories. You're only creating overhead for yourself by adding tasks like syncing breaking changes, managing constraints, more CI setups, etc. If you'd like to do this extra work, feel free to do it. Maybe I'm just not putting as much value on separate release cycles as you are.

[Drup]

Yeah, I'm not convinced either. Also, forced synchronised releases is a problem when the big one at the source of the dependency chain moves much slower than the small ones. That's the other way around here. :)

[aantron]

It actually seems like less of a headache to me, most especially for the libraries with extra system dependencies (lwt_ssl, lwt_glib). Splitting off their CI would simplify the CI of both lwt and those libraries. It is currently done conditionally, and it's annoying to deal with and understand. It grinds me down every time I maintain it.

We also aren't doing synchronized releases. Except for the Jbuilder port and initial transition, none of the minor libraries have been released at the same time that Lwt was released, and given my understanding of what those libraries are, I don't see that changing. The simultaneous releases for the Jbuilder port were actually an example of something that shouldn't have been necessary, and could have been done piecemeal, and with less risk, if we had separate repos and separate builds.

The managing constraints is not related to repos, but to different libraries, and these libraries are already separated, and thus require constraints (occasionally).

There aren't really any substantial breaking changes to sync, and I don't foresee that changing either, unless we literally rewrite Lwt or something (and we might :p). But that will be painful regardless of separate repos.

For making sure everything stays consistent, I already do local revdeps builds, plus read the opam CI revdeps logs. This covers all users, and would cover lwt_ssl and lwt_glib as well. Relying on that would warn us about problems later (at release time, instead of every build), but given the rates of development and what realistically can change, I don't see that as a problem, any more so than it is a problem for the rest of the OCaml world that uses Lwt. These are basically entirely separate libraries, and having them in the repo complicates it.

[Drup]

Yeah, it makes sort of sense for glib/ssl, given the extra dependencies and focus. You should make sure that they are tested by lwt's own CI though.

Given that lwt, while not being 18th yet, is now fairly independent and spawning subprojects, I wonder if it would make sense to move to your own independent organization to house everything. It might be a discussion worth having.

[aantron]

I'll give it some thought over the next few days, opinions very welcome.

[aantron]

The Camlp4 syntax is now at https://github.com/aantron/lwt_camlp4, and Lwt_log is at https://github.com/aantron/lwt_log. Lwt_log will get a release concurrent with Lwt 4.0.0, because the code is now packaged directly in lwt_log. Camlp4 doesn't need a release, because the code of Camlp4 was already packaged in lwt_camlp4.