Straight.el: Define custom :host types

[VojtechStep]

Right now, AFAICT, the domain resolution for different :hosts has a hardcoded dictionary of mappings from a host to a domain. I would like to be able to change this mapping.

I use ssh for interacting with remote git repositories, but since I am a lazy guy, I don't really feel like always typing in the 'github.com' part. So I use my ~/.ssh/config to resolve git@gh as [email protected] with the correct identity, and I do this for multiple hosts.
As a result, all my github remotes are of the form git@gh:user/repo.

I would like to be able to give this information somehow to straight.el.

ATM, I'm using advices on the functions straight-vc-git--{decode,encode}-url and it works well for the most part, except for the very crucial task of installing straight.el itself, since I can't advice functions that aren't defined yet (AFAIK), so during install, straight tries to clone itself via [email protected], which doesn't have an associated ssh identity and fails.

Feel free to close this issue if it's too niche of a use case. I have no problem with setting the default protocol to https for the first time running a fresh install of Emacs, or using any other workaround that I can come up with :smile:.

[raxod502]

This was actually a planned feature that I worked on a while ago, but never finished. I pushed the work in progress to a temporary branch in case you or somebody else would like to finish the feature.

[VojtechStep]

I might take a look at it.

[hristoast]

I'm interested to know what's unfinished for this. I found this issue after realizing I couldn't use a git.sr.ht repo as a source, I'd be willing to try and wrap up whatever work is needed to get this merged.

[VojtechStep]

You can use any git repo, you just have to set :host to nil and then specify the full URL in :repo.

As for integrating custom host symbols, I think @raxod502's linked commit should pretty much work, so I too would like to hear from him why he considers it unfinished.

[VojtechStep]

Looking into it more, it seems that:
1) The spec->url encoding helpers are there, but they are not used in straight-vc-git--encode-url proper
2) The url->spec decoding is not implemented

AFAICT, the decoding is used in two places:
1) Converting Emacsmirror (proper) recipes into straight.el recipes. Emacsmirror proper uses git modules to list packages, so straight retrieves the package definition by looking at the URL. This should not be a problem because a) the default is to use emacsmirror-mirror over Emacsmirror, which uses different recipe resolution, and b) the recipes only specify GitHub URLs (as mentioned here)
2) To identify compatible URLs. I don't know what breaks if this is not implemented, as I am not very familiar with this part of the codebase.
From reading the code, I think if a recipe specifies a URL that is not recognized by one of the patterns, it just returns the full URL and a nil host (protocol is also nil in this case, but that seems like a bug, because the protocol is always identified).
If the result is a literal URL, that means that even inconsistencies like trailing / or a .git suffix would result in a false negative on the match.
However this should not be a problem unless you go to the local repo directory and twiddle with the remotes, expecting straight to recognize that the new remote is compatible with the one specified in the original recipe.

That said, I don't think decoding is required to support custom hosts, but if it were to be implemented, it couldn't be done with the interning trick that was used for encoding, because the decoder doesn't know the host (that's its job), so it can't delegate to an appropriate decoder based on its name alone.

An alternative approach would be to have a customizable list of URL decoders, straight-vc-git-decode-url-functions, that are then ran with run-hook-with-args-until-success to get the first matching result. The downside is that it's inconsistent with the "automagic" approach to feature resolution that is employed in other parts of the codebase.

[hristoast]

You can use any git repo, you just have to set :host to nil and then specify the full URL in :repo.

As for integrating custom host symbols, I think @raxod502's linked commit should pretty much work, so I too would like to hear from him why he considers it unfinished.

Thanks - that wasn't clear to me after reading the docs but it does indeed work.

[raxod502]

that wasn't clear to me after reading the docs

Entirely understandable---is there a place this info could have been placed in the docs so that you would have found it?

[raxod502]

@VojtechStep everything you said in your comment is 100% correct.

I don't think decoding is required to support custom hosts, but if it were to be implemented, it couldn't be done with the interning trick that was used for encoding, because the decoder doesn't know the host (that's its job), so it can't delegate to an appropriate decoder based on its name alone

There is one silly thing we could do:

(let ((syms nil))
  (mapatoms
   (lambda (sym)
     (when (string-match "straight-vc-git-host-.+-encode-url" (symbol-name sym))
       (push sym syms))))
  syms)

It's a bit slow though, and also slightly pains me. Could cache it using the transaction system, but, like, that would be a little ridiculous.

[hristoast]

Entirely understandable---is there a place this info could have been placed in the docs so that you would have found it?

Thank you for asking. I think if we had an explicit example or mention of doing this in the Git backend section that would suffice. I'm happy to put together a PR for that.

[raxod502]

yeah, that would be lovely!