Radare2: Make r2 offline installable

[bannsec]

Is your feature request related to a problem? Please describe.
It is quite painful (even more-so lately with the new dependencies) to install radare2 in an offline environment.

Describe the solution you'd like
r2's installer for some reason utilizes scripted git pulls of specific branches. The better way in my opinion, and offline friendly way, is to utilize git's submodule support. This would both simplify your install scripts (since they wouldn't need to rely on pulling externally) and make r2 offline installable from a simple git clone --recursive

Describe alternatives you've considered
I have created my own jenky scripts that involve sed editing the r2 installer and copying over specific versions of dependency libraries. It always breaks when r2 updates any of it's dependency handling.

Additional context
There is some offline flag in the setup related to capstone i believe, but that only handles capstone and it is not a submodule, meaning you still have to manually clone capstone and move it into the right directory, then use the correct flag. Even with that, the numerous other external dependencies will still fail.

[radare]

we need to apply patches on this cloned repo, submodules are not distribuetd on github tarballs so the release tarballs will be broken, we have been discussing this for years and still github didnt solved the problem..well there are git subtrees, but it's a different monster

On 24 Nov 2019, at 16:37, bannsec [e-mail] wrote:

Is your feature request related to a problem? Please describe.
It is quite painful (even more-so lately with the new dependencies) to install radare2 in an offline environment.

Describe the solution you'd like
r2's installer for some reason utilizes scripted git pulls of specific branches. The better way in my opinion, and offline friendly way, is to utilize git's submodule support. This would both simplify your install scripts (since they wouldn't need to rely on pulling externally) and make r2 offline installable from a simple git clone --recursive

Describe alternatives you've considered
I have created my own jenky scripts that involve sed editing the r2 installer and copying over specific versions of dependency libraries. It always breaks when r2 updates any of it's dependency handling.

Additional context
There is some offline flag in the setup related to capstone i believe, but that only handles capstone and it is not a submodule, meaning you still have to manually clone capstone and move it into the right directory, then use the correct flag. Even with that, the numerous other external dependencies will still fail.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/radareorg/radare2/issues/15497?email_source=notifications&email_token=AAG75FQ7Y3RVIG476AJ5CWLQVKNT5A5CNFSM4JQ7UHBKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H3UJTCQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG75FXE7IXZUDMEDZ6ZDCTQVKNT5ANCNFSM4JQ7UHBA.

[bannsec]

Perhaps there could be a "sys/prepare_offline.sh" or something that would do all the necessary cloning?

[radare]

yeah we need a solution for this and i vote for simplifying it as much as possible

On 24 Nov 2019, at 17:14, bannsec [e-mail] wrote:

Perhaps there could be a "sys/prepare_offline.sh" or something that would do all the necessary cloning?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/radareorg/radare2/issues/15497?email_source=notifications&email_token=AAG75FQO4DFL2H6STBA2VZDQVKR7LA5CNFSM4JQ7UHBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFAO27A#issuecomment-557903228, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG75FXY37KNNJVKQUIIBJLQVKR7LANCNFSM4JQ7UHBA.

[ret2libc]

What about using submodules and just improve radare2-release to generate a tarball with all the dependencies inside?

[radare]

We can do sys/install handle all the git submodules so that will be seamless for the users. And the tarball is generated with “make dist”. Im happy about making this change now

• capstone
• spp
• sdb

Dunno if im missing anything

On 2 Dec 2019, at 08:42, Riccardo Schirone [e-mail] wrote:


What about using submodules and just improve radare2-release to generate a tarball with all the dependencies inside?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

[XVilka]

There is another way to proceed - generate the release tarballs with GitHub Actions:

• https://github.com/marketplace/actions/github-action-publish-binaries
• https://github.com/actions/upload-artifact
• https://github.com/marketplace/actions/github-upload-release-artifacts
• https://github.com/marketplace/actions/create-release
• https://github.com/actions/upload-release-asset

[ret2libc]

We can do sys/install handle all the git submodules so that will be seamless for the users. And the tarball is generated with “make dist”. Im happy about making this change now

• capstone
• spp
• sdb
  Dunno if im missing anything

• tree-sitter
• radare2-shell-parser

[ret2libc]

There is another way to proceed - generate the release tarballs with GitHub Actions:

* https://github.com/marketplace/actions/github-action-publish-binaries

* https://github.com/actions/upload-artifact

* https://github.com/marketplace/actions/github-upload-release-artifacts

* https://github.com/marketplace/actions/create-release

* https://github.com/actions/upload-release-asset

The problem with relying on github actions, I think, is that it makes us more and more dependent on GitHub. It would be harder to switch to something else, if needed. Anyway, I'm ok with any solution to generate the release tarball.

[Maijin]

As already discussed here, Submodules aren't followed for tar.gz but subtree are - https://github.com/radareorg/radare2/issues/11730

[radare]

Yes but now that nobody uses r2 we can break even more things . And using submodules and create a script to pull the things called from sys/install is the way to go imho

On 2 Dec 2019, at 15:25, Maijin [e-mail] wrote:


As already discussed here, Submodules aren't followed for tar.gz but subtree are - #11730

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

[Maijin]

"nobody uses r2" false. But ok I'm fine with submodules too.

[hmht]

Riccardo Schirone writes:

What about using submodules

I'd just like to note that submodules track specific revisions; A commit
to radare2-regressions pretty much requires a commit to radare2 to
update the submodule. As an effect, they don't really play nice with
branches.

This differs from the current system that just grabs whatever is on
master.

Wouldn't it be easiest to allow the user to replace
https://github.com/radareorg/ with /home/user/offline/ or whatever?
Though that wouldn't really simplify the build system.

[radare]

thats false. the current code checkouts a specific commit from capstone

On 9 Dec 2019, at 11:42, hmht [e-mail] wrote:

Riccardo Schirone writes:

What about using submodules

I'd just like to note that submodules track specific revisions; A commit
to radare2-regressions pretty much requires a commit to radare2 to
update the submodule. As an effect, they don't really play nice with
branches.

This differs from the current system that just grabs whatever is on
master.

Wouldn't it be easiest to allow the user to replace
https://github.com/radareorg/ with /home/user/offline/ or whatever?
Though that wouldn't really simplify the build system.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/radareorg/radare2/issues/15497?email_source=notifications&email_token=AAG75FVNAS7VE6VHTFSG6NDQXYOHRA5CNFSM4JQ7UHBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGIVVLQ#issuecomment-563174062, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG75FS77FH3Z6DKEHXOUXDQXYOHRANCNFSM4JQ7UHBA.