Habitat: [RFC] Patches

[ShalokShalom]

This here is a proposal about how to add integrated patch support, which is in my opinion a bit like an essential part of a package manager today.

1) Add patch options

_patch_files=(URL)
_patch_shasums=()

2) It is possible to add the following code blocks at execution time, so the plan.sh itself stays clear.

To do_download():

  for i in $(seq 0 $((${#_patch_files[@]} - 1))); do
    p="${_patch_files[$i]}"
    download_file $p $(basename $p) ${_patch_shasums[$i]}
  done; unset i p

To do_verify()

  for i in $(seq 0 $((${#_patch_files[@]} - 1))); do
    verify_file $(basename ${_patch_files[$i]}) ${_patch_shasums[$i]}
  done; unset i

And to do_prepare()

  for p in "${_patch_files[@]}"; do
    build_line "Applying patch $(basename $p)"
    patch -p0 -i $HAB_CACHE_SRC_PATH/$(basename $p)
  done

For the full implementation is of course an own set of commands suitable, like pkg_patches and pkg_patches_shasums, I use in this example the names from the bash plan.

If the Callback sections are not specified by the user, we can add those lines in addition to the do_default_ actions.

It is also possible to do that with .desktop files and so on in a similar way and i am really sure, a package manager in 2017 should can do this, since its basic functionality in all the other package managers since decades.

A lot of coders might think, that Habitat is still in an significant more primitive state of its development as for real, when they see that such fundamental features are not present.

And i find them helpful too, since these functionalities reduce the complexity of the code in a patched plan and make it convenient for both, packaging newbies and people who spend years on traditional package managers, to add patches.

They are an essential part in stable distributions and get used a lot there.
A lot of distros today look for alternatives of their outdated packaging software, lets help to make them their decision so easy as possible.

[nellshamrell]

The current thinking of the core maintainers is that this might add in more abstraction than we'd like, we generally prefer to keep things low level in Habitat in its current form. This is an interesting idea, though, and thank you for taking the time to submit an RFC.

[ShalokShalom]

Can you explain the potential negative side effect(s) of it?

It seems like one of the speakers at the hangout even commit, that he is personally biased towards such an implementation.

This here is called computer science and scientific standards include a bit more as "i dont like it", seriously.

I mean, we can switch to Assembler, use a micro kernel and drop all the other things which are considered as an additional "abstraction" layer or higher level.

I see Habitat is using Middleman instead of pure web languages on its homepage and i guess, that not even a single computer in the whole world runs without any abstraction stuff.

The same counts for other useful package managers, they ARE indeed an additional abstraction layer.

Habitat IS an abstraction layer, Habitat is indeed all about a higher level of deployment.

One person means at https://youtu.be/2t-A7qwsV2s?t=11m15s that its about the documentation.

1) I can help with that.
2) Nobody asked me to do that, the issue got simply closed.
3) The current, complicated patch feature is undocumented as well, so what?

What he says in 11:47 then, really takes the cake for me..

First of all:

There are packagers in the world, who simply pack without any programming skills at all.
And they are used to do this with a feature as proposed. They do it for major distributions.

Secondly:

This is simply some kind of discrimination:
It 'works for me' so i dont care if it works for others.

This is one of the key fundamental reasons, why people feel sometimes very rejected by technical people, since it basically means "you have to understand everything i do, otherwise there is no chance for you to come in."

This is also the very most often pronounced attitude i ever faced in 5 years of using open source software.
And then some people are wondering, why such less people spend feedback, money and other energy.

How does the implementation of this feature should prevent anybody from using the old way, as he suggest in 14:00?

To keep the things "low level in Habitat in its current form" includes for me, that they stay unwelcome for humans as they are "in its current form". Is this really, what you like to do?

I think its the sense of technology, to fit to the people. Not vice versa.

P.S: Plus, i really like to understand why my bug reports tend to get closed with such a "get rid of it" attitude or why they get ignored at all, apart from some labels?

[robbkidd]

Speaking as a user, not a maintainer here. I'm against adding more to the plan syntax DSL for patch handling, at least at this time. Having worked in both Chef (a DSL) and Omnibus (a packaging DSL), adding the wrong abstraction early would be painful for both maintainers and users.

That being said, the idea to document some patterns on how to patch within the existing framework is a good one. With some patching patterns documented, packagers can solve their problem either with the documented suggestions or with variations of them that better meet individual needs. It's possible that over some time, the community would learn whether the documented patching steps are sufficient or whether some satisfying patching pattern would emerge that would cry out for abstraction.

[ShalokShalom]

This again, makes a statement as a user, from a personal perspective, without any technical reason behind.
What makes this "the wrong abstraction early"?

In your sense?

[robbkidd]

I sense that the issue is more "we don't know yet that this is the _right_ abstraction." So the humans who have committed to maintaining and supporting the project have opted for the moment to encourage patching patterns within the existing plan syntax.

[eeyun]

@robbkidd is exactly right in this case. Closing this issue now isn't saying we're never going to do this, it's saying: until we see more users asking for an abstraction like this we'd prefer to document how to do it natively and see if that's enough to alleviate any pain. If it becomes clear that we need more abstraction in this regard we'll totally do it, but Chef as a company and many of the individuals on the Habitat team have built DSL's in the past and want to avoid the painful pitfalls we've seen before.

[ShalokShalom]

So, do you know any other binary package manager, which does not ship support for patches?
This abstraction means simply to add default values to the hab-plan-build.sh

All the other things happen exactly like in your bash example, with the exception of one abstraction layer _less_ since the patches get listed in the plan itself instead at an extra file.

This is so super simple, that even i can think about to implement it, while i thought your maintainers are for sure more likely to do this in the way they prefer, yes?

If i did any misconception about the easiness, then please clear me up dude.
I thought i drop a proposal which get discussed on a professional level instead refused by generic arguments.

Is there any flaw in my idea? How can this even create any issue?
I am most likely just not able to see these risks, so if any of you is interested to help me, i really appreciate this.

thanks a lot

[robbkidd]

@ShalokShalom For how many packages do you you need to download and apply patches?

[ShalokShalom]

2000

[robbkidd]

I'm unfamiliar with patch support provided by other "binary package managers." Can you provide some links to information about them?

[ShalokShalom]

https://wiki.archlinux.org/index.php/Patching_in_ABS#Applying_patches

[robbkidd]

How's this for a workaround?

You could create a "library" style Habitat package—let's name itshalokshalom/patch_during_build—that provides a shell script in which the functions _download_patches(), _verify_patches(), and _apply_patches() are defined. They would do essentially what you described in the RFC above. Any of your plans that need patching:

• add the shalokshalom/patch_during_build package in pkg_build_deps
• source it during do_begin()
• set _patch_files=(URL) and _patch_shasums=() at the top of the plan like any other property/variable
• call the appropriate patching function during do_download(), do_verify(), and `do_build()

For example:

pkg_name=awesome_sauce
pkg_origin=shalokshalom
pkg_version=1.2.3
[...]
pkg_build_deps=(shalokshalom/patch_during_build)
_patch_files=(blah blah blah)
_patch_shasums=(sum sum sum)

do_begin() {
  source $(pkg_path_for shalokshalom/patch_during_build)/lib/patcher.sh
}

do_download() {
  do_default_download

  _download_patches
}

do_verify() {
  do_default_verify

  _verify_patches
}

do_build() {
  do_default_build

  _apply_patches
}

[...]

[ShalokShalom]

if think it is also possible to create an if/when to detect if the file is local or in the network, in order to support both?

[robbkidd]

if think it is also possible to create an if/when to detect if the file is local or in the network, in order to support both?

You could put whatever location detection logic you like in your functions.