Conan: tools.patch cant create files

[nolange]

To help us debug your issue please explain:

• [x] I've read the CONTRIBUTING guide.
• [x] I've specified the Conan version, operating system version and any tool that can be relevant.
• [x] I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

Hello,

I am trying to apply some out-of-tree git patches, some of them create new files.
Some of them contain new files, and the tools.patch utility will then compail that absolute filepaths are not allowed (/dev/null).

Lacking this support, its cumbersome to add bugfixes and git commits in general.

Example patch (created with git format-patch):

From d0807313143bb35da65c2b858a2d9e17fd3fbf9e Mon Sep 17 00:00:00 2001
From: Norbert Lange <[email protected]>
Date: Fri, 7 Jun 2019 21:49:19 +0200
Subject: [PATCH] add and remove file

---
 newfile | 1 +
 oldfile | 1 -
 2 files changed, 1 insertion(+), 1 deletion(-)
 create mode 100644 newfile
 delete mode 100644 oldfile

diff --git a/newfile b/newfile
new file mode 100644
index 0000000..fdedddf
--- /dev/null
+++ b/newfile
@@ -0,0 +1 @@
+Hello mean world
diff --git a/oldfile b/oldfile
deleted file mode 100644
index 32332e1..0000000
--- a/oldfile
+++ /dev/null
@@ -1 +0,0 @@
-Old litter
-- 
2.20.1

My environment is:

Debian Buster x64
Conan version 1.16.0

[uilianries]

Hi @nolange !

Are you sure about the line feeds (LF vs CR/LF)? Usually this kind of error is related to patches created on Windows with CRLF and applied on Linux where is configured to LF. I had the same problem months ago. You can use _dos2unix_

Conan uses the python package patch which is widely used.

[nolange]

Yeah, I am pretty sure about the linefeeds (No OS or tool around that would produce them). Its an upstream issue then.

Still wont make it less useless to me =/

[uilianries]

Hi @nolange !

I can confirm your scenario, I created a gist which reproduces your issues.

In fact is a limitation from upstream and that issue is from 2015, so I can't they will solve it.

@lasote We have few options here, it's an upstream issue, from patch. We could provide a patch, but sounds dirty, or we could move to another python package with the same proposal.

[memsharded]

Hi all,

Conan has code dedicated to fill this functionality that the patch library is missing:

https://github.com/conan-io/conan/blob/551ddac6425341a861258617a2fbd09faf4a71ba/conans/client/tools/files.py#L218

for p in patchset:
        source = decode_clean(p.source, "a/")
        target = decode_clean(p.target, "b/")
        if "dev/null" in source:
            target = strip_path(target)
            hunks = [s.decode("utf-8") for s in p.hunks[0].text]
            new_file = "".join(hunk[1:] for hunk in hunks)
            save(target, new_file)
        elif "dev/null" in target:
            source = strip_path(source)
            os.unlink(source)
        else:
            items.append(p)

So, if it is not working properly, this could probably be fixed there, there might be some format or path that it is not taking into account. I think that would be easier and less risky than switching to a different patch package.

Still, some preliminary investigation of patch package alternatives would be useful, maybe for conan 2.0, @uilianries if you want to have a look.

[nolange]

@memsharded : I use strip=1 (practically the standard for patch-files), and the code does "strip" before handling the added/removed files.
Might be that /dev/null has been stripped to /null before running into the code you quoted above?

[memsharded]

I see @nolange , thanks for the hint!

This could be fixed by https://github.com/conan-io/conan/pull/5334, the problem was that strip also affects the new output files, and if a barename (without folders) is being used, then it is stripped to empty. If you could check the fix in that branch (run from sources with your real patch), that would be very useful. Thanks!

[nolange]

Sorry, you got some nightly .deb file I could test?

[memsharded]

@nolange, nop, sorry, what we have is pip packages in the PyPI Test repository, but those are generated only when a PR is merged to develop branch. The alternative is running from sources, from the source branch of my fork. If you want/can give it a try and need help to setup, don't hesitate to ask.

Don't worry if too much trouble, I have added a new test, and I think your above case is sufficiently covered by tests, so it should be ok.

[memsharded]

Merged to develop, will be in Conan 1.17 (and as soon as CI finished, in the PyPI test repo). Thanks for the feedback!

[nolange]

This is not fixed with 1.8.2.

Please test this with the attached package, the patch is from a normal git format-patch.
example_package.tar.gz

[uilianries]

Hi @nolange ! Just to be sure, did you mean 1.18.2 ?

[nolange]

Yeah, I meant 1.18.2.

Would it be possible to just use the patch binary on *nix and ship something like Busybox with patch et all for that other OS? This sounds like reinventing a really old stable and battle-tested tool.

[radonish]

The experience I have had (I'm using Conan 1.18.5) is as follows:

• A patch that only creates one or more new files works
• A patch that both creates a file and updates an existing file fails

My workaround was to separate the single patch that was both creating a file and updating an existing file into 2 patches.

[lasote]

https://github.com/conan-io/conan/pull/5761 has been merged to develop and this will be solved for next release Conan 1.20