Meson: add support for copying files from srcdir to builddir

[ignatenkobrain]

sometimes it's necessary to copy some files from srcdir to builddir...

At least in cmake in configure_file() parameter named as COPYONLY.

[nirbheek]

You can do this by using configure_file with a blank configuration_data() object.

[ignatenkobrain]

It's not copy-for-free, it still reads whole file and parses #mesondefine and cetera.

[nirbheek]

We should probably just add a new function called copy_file (or similar) instead of adding parameters to configure_file for optimized copies.

[ignatenkobrain]

@nirbheek I'm not opposed. I just wrote how it's done in CMake.

[nirbheek]

Till this is fixed, the official workaround is:

configure_file(input : 'from.txt',
  output : 'to.txt',
  configuration : configuration_data())

This will read from.txt and copy the contents to to.txt while preserving permissions.

[inigomartinez]

What about adding an option related to the build directory? I have been trying something similar this way:

install_data('test-data.dat', install_dir : get_option('builddir'))

Obviously it didn't work as there is no such option called builddir.

[ignatenkobrain]

configure_file(input : 'test-data.dat',
               output : 'test-data.dat',
               configuration : configuration_data())

[adobni]

Please include an option to copy a subdirectory. That will help to migrate some Doxygen operations that require additional resources like gif, CSS, HTML, etc. files.

[inigomartinez]

I have a similar issue where some images are copied renamed. The case is exactly like this:

The files paths are as follow:

• data/icons/hicolor_apps_16x16.png
• data/icons/hicolor_apps_24x24.png
• data/icons/hicolor_apps_48x48.png
• ...

These files should be installed like this:

• icons/hicolor/16x16/apps/note.png
• icons/hicolor/24x24/apps/note.png
• icons/hicolor/48x48/apps/note.png
• ...

The configuration workaround has two problems here.

1. It will parse the binary file, while it's unnecessary.
2. I'm not able to rename.

Something like copy_file looks like the proper solution, with source file name and destination file name parameters, among other parameters like permissions.

[nirbheek]

For that specific use-case it really looks like you want install_data() to be able to rename the file on install.

[inigomartinez]

I thought about install_data, but as far as I know by looking at meson reference, install_data doesn't have any parameter to specify destination file name. I was just working on it and my current workaround is a custom_target along the cp command (though not the best solution for a couple of reasons).

[tp-m]

There's also issue #1487 regarding installing files with a different name, for what it's worth.

[julianrichen]

I've been working on porting my build script to meson as well and not being able to specific the output location in the builddir is a bit of a problem.

I have a ./data/source.svg that is used to generate the applications icon but at various sizes. The sizes are the default sizes for applications on linux found in <icondir>/hicolor/<icon_size>/apps/<application_name>.png, so 16px, 22px, 24px, 32px, 48px, 256px, and 512px. I've tried both configure_file () and custom_target () but neither seem to work right (for this task).

The issue seems to be that because the <application_name> is the same for 7 icon sizes it puts the target/conf file into <builddir>/data/<application_name>.png for all of the targets. Which is somewhat correct, they are the same name, however they are different image sizes. The name of the file needs to be the same because the install directory specifies the size of the icon.

Atm my "solution" has been to create a bash script called meson_icons_generate.sh which has arguments passed at run for the applications name and icon sizes to be parsed and using inkscape directly exports them into the builddir. While this works, meson doesn't register the icons and rebuilds them every time it's run. Plus this feels very hackish...

./data/meson.build

icon_sizes = ['16', '22', '24', '32', '48', '256', '512']

install_sizes = []
foreach size: icon_sizes
    install_sizes += '-s'
    install_sizes += size
endforeach

meson.add_install_script ('meson_icons_generate.sh', '-n', PACKAGE_ID, install_sizes)

foreach size: icon_sizes
    install_data ('@0@/data/icons/hicolor/@1@x@1@/apps/@[email protected]'.format (meson.build_root (), size, PACKAGE_ID),
                  install_dir: join_paths (icondir, 'hicolor', '@0@x@0@'.format (size), 'apps'))
endforeach

I believe adding an option in configure_file () or custom_target to select the output dir would fix this. Or adding an install_rename argument in configure_file () or custom_target () would also work? Which sort of relates to #1487.

inkscape = find_program ('inkscape')

foreach size: icon_sizes
    custom_target ('@0@_@[email protected]'.format (PACKAGE_ID, size),
                   input: 'source.svg',
                   output: '@0@_@[email protected]'.format (PACKAGE_ID, size),
                   command: [
                       inkscape,
                       '-z',
                       '-w', size,
                       '-h', size,
                       '-e', '@OUTPUT@',
                       '@INPUT@'
                   ],
                   install: true,
                   install_rename: '@[email protected]'.format (PACKAGE_ID),
                   install_dir: join_paths (icondir, 'hicolor', '@0@x@0@'.format (size), 'apps'))
endforeach

---

EDIT: After the fact I found a better way but still feels hack-ish:

icon_sizes = ['16', '22', '24', '32', '48', '256', '512']

mkdir = find_program ('mkdir')
inkscape = find_program ('inkscape')

foreach size: icon_sizes
    run_command (mkdir, '-p', '@0@/data/icons/hicolor/@1@x@1@/apps/'.format (meson.build_root (), size))

    configure_file (input: 'source.svg',
                    output: '@[email protected]'.format (PACKAGE_ID),
                    command: [
                        inkscape,
                        '-z',
                        '-w', size,
                        '-h', size,
                        '-e', '@0@/data/icons/hicolor/@1@x@1@/apps/@[email protected]'.format (meson.build_root (), size, PACKAGE_ID),
                        '@INPUT@'
                    ])


    install_data ('@0@/data/icons/hicolor/@1@x@1@/apps/@[email protected]'.format (meson.build_root (), size, PACKAGE_ID),
                  install_dir: join_paths (icondir, 'hicolor', '@0@x@0@'.format (size), 'apps'))
endforeach

[jpakkane]

The big limitation here is that developers should not attempt to force any sort of structure or layout in the build directory because we can't guarantee any. Different backends want to lay out their contents differently because they have different requirements.

If you do not need the icon files during development, only after the application is installed, the simplest solution is to have a custom install script that generates the files in the install directory.

[julianrichen]

While the icons in this case are not critical so the application can run fine without them. That directory structure may contain icons used inside the application that are needed to convey an action, so not installing them at build can be a testing issue.

Admittedly not many people will generate the icon sizes from a source file at build. So adding a feature in core Meson is probably a waste of time. However, adding an install_rename in the custom_target () or another function that runs a command only if a target file has been changed would be useful for other broader cases. Maybe better to move that discussion to another issue (if it doesn't really exist)?

[gauteh]

I am trying to follow the official work around here (https://github.com/mesonbuild/meson/issues/860#issuecomment-262760875), but this fails with some utf-8 error while trying to parse the png file:

a_p = configure_file (
    input : 'ui/icons/icon_color.png',
    output : 'astroid.png',
    configuration : configuration_data ())

Could not read input file /home/gaute/dev/mail/notm/astroid/ui/icons/icon_color.png: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

I am trying to achieve this: https://github.com/mesonbuild/meson/issues/1487

[nirbheek]

Uh, you're right, that will happen I guess. You can use a custom_target() that uses a Python script to copy the file till #1487 is fixed.

filecopier.py:

import sys
import shutil

shutil.copyfile(sys.argv[1], sys.argv[2])
shutil.copymode(sys.argv[1], sys.argv[2]) # optional

meson.build:

filecopier = find_program('filecopier.py')
a_p = custom_target('astroid_png',
    input : 'ui/icons/icon_color.png',
    output : 'astroid.png',
    command : [filecopier, '@INPUT@', '@OUTPUT@'])

[gauteh]

Can't I just use cp ? or install. It's a bit make'ish.

[nirbheek]

If you don't care about non-unix, sure.

[mojca]

In my case I have a bunch of *.dll files in the source tree which are of critical importance if I want to run ninja test, so I cannot wait until the installation phase to get the files. At the moment I'm manually copying them to the build dir, but this is cumbersome, I'll probably use the suggestion above for now.

(That said, I have no clue where exactly to put them after ninja install; after the binaries are cross-compiled for Windows, the /usr/local/lib path for the final library makes absolutely no sense anyway.)

[mojca]

• The approach with configure_file() doesn't work because of
  'utf-8' codec can't decode byte 0x90 in position 2: invalid start byte
• The approach with install_data('mylib.dll', install_dir : meson.build_root()) doesn't work because this step is only executed during ninja install. I need it sooner than that.

[kusma]

Seems like this was fixed in Meson 0.47...

[DaOnlyOwner]

I have the same issue as mojca, cannot copy the .dll files automatically... install_data would work for me, but I receive a runtime error (ImportError: cannot import name '__file__') using the msi installed version.

[kusma]

@DaOnlyOwner: Try configure_file(..., copy: true)...

[DaOnlyOwner]

configure_file(..., copy : true) works but I still have to '$ ninja install' which results in the mentioned error

[kusma]

@DaOnlyOwner: Right. I have a python script that hardlinks some files into the build directory, registered through meson.add_postconf_script()... Perhaps that'll work for you? I agree that something built in for this would have been great.

[nirbheek]

configure_file also has an install: option.

[DaOnlyOwner]

Oh, that I didn't know. The doc says only install_dir. @kusma I do the same now, thanks ;)

[ggabriel96]

Related to @adobni, how can I specify subdirectories for the output using configure_file(copy: true...)? When just typing them, Meson saidERROR: Output file name must not contain a subdirectory..

[nirbheek]

You can use it from inside the (source) subdirectory that you want to copy the file into.

[thomask77]

For my project, I need to copy all files from a "Fonts" directory to the build directory.

After reading the comments, one solution is:

configure_file(input: 'Fonts/DroidSans.ttf',      output: 'DroidSans.ttf',      copy: true)
configure_file(input: 'Fonts/fa-brands-400.ttf',  output: 'fa-brands-400.ttf',  copy: true)
configure_file(input: 'Fonts/fa-regular-400.ttf', output: 'fa-regular-400.ttf', copy: true)
configure_file(input: 'Fonts/fa-solid-900.ttf',   output: 'fa-solid-900.ttf',   copy: true)

But:

• This will reconfigure every time a font changes
• input must repeat the output name. input: 'Fonts/@OUTPUT@' does not seem work
• All this is overly verbose. I would like to give output a list of files.
• I would really like to use a wildcard to copy all the Fonts. This is done at configuration time, so time/perfomance/etc. should not matter much.

Another solution would be:

font1_tgt = custom_target('font1', input: 'Fonts/DroidSans.ttf',        output: 'DroidSans.ttf',      command: ['cp', '@INPUT@', '@OUTPUT@'])
font2_tgt = custom_target('font2', input: 'Fonts/fa-brands-400.ttf',    output: 'fa-brands-400.ttf',  command: ['cp', '@INPUT@', '@OUTPUT@'])
font3_tgt = custom_target('font3', input: 'Fonts/fa-regular-400.ttf',   output: 'fa-regular-400.ttf', command: ['cp', '@INPUT@', '@OUTPUT@'])
font4_tgt = custom_target('font4', input: 'Fonts/fa-solid-900.ttf',     output: 'fa-solid-900.ttf',   command: ['cp', '@INPUT@', '@OUTPUT@'])

But:

• Each custom_target needs a unique name
• Must be stored in a unique variable. fonts += custom_target(.. does not work
• input must repeat the output name. input: 'Fonts/@OUTPUT@' does not work
• each of font1_tgt, font2_tgt, .. needs to be mentioned as a source in executable()
• Is there a way to define custom rules, so that I don't have to repeat command: cp..?

I'm tempted to do this:

run_command(
  'bash', '-c', 'cp $MESON_SOURCE_ROOT/Fonts/* $MESON_BUILD_ROOT'
)

But this only runs at configure time, and completely defeats the purpose of meson.

Do you have any recommendations?

[tp-m]

You could use an array and foreach? (In combination with set_variable() if needed)

[thomask77]

Ok, let's try this:

fonts = [
   'DroidSans.ttf',
   'fa-brands-400.ttf',
   'fa-regular-400.ttf',
   'fa-solid-900.ttf',
]

counter = 1
foreach f: fonts
    set_variable( 
        'font@0@_tgt'.format(counter),
        custom_target(
            'font@0@'.format(counter),
            input: 'Fonts/@0@'.format(f),
            output: f,
            command: ['cp', '@INPUT@', '@OUTPUT@']
        )
    )
    counter += 1
endforeach

executable(
    'imgui-test',
    [
        sources,
        imgui_src,
        font1_tgt, font2_tgt, font3_tgt, font4_tgt,  
    ],
...

• Even more lines of code than before
• Has all the drawbacks of the manual custom_target() solution listed above
• I don't know how to automatically add font1_tgt, font2_tgt, font3_tgt, font4_tgt in the executable() statement.

[QuLogic]

If these are tests, I would try to rewrite it slightly so that no copying was needed. For example, check an env var and look there, if not set check current directory (or the usual installed location if needed.) Then for the test, you can set the env var to the source directory with meson.current_source_dir().

[thomask77]

Those are TrueType fonts. This is only a small example - in practice, I want to copy several 1000 files (textures, sound effects, etc.) over to the build directory, because the executable needs them in the same directory (respective subdirectories relative to the binary).

Because the files may change, I want ninja to check timestamps and copy them at build time only as needed.

Think of it as a custom compile rule that uses cp instead of gcc.
(which would be very easy to do in plain make, complete with wildcard support..).

Later on, I need a custom compile rule that packages multiple game resources into a pack file.
But during development, they are just unpacked files/dirs alongside the executable.

[jpakkane]

I _highly_ recommend that you add a command line argument to your application that you can use to specify where the resources are. It will save you _so_ much trouble. (an envvar could also work)

I even wrote a blog post about this ages ago.

[kusma]

@jpakkane: There's another complication in the mix, at least in my use-case; some of my data-files are built by meson, and some are stored in the source-tree. Your suggestion would require me to split these into two different data-directories, I think. That seems a bit awkward...