Go: embed, cmd/go: add a way to exclude a file from embedding

Created on 1 Nov 2020 · 8Comments · Source: golang/go

Problem: if you embed a directory, it may embed unwanted files from the directory.

Proposal: Comments with //go:embed-exclude apply a filter of files to exclude from the embed.

Example:

Given a directory with static/a.txt and static/.dot:

//go:embed static
//go:embed-exclude .*
var staticFiles embed.FS

will just embed static/a.txt.

Refs. #42321.

NeedsInvestigation

Source

carlmjohnson

👎5

All 8 comments

_Originally posted by @SamWhited in https://github.com/golang/go/issues/42321#issuecomment-720139496_:

I would actually like the ability of exluding specific files. even if we document dot file inclusion it would be good to have a way to ecxlude them, same as excluding something like assets.go would be nice.

I tentatively disagree. It seems to me that if you wanted to exclude certain files you shouldn't have selected those files in the first place using the glob.

I suspect allowing excluding individual files will end up with projects having giant lists of temp files to be excluded, one person excludes the .swp files that they accidentally included, then the next excludes the .DS_Store files their operating system included, then the next person excludes , etc. (rather like how we see projects with huge .gitignore files instead of people putting their specific editors in their user or system wide gitignore file). Instead I'd suggest it's better to not use a glob or configure your build to check for files you don't want.

carlmjohnson on 1 Nov 2020

👎1 👍1

Instead I'd suggest it's better to not use a glob or configure your build to check for files you don't want.

That's a good suggestion, but I'm not sure it will work if someone has a difference between their build server and their testing server. The build server (possibly a laptop) might have extra files that don't exist on the testing server (for sake of argument, cloud CI), so the tests won't detect that the extra files have snuck in.

carlmjohnson on 1 Nov 2020

I'm not sure it will work if someone has a difference between their build server and their testing server

This also seems like a problem that the embed mechanism doesn't need to solve. If I have a difference between the source on my build and testing server I have a problem with my build and testing server, not with my Go code.

EDIT: to put it another way that may make my thinking here clearer: imagine on my testing server I accidentally download different dependencies into my vendor/ tree than I have on my build server. This results in the wrong binary being tested, and a problem crops up in my build that tests didn't catch because of it. Would you expect the vendor feature to solve this problem for you? I'd suggest that you wouldn't, you'd likely use a different tool to fix the problem (eg. a makefile that downloads the correct set of dependencies, Go Modules, etc.). Similarly, if you tell embed to embed all the files, why would you expect it to also provide a way to fix the problems that crop up when you accidentally put files in the wrong place that you didn't want to embed? This is going to be a neverending game of cat and mouse where you exclude something, then your test server accidentally winds up with different files that aren't excluded and you have to exclude those.

SamWhited on 1 Nov 2020

👍1

I tentatively disagree. It seems to me that if you wanted to exclude certain files you shouldn't have selected those files in the first place using the glob.

I would disagree. Due to reasons discussed in https://github.com/golang/go/issues/41191#issuecomment-686621090 I could have assets.go in my dist folder. While I want to include the entire dist folder and not bother with specifying it type by type I'd still want to exclude the assets file.