Godot: Listing directories in exported project returns only `*.import` files

[timCopwell]

Godot version:
Godot v3.0.6 Stable x64

OS/device including version:
Windows 8.1

Issue description:

Shortly I tried to scan my res://audio/supersounds directory for every .wav file it had into array, and then play track by picking randomly with randi() % array.size(). But i've met problem where already exported project return empty array. So thinking about it might be problem with .wav, i've added some .png and .ogg into the folder and even removed file type check and it did some progress, it returned only .import type files. That's not what I wanted. In editor everything showed how expected.

Some guy in godot questions topic did reproduction and met the same problem. But he somehow got some .png and other types in output. Being surprised I also went and created empty project with same function to scan, but added custom .png into res:// and tried to list the thing. This is what've got:

[Node.gd.remap, Node.gdc, Node.tscn, airSpike.png.import, default_env.tres, icon.png, icon.png.import, project.binary]

Apparently default files and main scene are somewhat defended against it!?

After... I went already here to search for answer I thought it might be #25347, but i checked it's reproduction and it works fine, so file_exists() isn't the case.

Im at dead end now.

Steps to reproduce:
Create new scene, add root node, add following script to it:

func _ready():
    print(list_files_in_directory("res://"))

func list_files_in_directory(path):
    var files = []
    var dir = Directory.new()
    dir.open(path)
    dir.list_dir_begin()
    while true:
        var file = dir.get_next()
        if file == "":
            break
        elif not file.begins_with("."):
            files.append(file)
    dir.list_dir_end()
    return files

Export, launch. Executable debug console should print something similar to this:

[Node.gd.remap, Node.gdc, Node.tscn, airSpike.png.import, default_env.tres, icon.png, icon.png.import, project.binary]

Expected:

[Node.gd.remap, Node.gdc, Node.tscn, airSpike.png, airSpike.png.import, default_env.tres, icon.png, icon.png.import, project.binary]

Minimal reproduction project:
exported.zip
project.zip

[bruvzg]

Original import sources are not preserved on export (except some special cases like icon.png), only imported stuff (converted to internal format) in the hidden .import folder is included.

You can still use load / preload these files using original path.

[timCopwell]

Original import sources are not preserved on export (except some special cases like icon.png), only imported stuff (converted to internal format) in the hidden .import folder is included.

You can still use load / preload these files using original path.

Ok, but i don't want to load any files. I need to do what i need to do i.e. get full list of files i have in my directory. Maybe there is another way, if yes then how?

[bruvzg]

Maybe there is another way, if yes then how?

1. Make sure Godot is not running.
2. Create empty .gdignore file in the sub-folder you want to preserve as is

3. Delete all *.import files from this folder if there are any
   

4. Add file extensions to "export non-resource files" in export options
   

Resulting PCK content:

Output: [airSpike.png]

[bruvzg]

Another option: you can try to use this contraption to directly create PCKs from folder without any changes to files or to extract exported PCK, add files and repack it back.

[timCopwell]

I simply can get why this does not works properly. If it works in editor it's intended to work everywhere, else this is either a huge issue or listing function is completely pointless to have in engine. I don't believe in workarounds cause they tend to fail everytime you add minor changes to anything :S

[tempdoom]

Would like to say that this solution fixed the issue of importing for me, now I've got files which export properly without a *.import, however now I'm getting a "no loader" error when I run the program, and I can't use any of the files while running in the editor.

[LinuxUserGD]

Another option is to use the whole project folder and put the executable inside.

[bruvzg]

properly without a *.import, however now I'm getting a "no loader" error

load/preload are looking for xxx.import and associated resources stored in .import hidden folder, it the same way you won't be able to use it to load arbitrary .png file outside your project.
See https://docs.godotengine.org/en/latest/getting_started/workflow/assets/import_process.html for more info.

But you can load it manually (process is different for various file type: for images you should use Image, for sound files load it as file and set AudioStream*.data).

Won't work unless test.png is imported (no test.png in PCK, but test.png.import and .import/test.png.{hash}.stex instead):

    $TextureRect.texture = load("res://test.png")

This will work with any .png file, inside PCK (path with res:\\ prefix), or somewhere in the user filesystem (path with no prefix):

    var image = Image.new()
    image.load("res://test.png")
    var texture = ImageTexture.new()
    texture.create_from_image(image)
    $TextureRect.texture = texture

[bruvzg]

Related issues: #22433, #18367, #17848, #17748

[timCopwell]

Another option is to use the whole project folder and put the executable inside.

Isn't it a kind of absurd to distribute to other users access to all the sources of your game. Yes, the engine is open-source, but let's say you have something related to micro-transactions in your game, or your saving data is encrypthed and the key is hidden within your code. You have to think ahead and outside the box of using your game on your machine only.

[LinuxUserGD]

@timCopwell I have the source code of my game on Github, so it doesn't matter for me.

[bruvzg]

Another option is to use the whole project folder and put the executable inside.

You won't be able to load any changed/added files (or any files at all it you do not distribute .import folder as well).

Isn't it a kind of absurd to distribute to other users access to all the sources of your game.

Export is not much of protection. You can easily decompress PCK, you can convert scenes back to text versions and decompile gdscripts to the state almost identical to source. PCK does not store original files, but you still can restore it from .import folder stuff.

[tempdoom]

load/preload are looking for xxx.import and associated resources stored in .import hidden folder, it the same way you won't be able to use it to load arbitrary .png file outside your project.
See https://docs.godotengine.org/en/latest/getting_started/workflow/assets/import_process.html for more info.

But you can load it manually (process is different for various file type: for images you should use Image, for sound files load it as file and set AudioStream*.data).

Won't work unless test.png is imported (no test.png in PCK, but test.png.import and .import/test.png.{hash}.stex instead):

  $TextureRect.texture = load("res://test.png")

This will work with any .png file, inside PCK (path with res:\\ prefix), or somewhere in the user filesystem (path with no prefix):

  var image = Image.new()
  image.load("res://test.png")
  var texture = ImageTexture.new()
  texture.create_from_image(image)
  $TextureRect.texture = texture

Is there an *.ogg/streamplayer equivalent to the code mentioned above?

[bruvzg]

Is there an *.ogg/streamplayer equivalent to the code mentioned above?

Something like this:

    var ogg_file = File.new()
    ogg_file.open("res://test.ogg", File.READ)
    var ogg_stream = AudioStreamOGGVorbis.new()
    ogg_stream.data = ogg_file.get_buffer(ogg_file.get_len())
    ogg_file.close()
    $StreamPlayer.stream = ogg_stream

[nathanfranke]

Unfortunately bruvzg's workaround has been fixed and no longer works in 3.2 beta 2

Edit: My new workaround is to have a file with the directory list inside of it. You can then use File and get_line()