Godot-proposals: Improve workflow of File and Directory

[nathanfranke]

Describe the project you are working on:
Considering reworking much of the File and Directory workflow for 4.0, and making this issue to brainstorm and discuss what we should change.
Kind of a continuation of https://github.com/godotengine/godot-proposals/issues/1225

Describe the problem or limitation you are having in your project:
Currently, the system for files and directories is quite verbose and takes a lot of boilerplate.

Describe the feature / enhancement and how it helps to overcome the problem or limitation:
I want to take inspiration from Java's File API.

• Each file object must already be connected to a file or directory
• "File" can represent a file or a directory
• Directory contents can be iterated with get_contents()

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:

# Construct the file
# File should be immediately constructed with a path, and flags optionally
var icon = File.new("res://icon.png", File.READ)
print(icon.get_path()) # res://icon.png
print(icon.get_name()) # icon.png

var music_directory = Directory.new("res://Music")
# Idea: get_contents() can return a list or stream of `File`s
# Q: Should there also be a method that returns subfolders? What if the user wants both files and subfolders?
#     I don't think that there's much of a good reason to be honest since they are both different data types.
for song_file in music_directory.get_contents():
    print("Loading song file at " + song_file.get_name())
    # Load could also take a file
    var song = load(song_file)

If this enhancement will not be used often, can it be worked around with a few lines of script?:
Files and Directories are used very often

Is there a reason why this should be core and not an add-on in the asset library?:
File is already part of Godot's core

---

Edit: On second thought, not sure if File and Directory should be compacted to one class, since it is misleading that File.new("res://icon.png").get_contents() compiles. People may even think that get_contents has something to do with the binary part of a file.

[bruvzg]

Idea: get_contents() can return a list or stream of Files

Open each file every time folder is iterated? In this case File constructor should be lazy and do absolutely nothing until its content is accessed, which probably is not the best idea, people expect File constructor to lock files or error out immediately.

[DanielKinsman]

Given gdscript's python inspiration, could be worth looking at their newer pathlib stuff? https://docs.python.org/3.8/library/pathlib.html

[nathanfranke]

@bruvzg the constructor should only fail if the path has an invalid syntax. There should be no fail for "file not found" unless you actually try to read from it.

Why? Because for example:

var file = File.new("res://abc")
if file.exists():
    pass

[jonbonazza]

I agree that the current file apis are cumbersome and Java's file api is a good source for inspiration.

Also, constructors should never throw an error or fail.

[Serenitor]

agreed, especially the boilerplate code for iterating over the contents of a directory is really quite unwieldy:

var dir := Directory.new()
if dir.open("user://saves") == OK:
    dir.list_dir_begin()
    var file_name := dir.get_next()
    while file_name:
        if file_name.begins_with("."):
            file_name = dir.get_next()
            continue

        # do something ...

        continue

My initial thought about improving situations like those at least somewhat would be for GDScript to support assignment within while conditions, but I also like what's been proposed and am in favor of the consideration to combine the two APIs.

[dalexeev]

Definitely need to rework the mode flags. First, it's easy to confuse READ_WRITE and WRITE_READ. Second, if, for example, you need to write data to a log file, you get the following code:

var f = File.new()
var mode = f.READ_WRITE
if !f.file_exists():
    mode = f.WRITE
f.open("user://path", mode)
f.seek_end()

I suggest replacing these flags with READ, WRITE, CREATE, APPEND (or SEEK_END?), TRUNCATE.

iterating over the contents of a directory is really quite unwieldy

I just use the following pattern:

var dir = Directory.new()
dir.open("res://path/")
dir.list_dir_begin(true) # skip_navigational
while true:
    var fname = dir.get_next()
    if !fname: break
    if !fname.match("*.ext"): continue

    ...

Firstly, the entire first file is now processed in a loop, like all the others, and secondly, since all processing is at the beginning of the loop, I can safely use continue further in the loop body if necessary.

var file = File.new("res://abc")
if file.exists():
    pass

In principle, this is also acceptable, only we need a different method, for example file.is_open(), since the error can be of a different type (for example ERR_FILE_NO_PERMISSION). However, this way does not give a gain in the number of lines.

But, in built-in classes, usually .new() takes no arguments. I don’t know what it’s connected with, but it is definitely true.