Godot: Centralized project folder scans

[willnationsdev]

Issue description:
I feel that Godot needs a centralized space in which it iterates over the project's files and then allows other systems to design their own caches, without having to re-implement the project file iteration process (or waste time re-doing it in the first place).

In the Inheritance Dock plugin I built, as well as in @xDGameStudios recent EditorGroupSettings PR #24262, we've had situations where we needed to become aware of when changes to the filesystem have occurred, and then iterate through all of the files, check them (somehow), and if they meet a condition, then cache them or do something with them. For xDGameStudios, it was checking if it was a scene. For me, it was checking if it was a scene, resource, or script. There are probably other systems in Godot that similarly need to react to changes to a project and either re-collect data or apply changes to incoming/modified files.

As far as I know, the EditorFileSystem class has particular information that it tracks for the editor as a whole on each file. However, if there is a subsystem that needs to keep track of custom data about each file or perhaps a subset of files, then there is no need to store this content in the EditorFileSystem's cache. It would be useful to be able to hijack the same file iteration process to let other plugins and subsystems know when changes have happened, evaluate each file, and update their private data.

Suggestion:

1. Add a HashMap to the EditorFileSystem, binding a cache name to a void-returning, single-String-accepting callback: void (*)(const String &). Could be implemented as a Funcref.
2. Before every scan, compile a list of the callbacks and call each of them on the scanned filepath.
3. Expose methods to the scripting API (and obviously, other parts of the engine) that allow them to register and delete these callbacks.

P.S. while checking out EditorFileSystem, I also see references to a sources_changed property that doesn't appear to actually get mutated anywhere, aside from being clear()'d. We can probably factor that out.

[xDGameStudios]

I couldn't agree more since I also add this problem... in my case the data I wanted was already in the EditorFileSystem cache, I just needed to go through the data and check the type, but for more complex data it gets very difficult...

Imagine an attribute system like in C#... and I created a plugin that would add a new attribute, I would want to access information regarding that specific attribute and it shouldn't be the EditorFileSystem responsibility to have a cache with all the attributes, I would just register to the EditorFileSystem scan. Hope I'm not giving a bad example, here.

[girng]

what's wrong with just scanning a specific directory?

[xDGameStudios]

what's wrong with just scanning a specific directory?

The problem is not scanning a specific directory but scanning the whole project... and the ideia was to "enjoy the ride" of the EditorFileSystem and gather the information we want as the EditorFileSystem gathers its.

It is not very performant to have the EditorFileSystem scan all the files (as it already does), then for example you have a plugin that does the same thing to gather some other information and then yet another plugin requires that too, if you look at this you could scan the entire project just once.

Plugins would register to the scan, and have a callback that would be executed on every file... and gather the information they might need.