Godot: Godot is randomly unable to write to config files and file cache

[Zylann]

Godot 3.0 beta 1 and master and 2.1.4
Windows 10 64 bits

Every so often, under pretty random occasions, I see error messages in which I see Godot is unable to write to some config file or file cache, things like Unable to write to file <path>, file in use, locked or lacking permissions.

Example here, I got this one 1 second after opening a scene:

Another one with file cache, when I use Ctrl+S:

[CodeWarriorGaming]

Same thing happens to me as well.
3.0.beta.build1

[oliver-ruehl]

This happened to me too, but at one point the Windows 10 Firewall popped up and i gave permissions, since then the problem is not happening for me. Not sure what causes this.

[Zylann]

Also happens in 2.1.4

[AlmightyScientist]

I'll try to give it a check.

EDIT:
This error comes from theses lines :

https://github.com/godotengine/godot/blob/132ba0ed9799db9ed1449fa2f2fc597d4b91612d/drivers/windows/file_access_windows.cpp#L119-L139

IMHO, the problem is we use a String for the path while we use UNICODE paths.
We should use std::wstring there. A lot of people complained about PathFileExists returning false everytime (even if the file exist) just because of that.

I can't test this as I'm on linux, though.

[eon-s]

Sounds like is related to the run project bug, windows is doing something weird when trying to open (or close) files.

[vnen]

String is Unicode already, and c_str() returns a wchar_t*, so I don't see what's the issue there.

[RayKoopa]

Not having this issue, but I'm on Windows 10 x64 1703, not the recent 1709 in case you are.
Tried disabling UAC and check if that stupid thing still dislikes what you do even if you wrote into the AppData directory? I constantly have it turned off.

[noidexe]

I have disabled Thumbs.db file generation and indexing service which are both known to lock files, mostly due to NTFS being so 💥 💥 slow.
I still get the issue using the latest master(as of 14/12/2017).

[reduz]

Does anyone still have this issue? Maybe you ran Godot as administrator and the files got locked.

[Zylann]

Never ran Godot as administrator.

[wtarr]

I tried running the godot 3 beta2 here on windows as both admin and non admin, but still seeing this permissions/lock issue. I tried adding an exclusion to my anti virus (avast) too for the godot app data folder, it didn't seem to help resolve the issue.

[wtarr]

Just to update, when I temporary disabled avast anti virus, the error stopped. When I re-enabled it occurred again. My folder exclusion didn't seem to be enough to stop the file error.

[eon-s]

Can't Godot just try to save again after a second on the first failed attempt? That may work around the aggressive UAC and similar systems.

[CodeWarriorGaming]

@Reduz. So far, I haven't had the problem with v3.0.beta.build2, but I haven't had a lot of coding time with it yet. I'll try to hammer it a bit this week and report back.

FYI, I never had the issue on v2.4 or 3.0.alpha.build1. It only occurred for me on 3.0.beta.build1, and happened very often after changing a script and then attempting to run (saving automatically before running). Often, it would take me to the project screen (no crash, just the project screen would pop up and I'd have to manually close it). After subsequent tries it would finally work.

I've never run Godot as admin, and all versions were used under the same conditions (same drive, security permissions, etc). OS is Win 10 btw.

[paultz54]

Windows 10 vers 1709 - disabling AVG antivirus has so far fixed this for me

[akien-mga]

I guess that each time a file is saved, the antivirus scans it to see if it's a known threat, and if we try to re-save at the same time it throws an error. Windows really makes writing to files a big hassle...

A way to work it around would be to ensure that we don't save files so often. An alternative could be to save first to attempt to save a couple times with wait times in between before really failing. Not sure how other Windows software handles it.

[wtarr]

Just throwing ideas out there, would it be possible to maybe buffer the changes to some sort of queue, have some dedicated publishing thread that will write these out in the background that would test for locks before trying to write? Maybe it does that already, I'm not really familiar with the code.

But maybe before even going down that road, you would have to think about what might have changed to cause this (os?/av?), I don't remember this happening before, I'm seeing the same behaviour now with a stable 2.1 version I had lying about.

The lock seems temporary, I cant see any process attached to the file. I was using Handle (i'm not really experienced with this app btw) but it didn't show anything in the app folder being locked.

https://docs.microsoft.com/en-us/sysinternals/downloads/handle

[Zylann]

I'm pretty sure there is a bug to fix somewhere, not a workaround, because Godot didnt do that before and I never saw any other software have this problem on Windows (antivirus or not).

[reduz]

Maybe if the binaries are signed, the antivirus will not do this? Other than that, I can't think of anything else that can be done.

[reduz]

Will definitely not do a crappy workaround because of an antivirus.

[mhilbrunner]

@akien-mga

I guess that each time a file is saved, the antivirus scans it to see if it's a known threat, and if we try to re-save at the same time it throws an error.

Sounds like files shouldn't be re-written that often or that soon after it was just saved.

[paultz54]

I get a related problem whenever i run a visual studio c# debug session (not godot c# which i haven't tried) ,every time i make a minor change to the source then run, a message pops up from avg saying "wait while we check this unknown exe"
very annoying delay , no errors are thrown. must disable avg to get work done

[vnen]

Might be a problem similar to #11391: some files are written more than once in a very short time. It's possible that Godot is the one locking its own files.

In any case, I'm also against making any hack to work around it. The underlying cause should be solved.

For instance, if the error happens with editor settings often (as shown in one of the OP screenshots), we can investigate what causes it to be saved and if it could happen many times in a row.

[hugoel]

Any solutions? Its a real annoying bug yo.

[hugoel]

lol i had the scene playing, thats why

[eon-s]

Could it be related to that issue with running window not being properly closed sometimes? maybe files from previous running scenes remain open (there may be a windows tool to check current open files).

[Nutriz]

I had this issue too, each time I use Godot, when playing the project

[reduz]

I think I understand what's going on. To avoid your project corrupting, Godot saves files to a temporary location first, then moves it to the final location if all went well.

What most likely happens is that you are using a crappy antivirus which read-locks the recently saved file to scan it, and then Godot cant move it to the new location.

There are two possible solutions to this:

1) You guys stop using a third party antivirus and use Microsoft built-in, which never gave me any problem.
2) I add an option to disable this behavior in Godot. That will make it work fine, but you are prone to losing data if godot or your computer crashes.

Or both, maybe that option should just be added.. then it's up to you.

[reduz]

alternatively, godot could attempt moving the file a few times on windows before failing.

[reduz]

Please try with my changes (if it cant save, it will retry a few times). Alternatively, you can disable godot behavior with this setting, but if a file is saved wrong it will be corrupted forever. Chance is low though.

[Zylann]

I think I understand what's going on. To avoid your project corrupting, Godot saves files to a temporary location first, then moves it to the final location if all went well.

@reduz when was this implemented? As said before, this problem likely started to occur during the 2.x cycle. And if that's the case, would be interesting to prove (there are utilities to find what is locking a file on Windows, so I guess it's possible to find out), though the probability is high given symptoms :p.
However, I tested this scenario in Python but got no problem (even tried to write to a .exe to raise suspicion).

[noidexe]

It seems the 100ms wait and retry fixed it. It happened to me a lot when having GitAhead running in the background(I think it's locking new files for a moment) and sometimes when saving a script or scene while the project was running.
I cloudn't trigger the error after trying for about 20 minutes using build 702e28f2657b3b0f50c1e118caf207c0a438f8f1

[wtarr]

Had a good few tries now at it launching the editor, toggling new editor setting off/on each time, adding/deleting nodes and saving. I don't see the error prompt now when I have the "Safe save on backup an rename" flag toggled off. Using a latest build from source btw.

[Zylann]

@reduz I tried with latest master, and still got the error twice... four...

[reduz]

@Zylann did you try disabling safe save? Do you have antivirus on?

[paultz54]

I wonder what would happen if safe save saved the file with an extension that didn't trigger the antivirus , then moved the file then renamed it.

[ArdaE]

I get these errors even with Avast completely disabled (in addition to exclusions of the folders and all the affected file types). I don't have any other virus software and Windows Defender Antivirus is disabled too, so I'm not sold on the idea that this is virus checker related (unless Avast has a bug and still checks files silently when disabled). Having said that, the new "Safe Save On Backup Then Rename" has fixed the issue for me only after restarting Godot Editor. If that option is to be kept as is, it should have a note next to it that says that it requires a restart of the editor.

@paultz54: The second issue you mention is a different one. It's the normal scan of an unrecognized executable. Simply add an exclusion for any executable you keep recreating (or the folder it is in) in your virus checker.

[Zylann]

@ArdaE do you mean that disabling the safe save worked for you? Do you still get the errors when this option is enabled and antivirus is disabled?

[ArdaE]

@Zylann, yes, disabling safe save solved the issue for me. When "Safe Save On Backup Then Rename" is _enabled_ (default/original behaviour), I get errors no matter the enable/disable state of my antivirus software. When "Safe Save On Backup Then Rename" is _disabled_, I don't get errors no matter the state of my antivirus software. Mind you, Avast is still installed; just in a disabled state. Avast also has a separate "silent mode", so I assume disabled means really disabled.

[eon-s]

Any chances to get the fix on 2.1 too?

[NarayanaWalters]

disabling "Safe Save On Backup Then Rename" fixed it for me as well. is there any risk to having this disabled?

[noidexe]

If it crashes while saving you'll end up with a corrupted scene

On Feb 7, 2018 6:53 PM, "NarayanaWalters" [e-mail] wrote:

disabling "Safe Save On Backup Then Rename" fixed it for me as well. is
there any risk to having this disabled?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/14339#issuecomment-363923473,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgJ7bcwtyxnjAT9hTfxv4Sx52YXKW8Xks5tSht1gaJpZM4Q4pSb
.

[NarayanaWalters]

anything I should/can do about it?

[eon-s]

To prevent game engines from messing up your work is always good to use at least a local version control @NarayanaWalters

[ArdaE]

@reduz Is there a reason why you put the following check inside the rename attempts loop?

    if (rename_error && close_fail_notify) {
        close_fail_notify(save_path);
    }

In my case, the second attempt almost always works. After a quick test, the most I've seen is 2 failures back to back (where ReplaceFileW fails with error 1175 : ERROR_UNABLE_TO_REMOVE_REPLACED), in which case the 3rd attempt works. However, because of the location of the error reporting code, an error is reported regardless, even though one of the attempts succeeds.

If I move the code above to the outside of the rename attempts loop (i.e. just before the line save_path = "", then everything works fine even with safe save enabled, and I haven't seen a single complete failure yet (after some limited but repeated testing for a couple of minutes).

[ArdaE]

Another peculiar thing I've noticed is the non-linear correlation between failures and the delay between retry attempts. With the currently hardcoded 1s delay between attempts (the comment in the code says 100ms, but it really is 1 second if delay_usec takes microseconds as the name implies), I usually get 2-4 failures for one or two files per save attempt (i.e. rename finally works about 2 to 4 seconds after the first failed attempt for each file). With the delay set to 1ms, I still usually get 2-4 failures, with a maximum of 15 repeated failures I've seen so far; i.e. rename usually succeeds about 2-4ms after the first failure (or 15ms in the worst case I've seen) + whatever the processing delay within the loop is. With the delay completely removed, I still usually get 2-4 failures back to back, with a maximum of 12 failures I've been able to get back to back. So what regularly requires 2-4 seconds of delay to successfully rename a file when the delay between attempts is 1 second, resolves itself within about 2-4ms (or 15ms in the worst case I could reproduce) when the delay is set to 1 ms. If the delay is removed completely, the rename operation succeeds much faster on average, limited only by the computer's speed, after a similar number of retries.

So, here's my proposal:

• Move the error reporting code so that it reports an error only if ALL rename attempts fail
• Change the delay between rename attempts to 1 ms
• Increase the max number of rename attempts to 100 (a safe, large number that is insignificant if there's only 1ms delay between attempts).

The current 1 second per retry delay, if left in the code, causes unduly long save times without really achieving much in my testing. With the above changes, the save operation is super fast, and I don't get any errors (even with safe save enabled).

P.S. I would push these changes myself, but I'm not set up to do pull requests for Godot yet.

[mhilbrunner]

Should we reopen this and move to 3.1 for discussing the changes suggested by @ArdaE above, or use one of the other issues (https://github.com/godotengine/godot/issues/16274, https://github.com/godotengine/godot/issues/6393)?

Recently ran into this as well, using a laptop I loaned from a colleague - I think it was the installed antivirus, but still, having Godot lock up for a few seconds if the files are not writable becomes annoying really quick.

[profan]

On a related note, this line here:
https://github.com/fire/godot/commit/186e73c0249f50bb20e76c8d09bb21593089d642#diff-fda82590f36bad1a4bc15cabc0b68409R146

Actually waits for 1 second per attempt, not 100msec right now, one zero too many.. Should this be changed?

[mhilbrunner]

@profan Yup seems like a typo, should be fixed. Want to submit a PR? :)

[Akronae]

On windows there is a tiny nice feature that microsoft (<3) cooked us, it blocks certain files coming from other computers without advertising you. You just need to unblock the file like this:

(right click -> properties, on file)

[Zylann]

I got a new computer and noticed that when I save any file (in Godot or Sublime Text), it often takes half a second to save it, which is unusually slow. My new computer doesn't have any third-party antivirus installed, only has the built-in Microsoft one. I didn't get the error popup in 3.0.4 though. On a side note, my internet is really slow these days.

[jahd2602]

It is happening often in Windows 10, even if I have "Safe save on backup an rename" flag toggled on or off.

I have only the Windows Defender antivirus enabled. However, it hasn't shown any warning or anything.

[OvermindDL1]

I have only the Windows Defender antivirus enabled.

That's enough to cause the issue.

[ArdaE]

@jahd2602, do you have Avast installed? The first time you disable Avast, it gets fully disabled; however, after that times out or you enable it, disabling Avast again doesn't really work. It reports itself as disabled, Windows reports it as disabled, and it probably doesn't give any virus warnings, but it still keeps checking files in the background, triggering this Godot-Avast incompatibility issue. If you have Avast installed, restart your computer, disable Avast, and try saving in Godot before Avast's disable times out (after that happens, like I said above, there's no fully disabling Avast again regardless of what its UI says and what it reports to Windows).

[jahd2602]

@ArdaE No, I don't have Avast or any other antivirus installed, other than the default Windows Defender. As @OvermindDL1 points out, probably Windows Defender is what causes the error. I saw the 100ms retrial, and maybe what's happening is that it fails at first, shows the error, then it tries again, and finally saves successfully. However, in this case, the pop-up should not appear. It should only appear when, after retrying, it still does not save.

[ArdaE]

@jahd2602, you must be using a stock build of Godot. It looks like someone did implement half of my suggestions from earlier on the master branch, which includes a fix to the location where a write failure is reported. If you know how to build Godot from source, give the master branch a try.

[jahd2602]

Ooops, yes, I am using 3.0.2, I will compile master then.

[mhilbrunner]

@jahd2602 Please ping me to close this if this is fixed on master :)

[jahd2602]

@mhilbrunner I've been using godot from master for 2 hours and the error hasn't appeared. So it is fixed, I think.

[akien-mga]

It's actually supposed to be fixed in 3.0.3 and 3.0.4 too, as #17869 was cherry-picked.

[techtide]

Suffering the same problem again v3.0.4.stable.official.d916135

[JA-Developer]

Hola, yo tengo el mismo problema en 3.0.5, tanto en la versión normal como en la mono

[ArdaE]

@techtide & @JA-Developer, if you know how to build the code, try my complete set of suggestions from above. So far, the two bugs I've highlighted have been fixed (regarding the units of the delay and the location of the error reporting), but the following two suggestions that worked for me haven't been implemented:

• Change the delay between rename attempts to 1 ms
• Increase the max number of rename attempts to 100 (a safe, large number that is insignificant if there's only 1ms delay between attempts)

If these suggestions fix the issue for you without introducing any other issues, I think they should be considered seriously.

P.S. I know the better approach would be for me to just make a pull-request, but I'm not set up for that yet, and I'm too busy with my non-godot projects. Also, when I do find the time to experiment with Godot, it's almost always on my Mac where this issue doesn't exist.

[kennethmundt]

This bug still persists in 3.1.1 stable. Platform: Windows 10 pro version 1809

[rico345100]

Same here. Using 3.1.1 and still happens.

[AmagicalFishy]

I'm using 3.2 and just ran into this problem. But, I'm not on a Windows machine and have no anti-virus running.

[WolfGangsta]

From Windows 10: the problem (or something very nearly like it) still exists in 3.2.1. Godot can't save any files or write new ones, making it impossible to use without running it in Admin mode!

Edit: As for antivirus, all I have is Windows Defender.

[Calinou]

@WolfGangsta I reopened #6393, please continue the discussion there.