Wrye-bash: Support esl extension

[Utumno]

See: https://afkmods.com/index.php?/topic/4966-wrye-bash-all-games/&do=findComment&comment=167935

[Sharlikran]

This will be part of SSE as well.

Which branch should I fork off of for Fallout 4, dev?

[Sharlikran]

I am not under NDA yet because the Beta/Early access has not started. I don't know if we will be placed under NDA but that is usually what happens. While your time is limited, you want to get the beta2 ready (as time permits), and you are busy with other refactoring, if you have time to look at my changes I would appreciate it. Because once testing begins, if I am under NDA, then I can't talk about it anymore.

What is needed:

• [x] esl files recognized on mods tab
• [x] esl files recognized on installers tab
• [ ] esl files can't be ESM-Ified (Do not allow flipping of 0x00000200) Reason: This is a special function, you compact the FormIDs, there must be less then 4000 forms, and other requirements. Force the user to use CK so they don't release invalid mods.
• [ ] esl sorted efter .esm files and before .esp files. Reason: They are considered masters, and have Fallout4.esm as a master.

Once the Beta/Early Access starts I will want to test where the in game Mods tab places the files to confirm my current assumptions as to how Bethesda will sort them along with any other tests that come up at that time.

My current branch is: esl-support

[Sharlikran]

SettlementKeywords.zip - FYI Don't use that ever. The mod would work but any SKE mods would have to be updated. Which is why Bethesda warns you about making ESL files. I compacted the FormIDs to test the functionality.

ESL Files are in the Fallout 4 Wiki: Data File

That is a sample ESL file for now if you want to look at it. However, Fallout 4 and xEdit won't load them yet. The flag set in the header will be 0x00000200 for ESL files instead of 0x00000001 for ESM files.

[Sharlikran]

utumno-wip seems to be fine installing and uninstalling esl files. I had to refresh data, and the package a few times, but eventually it worked itself out. If you left it this way it would work for testing. The only thing that would make it better is forcing them to sort after ESM files and before ESP files. After all they are master files so they won't load after ESP files, and you shouldn't load them above their respective masters. According the the Wiki I linked, they are being called Light Master Files.

[Utumno]

Thanks, I wouldn't mess with load order except if really required - are they listed as masters in the esp files ? Then Bash should already warn if they load after their dependencies

[Utumno]

When I say required I mean _does the game load them before all esps_ ?

[Utumno]

I pushed some new commits in utumno-wip with rest of changes here - RuleSets need testing

[Arthmoor]

We'll be able to check this shortly as there is a public Steam beta available now that supports the ESL file format.

[Sharlikran]

I wish Cartogriffi would have told me there would be a Public Beta instead.

[Sharlikran]

I got a Pure Virtual Function Call

[Sharlikran]

After the pure virtual function call, how that happened I don't know, I removed the ESL file from plugins.txt. I went into the in game Mods menu and activated my test ESL file I attached earlier. It put an asterisk in front in plugins.txt so no new symbols. It remained where I placed it, among the other ESM files.

For my second test, I moved it way down among the ESP files where you presumably would never want it. As soon as I confirmed I was done moving it, and clicked Done, the file was moved automatically as if I never moved it at all. It didn't return to where I had it among the ESM files, it placed it immediately after the last ESM flagged file (a false flagged ESP for me), and immediately above the first ESP file.

If you want to confirm that, then you will see it does exactly what I thought it would do. Bethesda's own sorting system will not allow it to be sorted after ESP files. However, it does seem to allow it to be sorted among other ESM files and I will ask Cartogriffi about that.

Bethesda's sorting is wacky sometimes. I don't want to test it but they may not care where it is when it's inactive. Do your testing when it's active.

[Sharlikran]

He said the public beta was already planned. He's going to look into who I can talk to about the sorting.

[Arthmoor]

I haven't been able to check it yet because I'm looking through the updated Fallout4.esm file, but are these ESL files being loaded into the game after the main DLC files? If not then that's going to be an issue, but otherwise if they always load after that the only concern is to make sure they show up higher than things like the UFO4P when sorting a load order, but then that also falls outside Bash's domain.

Has anyone mentioned this stuff to @WrinklyNinja yet for consideration in LOOT?

[Ortham]

ESL file support in LOOT is tracked by loot/loot#816. I haven't been able
to look into it as I don't own Fallout 4, so I'll just go off whatever I'm
told or stumble across.

On Mon, 28 Aug 2017, 21:07 Arthmoor [e-mail] wrote:

I haven't been able to check it yet because I'm looking through the
updated Fallout4.esm file, but are these ESL files being loaded into the
game after the main DLC files? If not then that's going to be an issue, but
otherwise if they always load after that the only concern is to make sure
they show up higher than things like the UFO4P when sorting a load order,
but then that also falls outside Bash's domain.

Has anyone mentioned this stuff to @WrinklyNinja
https://github.com/wrinklyninja yet for consideration in LOOT?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/wrye-bash/wrye-bash/issues/382#issuecomment-325465010,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUXEpEfwJkYsdKcuLJP8MLt2Y8Opj03ks5scx36gaJpZM4O6Jd5
.

[Utumno]

Lo posts:

https://afkmods.com/index.php?/topic/4966-wrye-bash-all-games/&do=findComment&comment=169498

https://afkmods.com/index.php?/topic/3750-wipz-tes5edit/&do=findComment&comment=169274

Finally what do I believe ? :P

EDIT: https://afkmods.com/index.php?/topic/3750-wipz-tes5edit/&do=findComment&comment=169519

[Sharlikran]

I'd go with whatever WrinklyNinja and Hlp are observing because they are looking at things at runtime. Hlp works with the script extender team and the DLL hooks into that to produce the results. I just don't see how you could get more accurate then that.

If you need to clarify something, maybe ask Wrinkly and Hlp with just yes or no questions. I don't see this being anything easy to explain because people look at things differently.

[Arthmoor]

They aren't the only 2 looking at it at runtime, but they ARE getting different results from other people who have done the same. It seems ESL files are tricky and do what they want to some extent. The only way to know for certain precisely how the game handles them would be to get Bethesda to comment on it.

[Sharlikran]

Bethesda isn't going to tell me that. I can ask but it's not worth waiting for them to respond, because they have never been able to answer anything that technical before. I don't think they fixed the SPGD issue I have been bothering them about with the release of the CC for Skyrim SE.

I think it just boils down to making a decision and doing what's easiest to program into Wrye Bash but keeping it as accurate to the observations given as possible.

With that said, what wrinkly ninja listed in that post does explain things well enough in my opinion. Also if you didn't see it Utumno I updated the UESP wiki to pull out the TES4 Header specifically since it is a unique record. It really shouldn't be included with the other records because it only occurs once in the file. The flags for that records are as Wrinkly is listing.

[Ortham]

They aren't the only 2 looking at it at runtime, but they ARE getting different results from other people who have done the same. It seems ESL files are tricky and do what they want to some extent. The only way to know for certain precisely how the game handles them would be to get Bethesda to comment on it.

I should note that the testing I've done is with Fallout 4 v1.10.26.0.0, it could be that SSE handles ESLs slightly differently, though I hope not. I'll perform the following test on both games tonight to check:

1. create two ESLs that override an item's name differently
2. set the master flag on one but not the other
3. put the two plugins in plugins.txt, with the non-master-flagged plugin above the master-flagged plugin
4. launch the game, and spawn the edited item

If I'm right, the item's name will be that given to it by the master-flagged plugin.

[Ortham]

Test results here, they validate my earlier findings and don't show any difference between Fallout 4 and Skyrim SE.

[Utumno]

If I'm right, the item's name will be that given to it by the master-flagged plugin.

Wait - this means that the game loads the master flagged esls _after_ the non master flagged ones ?

EDIT: you probably meant "by the non-master-flagged plugin." IIUC the post https://afkmods.com/index.php?/topic/5079-plugin-files-and-you-esmeslesp/&do=findComment&comment=169544

[Ortham]

Wait - this means that the game loads the master flagged esls after the non master flagged ones ?

No, it loads them in the order that's in plugins.txt, ignoring master flag. If the non-flagged is before the flagged, the flagged loads later, otherise the non-flagged loads later.

[Utumno]

Thanks @WrinklyNinja - so you meant non-master-flagged (see my edit) ?

[Utumno]

Fixed:

https://afkmods.com/index.php?/topic/4966-wrye-bash-all-games/&do=findComment&comment=169834

https://afkmods.com/index.php?/topic/4966-wrye-bash-all-games/&do=findComment&comment=169837

[Utumno]

Since ESL came into play, the load order id (FExx) and the actual order of loading are no longer identical. A form named FX01nnnn can be loaded before a form named 08nnnnnn.

To find the actual order you have to follow the logic that the runtime applies, which should be:

vanilla esm in order of publication (the table is inside the runtime).

vanilla esl in the order of the ccc file.

other master files (esm/esl or esp with master flag) activated in plugins.txt in the order they are listed with dependent esm/esp/esl "injected" before if they aren't selected yet.

remaining files activated, with once again dependent esm/esp/esl "injected" before if they aren't selected yet.

That is what I concluded by "reading" the code in the next to last version of Fallout 4 and Skyrim SE. Though there was also a difference between F4 and SSE I don't remember right now.

We probably need to create a test suite of plugins that can be used and shared to confirm every hypothesis we have so far. Just no time for it at the moment.

From https://afkmods.com/index.php?/topic/4966-wrye-bash-all-games/&do=findComment&comment=170810