Wrye-bash: Cosaves are not edited when editing the save's masters list

[Utumno]

Carried this from #192 cause that one contained a busload of unrelated issues fixed for 306.

---

OP from @Sharlikran:

On the saves tab you can assign a new master to the header of a save file. For example Someplugin.esp might have an update to SomepluginV2.esp. class CoSaves: seems to be using bush.game.se.shortName but class ObseFile: does not seem to do so.

Under with sio(buff) as ins: there is a line if self.signature != 'OBSE': that should probably be using bush.game.se.shortName which would be an easy change. However, under def save(self,path=None,mtime=0): it uses buff.write('OBSE'). Which is also an easy change.

Behavior without changes:

Wrye Bash errors in def load(self): for line if self.signature != 'OBSE':.

NOTE: You have to click the save button twice and the change is not reflected until you click in the box again. Which then Wrye Bash auto updates the window with the new file name. After which you have to save it again.

Behavior with minor changes:

Wrye Bash no longer has an error in the trace log. However, when comparing the .ess and .skse files only the .ess files list of master files are updated. The .skse file is untouched.

Expected behavior:

Both the .ess and .skse files should be updated to reflect the changes in file names for the save game headers. You should only have to click the box once and the change in the file name should be reflected the first time rather then having to repeat the process.

---

Traceback (most recent call last):
  File "bash\basher\__init__.py", line 2437, in DoSave
    saveInfo.header.writeMasters(saveInfo.getPath())
  File "bash\bosh.py", line 1120, in writeMasters
    obse.load()
  File "bash\bosh.py", line 978, in load
    raise FileError(self.name,u'File signature != "OBSE"')
bash.bolt.FileError: Save 1705 - Lilly  Skyrim  00.27.52.skse: File signature != "OBSE"

---

(Utumno)

Ok - dunno what this means but the line that prevents the masters being edited in the OBSE case is no less than if opcodeBase == 0x2330: this comes from 3ac0142aa0096dc9f301e9a69c9b729e865315ee and stayed put since.

Still maybe the OBSE masterlist cosave needs not be edited - improbable

Also do not know how this code applies to SKSE if at all

---

• What this bug is about - cosaves are NOT edited in Oblivion (please confirm) or Skyrim
  
  
  
  • [x] Add big fat warning for 306
  
  
  • [ ] SKSE class for 307
  
  
  • [ ] OBSE class patch to work with if opcodeBase == 0x2330: - whatever this means
  
  

[Utumno]

In reply to post : https://afkmods.com/index.php?/topic/4966-wrye-bash-all-games/&do=findComment&comment=168538 by @hlp:

Utumno The master table should be, for all games, in chunk SDOM (as MODS in reverse :) ) owned by the first plugin which is the script extender itself.

All chunk names are acronym spelled in reverse so they read correctly in the source.

WARNING: F4SE just introduced the LMOD master table which is the list of ESL files used by that save. The DOML chunk has the same format as the SDOM chunk and immediately follows it. Of course it should be considered optional.

opCodeBase for OBSE, FOSE and NVSE is 0x01400 = 5120, Any plugin will use a larger opCodeBase (greater than 0x02000). A mod using the 0x02000 opCodeBase is not allowed to use serialization.

"opCodeBase" for SKSE and F4SE is 0 (see below).

Sharlikran

Since Papyrus, opcode are no longer necessary nor attributed centrally by the xSE team. So rather than using opCodeBase, plugin comes up with their own unique "signature" as a 32 bit integer. That number is usually a 4 character string as can be seen in the plugin_example project provided with SKSE sources which uses 'TEST' as a signature.

The script extender (SKSE and F4SE) use 0 as their signature.

You may want to use the term signature rather than opCodeBase in your source as this will be more generic going forward.

Also, as you found out, 2330 identifies pluggy which has been treated as "part of the standard" by some oblivion tools. Elminster also made code specific for pluggy in TES4Edit.

The number of plugins should be the ones that have registered to the serialization interface. Plugin that do not use the cosave should not be counted IF I remember correctly.

Each plugin save and load in turn, so you cannot have the same opCodeBase twice. The way they split their data in single or multiple chunk is up to them.

Thanks indeed for clarifications - couple questions:

1. The code in mapMasters https://github.com/wrye-bash/wrye-bash/blob/dev/Mopy/bash/bosh/_saves.py#L403 remaps espm masters of the pluggy file - hence the if opcodeBase == 0x2330: test. This means that we never ever edited the espm masters for the OBSE plugin itself (opCodeBase 0x01400 or 0x0 for papyrus ones). Do I understand this correctly ?

2. If above is yes, the only things I must do to rename an espm master is rename it inside the SDOM (or DOML) chunk ? Or do the espm names occur in other places ? To be noted that I don't get why they should occur inside xSE plugins at all apart from the xSE plugin itself (signature/opCodeBase 0x01400 or 0x0)

3. Can you link me to some specification (and/or just the source code) ?

[Sharlikran]

I know you are asking Hlp but his response on the forum already answered your question about whether or not there are names for the espm masters in other segments/blocks/chunks of the cosave. Within the cosave (.obse, .nvse, .fose, f4se, skse) save files there is only one list of file names in SDOM.

The list of file names for mapMasters is in a separate file made by the 3rd party OBSE extension Pluggy. I have not found an Oblivion mod yet that will cause a .pluggy cosave file to be written. I haven't seen one yet. I can't link you to a Wiki Page or source code because that doesn't show up in Google. The only way I found it was to download it from the nexus.

For your third question, all of the documentation for OBSE is at this link. There is no save file format information in that. That team may not want to have a public github, so the source code is in the download obse_0021.zip. It would be in C++ but I think you know it well enough IIRC.

Just curious, why isn't the file format in this post helpful to you? The Python 3 source code to print the file contents to the screen wasn't helpful either?

[Hugues92]

The source code for the extenders is distributed as part of the distribution archive (check http://XXse.silverlock.org).
Inside those, the cosave is handled by serialisation.cpp and the extender related code should be in core_serialisation.cpp or InternalSerialization.cpp depending on the age of the extender.

There are no official public repository for any of them.

[Hugues92]

As for 2) yes renaming the name in the table(s) is enough for the extender and any plugin that make uses of the functionality provided by the extenders.

Unfortunately there can always be someone who did a non standard use of the plugin name or the plugin index as part of their plugin (most likely in old plugin for OBSE where the serialization interface was a lot less developed), but there's nothing you can do for those I would think.

[Utumno]

Thanks @Hugues92 - following the discussion in the forums (especially https://afkmods.com/index.php?/topic/4966-wrye-bash-all-games/&do=findComment&comment=168549 ) my other questions boil down to:

OBSE itself (at least after https://github.com/wrye-bash/wrye-bash/commit/3ac0142aa0096dc9f301e9a69c9b729e865315ee ) and the pluggy plugin chunk contain the save's master names. Probably other plugins too - ideally we should detect that, ideally.

Now let's put this to sleep for some time - Sharlikran I replied to you in the forums

EDIT: Ninj'ed

[Utumno]

@Hugues92: what I miss I guess is the structure of the SDOM chunk for the xSE plugin - can you fill that in ?

[Hugues92]

it looks like this:

Save file format:
Header
MODS // stored in order of modIndex when game was saved
UInt8 numMods
UInt16 nameLen
char modName[nameLen]
...

[Hugues92]

Sorry I cannot seem to keep the indentation correctly

[Sharlikran]

@Hugues92 Creating and highlighting code blocks

@Utumno Why isn't the code example I gave sufficient? Regardless of a lack of classes and doc strings the code very simple. Make a new project in Pycharm. Run my code with PyCharm using Python 3. You need to copy any cosave from any game into the working directory for the project and add the file name to path = ''. Then run it.

My code now has a way to print out the file names. It's still very basic and doesn't do much but it's just an example.

All you are doing is looping through the chunks, similar to this.

num_plugins = 2
num_chunks = [6, 10]

print "Cosave Header: "
for cosave_file in range(num_plugins):
    print "Cosave Plugin Info: ", cosave_file
    for chunk in range(num_chunks[cosave_file]):
        print chunk

| CoSave Header | Type / Size | Desc |
| ------------------ |:-------------:| ----------------------:|
| signature | uint8[4] | Name of cosave file |
| format_version | uint32 | Format Fersion |
| obse_version | uint16 | OBSE Version |
| obse_minor_version | uint16 | OBSE Version |
| oblivion_version | uint32 | Oblivion Version |
| num_plugins | uint32 | Number of OBSE Plugins |

| Chunk Block Header | Type / Size | Desc |
| -------------------|:-------------:| ----------------------:|
| opcode_base | uint32 | Unique Plugin ID |
| num_chunks [2] | uint32 | number of chunks |
| plugin_length | uint32 | Total Size of Data [1][3] |

[1] Not needed to read chunks, unused in my code.
[2] Use num_chunks from the Chunk Block Header to determine how many individual chunks you need to read.
[3] When writing the new list of file names plugin_length needs updated for SDOM or DOML.

| Chunk | Type / Size | Desc |
| ------------- |:-------------------:| --------------------:|
| chunk_type | uint8[4] | Name of Chunk |
| chunk_version | uint32 | Chunk Version |
| chunk_length [4] | uint32 | Chunk Length |
| chunk_data | uint8[chunk_length] | Data of chunk_length |

[4] When writing the list of file names, chunk_length needs updated for SDOM or DOML.

When I use uint8 that's from the UESP File Format Conventions wiki. It means a byte the size of the data. uint8[4] means 4 chars or 4 bytes. uint8[chunk_length] means if the length is 200 you read in 200 bytes.

[Utumno]

Thanks @Hugues92, what I missed was "Uint8 numMods"- @Sharlikran when you posted somewhere in the thread it had no "unpack_names()" - I can't follow code here and there - I just need specific questions answered. Good you start playing with python though keep up

[Utumno]

@Sharlikran let's remove personal stuff from this issue - please keep the links to your code and please edit your post above ( https://github.com/wrye-bash/wrye-bash/issues/236#issuecomment-329364821 ) has some inaccuracies (look at the header table for instance)

[Sharlikran]

Updated.

[Utumno]

Thanks!

On Fri, Sep 15, 2017 at 4:26 PM, Sharlikran [e-mail]
wrote:

Updated.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/wrye-bash/wrye-bash/issues/236#issuecomment-329782167,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABTUV3NT9t86v6HGhNIEyFfrr3vTWD6Oks5sinsigaJpZM4F8Czw
.