Godmode9: Bad data added to the end of these cart dumps when decrypted?

[mariomadproductions]

Dragon Quest VIII - Journey of the Cursed King (USA):
CRC32: b4858ee2
MD5: c7d8e05a97e583c38b9cadb6d02bc27b
SHA-1: ef18ca1817ec679a66e96383f1b21d2a120585df

Legend of Zelda, The - Majora's Mask 3D (USA) (En,Fr,Es) (Rev 1)
CRC32: b912dfd7
MD5: a6a378c282ac1aa309480a48b35ec4a2
SHA-1: 7abdf89e5c579d274a10417e1d639a0e67876d53

Pokemon Alpha Sapphire (USA) (En,Ja,Fr,De,Es,It,Ko) (Rev 2)
CRC32: 5fa9a01a
MD5: b1aa4e2bf4bd814fbf60a5f749266f6d
SHA-1: c537558c568e69eda1efa9cb596a5865ac011641

Poochy & Yoshi's Woolly World (USA) (En,Fr,Es)
CRC32: 05fc0c10
MD5: 831e7b1fdbd6bf5fc0b55aaaefbd6f19
SHA-1: 6ceac9ee7828db23eaaa351a3b9b3f98a09ffc15

Radiant Historia - Perfect Chronology (USA)
CRC32: 3ede3158
MD5: 86c0dd365f78de826bbd0c5c9d6a4496
SHA-1: 8eea954343c2e73f623a1e9839dc1b89e898a780

On latest GodMode9 (v1.7.1) and some previous GodMode9 versions these supposedly get bad data put on the end when they are decrypted. jimmsu will hopefully post with some more details.

[d0k3]

You see it yourself, this misses info, actually a lot. What do you mean by bad data added? Is the decrypted size > than the encrypted size? What do you suppose to be bad data?

My first guess: GM9 'blindly' decrypts the whole RomFS, it doesn't matter if the whole area is actually used. Thus, blanked data may actually be decrypted, leading to garbage.

Anyways, waiting for more info. I can't do anything with the info you added so far.

[mariomadproductions]

Technically you could acquire those encrypted ROMs and decrypt them and then compare the filesizes. I may be able to do that myself soon.

[d0k3]

To be blunt... I could, but I'm on a really poor (honestly, you have no idea how bad) internet connection right now, and this won't change during the next few days. I could use some help. Maybe someone else can check one of the mentioned cart dumps?

This is also missing more info... does it happen when decrypted inplace? Does it also happen when decrypting to 0:/gm9/out?

[Zaphain]

I'm the person that initially dumped them. It happened when I decrypted straight from the cart to gm9/out. Encrypted matches the database. Decrypted does not. At the same time I made 21 other dumps which match both encrypted and decrypted; It's just these 5 I have that turned out different. Jimmsu took a look at them and said there was more data at the end of partition 1 that shouldn't have been there. If you need me to do more, just ask.

[d0k3]

Is there a database for decrypted cart dumps? If so I wonder what tool was used to decrypt (if not GM9) for that database.

You could check if it's a cart specific thing. Is there a difference when you dump first, then decrypt (vs decrypt right from cart)?

You could also check the difference between expected result and the one you got in a hex editor (HxD f.e.) and post a screenshot.

It's enough of you do this for one of these carts.

[mariomadproductions]

There is, as a sort of sub-database to the No-Intro 3DS cart dump database. But its justs fields for each dump's decrypted checksums - no info on the tool used, but I suspect it is one of the versions of the python script "3ds_decrypt_encrypt" by the user "b2071988" on the forum called "X Shepherd". Where X is the three-letter word of the type of dump we are discussing.

[Zaphain]

Later today, when I'm available, I'll test with decrypting using one of my encrypted .3ds dumps (majora's mask) and see if the result is different. I'll also check the difference between that and the same game decrypted by that python script. I'll use HxD for it as you suggested. I did test with that script before and it does give the expected hash from database, so mariomadproductions is probably right with his assumption that that is where the hashes originate from.

[Zaphain]

For "Legend of Zelda, The - Majora's Mask 3D (USA) (En,Fr,Es) (Rev 1)"
crc32 of encrypted: b912dfd7 (matches the database).
crc32 of decrypted with GodMode9 from the cart or from encrypted file in-place or to gm9/out: eb8196b1 (doesn't match the database).
crc32 of decrypted using aforementioned python script: 391e9fbe (matches the database).

Here's an image of the difference in the data HxD shows me: https://i.imgur.com/pd227Y7.png
File on the left is the godmode9 decrypted game. File on the right is decrypted with the python script.

[redunka-zver]

I remember seeing the same thing while messing with Majora's Mask several months ago.
That garbage data is padding between code and banner sections that wasn't decrypted properly, it seems?
Probably should've reported it back then, but it wasn't that important for me at the moment, and I totally forgot about that later.

[d0k3]

Thanks @Zaphain. That's the RomFS, not the ExeFS, and it's at best doubtful this way of decrypting it is not correct. GM9 handles the RomFS as one block of data, and the tool that was used here does some kind analysis. Thus, a unused (padding) block of zeroes is untouched in that other tool and GM9 decrypts it (it will also encrypt it back to zeroes).

The required analysis may also not work in GM9, cause GM9 needs random access, on the fly crypto.

I'll have a more thorough look later.

[james-d-elliott]

I'm thinking the RomFS metadata may contain the required information to do this and is probably how the other tools do it?

[d0k3]

That's correct, but that can't be used on the fly or random access. There are already some tricks involved, but doing this would slow it down in a way you guys would notice (or wouldn't be possible at all). Again, it's doubtful either way can be called "correct".

If we were nitpicking here, any modification that causes the sig to be invalid (aka any modification) renders the cart dump incorrect.

[d0k3]

There is, as a sort of sub-database to the No-Intro 3DS cart dump database. But its justs fields for each dump's decrypted checksums - no info on the tool used, but I suspect it is one of the versions of the python script "3ds_decrypt_encrypt" by the user "b2071988" on the forum called "X Shepherd". Where X is the three-letter word of the type of dump we are discussing.

Okay... so, I downloaded that Python script by b2071988 (most recent version from what I see), but that script doesn't handle RomFS in any special way. It's doing the exact same thing that GM9 does. @Zaphain - can you point me to the exact script you used? In pm (GBAtemp / IRC) if it should contain anything sensitive.

Edit: just to be clear, I'm using v4 of the mentioned script. I understand that's the newest one.

[d0k3]

Okay, here's something to try:
https://f.secretalgorithm.com/GIxho/godmode9.firm

That one should generate the same output that Python script does. @Zaphain - could you test? When you have time, of course. Also, other testers are appreciated.

[Zaphain]

I used a windows executable of v3 that b2071988 put together, available from a post he made there.

I quickly tested with majora's mask with the version of godmode9 you just linked and it gives the expected result now. : ) I'll be testing with the other 4 problem games and give you the result when I've finished.

[Zaphain]

All decrypted game crc32s now match the database when decrypted with that godmode9 test version.
Dragon Quest VIII - Journey of the Cursed King (USA) -> 762478d0
Legend of Zelda, The - Majora's Mask 3D (USA) (En,Fr,Es) (Rev 1) -> 391e9fbe
Pokemon Alpha Sapphire (USA) (En,Ja,Fr,De,Es,It,Ko) (Rev 2) -> d4dda2e3
Poochy & Yoshi's Woolly World (USA) (En,Fr,Es) (Rev 1) -> 2a5a8388
Radiant Historia - Perfect Chronology (USA) -> 603ee563
So, I can confirm that it does output the same that the script gives.

[d0k3]

Alright, thanks a lot for testing!