Why I haven’t posted

Lazyness. Quickest answer but that’s not the whole story. It’s my first year in college and the experience has been to say the least overwhelming. At first everything is chill and fine until the exams come in. My grades were actually great but the amount of time that I spent studying leaves almost to no time to reverse engineer or to write. The time that I get to myself I spent watching youtube videos, series or playing.

Don’t get me wrong I still love to reverse engineer and explore the game engines of my favorite titiles but the circunstances lead to a lack of time to do so.

What has changed

Finally built my rig! Here are the specs:

• Ryzen 3 1200 (OC’d to 3.6Ghz)
• MSI Geforce GTX 1050 TI LP
• AEGiS G.SKILL 8GB DDR4 3000MHz RAM (supposedly XMP ready but i’m still unable to set its speed to 2933 so at the moment I’m locked at 2133)
• MSI B350M BAZOOKA
• Seasonic S12II Bronze 520W
• 1Tb HDD Seagate Barracuda
• A case from NOX with a windows :)
• BenQ GL2450 24’’ FHD Monitor

Peripherals:

• Fnatic Gear Rush G1 (MX Brown)
• MSI Interceptor Mouse
• Gamecom Plantronics 380

I have lots of ideas for my projects and I’m excited to start working on them, specially with my new workflow.

Resonance

To start, I’m dropping capstone and and keystone from my Resonance project due to lack of communication from the devs. Some PR have been sitting on their github since 2015 and aquynh, the main developer, seems more interested with people using its projects rather than actually improving them. He’s quick to add the projects using his engines to their respective websites but what irritates me is the fact some PRs are just left to die. The turning point was when I suggested a new disassembly flag which would let the developer choose how the 0x66 prefix would’ve been disassembled when working with x86_64. My suggestion came from an amazing talk from Christopher Domas about the x86 instruction set, you can watch it here, worth every minute. Basically AMD and Intel disassemble the 0x66 prefix differently (AMD does it correctly while Intel doesn’t) which can lead a malicious program following different paths when executing on a processor from those brands. The worst part is that all disassemblers make the same mistake as Intel. To solve the problem I suggested a AMD disassembly mode which would do it correctly and the Intel(default) which would have done things as they’re before. All code would’ve portable to the newer patched version but since then I’ve got nothing but radio silence from the developer. I even tried to contact him on twitter but he completly ignored me.

What irks me more is the fact that 3 months after the PR has been there sitting I decided to push the changes I’ve done to Resonance until February to the main repo. One of the changes included the Keystone assembler engine and to my surprise my project had been added to their Showcases. Not only my suggestions have been ignored but my projects have been broadcaat has an example. That means the main dev DID see my changes, so hasn’t he replied to my PR. Also he seems to go to every project that uses capstone, keystone or unicorn and opens a issue saying “cool!”/”awesome!” but refuses to give any support.

I’ll stop talking about crap projects and showcase two that seem more promising: Zydis and asmjit. Both have much more interesting licenses and their devs seem to be more open to the suggestions.

Custom Texture Loader

I started re-writing it and moved from MVSC to GCC and NASM. The only thing really tying me to Visual Studio was the inline assembly and naked functions and with that problem solved nothing is stopping me. Visual Studio is an amazing IDE but it ran so slowly and poorly and poorly on my old laptop that it drove me nuts that I’ve grown a certain aversion to it. Also I want to leave it clear that if I intent to do some Windows development I’ll surely use VS as it is the best.

TRG disassembler

Eh… Not really interested in this. As the file format seemed to be so bloated and stupidly arranged I kinda lost interested. I’ve said this a million times in the past but if someone with experience is willing to help reversing this game I will be really glad since I’ll be able to split my work and make much progress. The problem is that when I’m working on a part of the game I’m always questioning what the other is doing and without any help I need to play both of them which is tiresome.

MSDMAN

If the name isn’t clear then wait for the release ;).

End note

I’m also learning more about binary exploiting and pwning since it’s an area that really interests me, so expect some posts about CTFs and binary reversing that aren’t games. See you on the next post!

P.S - Happy (LATE) new year!

Hard to find

Before trying to figure out how the textures are stored I downloaded the game for every platform available, hoping I’d get some symbols left accidentally by the devs. They were: PS1, N64 and Dreamcast according to the wikipedia article and other sources.

Some while ago I was reading the PCGamingWiki entry on the game and it looks like the game was also released for the MacOS. The dev team behind it is Aspyr Media but even they don’t acknowledge its existence. Checking their website it’s not on any of their game’s lists: normal or legacy while Spiderman 2 MacOS, the master-shit is. No fucking idea.

Another weird thing I noticed is that the Google Sidebar that displays info about the game actually reports it was ported to Mac. Unfortunately I have no idea where it’s getting the information from.

Fun fact: Remember I said it is not present on the Wikipedia page, right? Well.. It is on the List of Macintosh games, go figure.

Finding them

To be honest finding them was really easier. A quick google search lead me to a website that seems dedicated to host old content. The website in question is MyAbandonWare. After that was matter of time to figure out how open .sit files. And was again with an unknown file format, toast. Thankfully, 7-zip is able to extract it. One weird thing is that it contains 2 spidey executables, one of them is big and the other is about 20Kb and contains some strings regarding the executing it( similar to the “This program can not be run in DOS Mode” ).

What’s the big deal about them

They contain symbols. Symbols are accessory information(function names, variables,..) used to aid the debugging process and they are almost always stripped. But for some reason non-Windows games have the fame to usually not being stripped. Awesome!!

If you look at my texture extractor you’ll see most variables don’t have a name because I have no ideia of what they do. Although they play a part in getting the texture and I see how they work, I wasn’t able to extrapolate their intended use. Now that I have access to the function names correlating instructions and strings I’m able to correctly name the functions in the Windows version making the reversing sooo much easier. Because of that now I know that the textures are “twiddled”, a method used to increased performance in rendering. Even though this is not a problem in PCs it was kept..lazy devs.

What now?

At the moment I’m finishing a TRG disassembler but right after I’m done I’ll refactor my old tools and document the PSX file format. If this information hadn’t come out I wouldn’t even think of updating them.

Download

Download here

Motivation to do it

Right after the first version of the PKR extractor I was thrilled and wanted to explore more of the game’s files. Unfortunately I was only able to extract the files, repacking them is another story. Although kamiloxnumetal already had released tools able to unpack and repack PKR, having to do it every single time I modified a file is insane.

Finding the responsible function

While reversing it was clear that the game setups internally all the information of the PKRDirs and PKRFiles, after that the PKR file is simply used to load the files. This made things easier since there was only one function that I needed to focus.

Here’s the function I was talking about, it starts at 0x00519194. This is the relevant part where all the magic happens, if you’re curious of what happens before what is shown in the picture it goes to a specific PKRDir and searches if there’s an entry of the specified file name. If there is then it performs some sanity checks and then proceeds to get the file. Here’s the order of things:

1. Allocates a buffer with the size of the compressed file (compressedSize is equal to uncompressedSize in non-compressed files)
2. Fseeks to the offset and reads the file to the buffer.
3. Calls sub_51AD0C aka the decompression routine. If the file uncompressed it simply returns the buffer, if not it uncompresses the file, deletes the old buffer and returns the new one.
4. Perfoms the CRC check

As you can see the only if the CRCCheck succeeds the a4 and a5 parameters are set and that is the only case the function returns 1(true). Awesome! a4 and a5 are definitely “out” parameters that store the buffer of the file and the size, respectively.

Writing the loader

Getting the code to run

My first idea was to write a program that started spidey.exe with the SUSPENDED flag and then I’d inject the loader. Unfortunately this isn’t so easy as it seems because when you start a process with the SUSPENDED flag the needed structures for DLL loading are not setup yet. There are some tricky ways to do so and I even thought in trying to suspend the process right after it started but it still wasn’t the best way.

That was when I thought in proxying a game’s DLL, the one I chose was binkw32.dll(It is used to load and play the game’s cutscenes, the bik files). Although this process is mostly used to detour the calls to a certain DLL, here I used it to install my loader before the game reaches its entry-point (running my code before the game starts). Using a tools such as ExportToC++ it’s a really easy process. It basically redirects all the calls to binkw32 to the original one, so one less thing to worry about.

Writing the loader

Here’s the interesting part from my loader.

	//Disable buffer allocation of pkr extracted file
	if (!NopMemory(0x000519375, 0x005193A1 - 0x000519375))
		return FALSE;

	//Disable fread of the file and the error
	if (!NopMemory(0x005193C5, 0x005193CA - 0x005193C5))
		return FALSE;
	if (!SetMemory(0x005193D0, &twoByteJmp, 1))
		return FALSE;
	if (!NopMemory(0x005193D2, 0x005193FD - 0x005193D2))
		return FALSE;

	DWORD jmpOffset = (DWORD)&LoadFile - 0x0051940C;
	//Hook file loader function
	if (!SetMemory(0x00519408, &jmpOffset, 4))
		return FALSE;
	//Disable CRC checks
	if (!NopMemory(0x0051942D, 0x00519445 - 0x0051942D))
		return FALSE;
	if (!NopMemory(0x0051941E, 0x00519423 - 0x0051941E))
		return FALSE;
	if (!SetMemory(0x0051AD70, &retOp, 1))
		return FALSE;

Before actually starting to modifiy the game I decided to write two functions that would help me with the process of patching. NopMemory as the name indicates it NOPs(0x90) memory. The first parameter is the starting address and the second is the amount of bytes to patch. SetMemory receives as the first parameter an address to be written to. The second is the buffer that will be written and the third is the size of the buffer.

The comments I left are not 100% accurate since I wrote them at the start and forgot to update them. So here’s how they really work.

• The first NopMemory is correct and it’s used to remove the buffer allocation(new (pkrFile.compressedSize)) because if I didn’t do so it would cause a memory leak.
• The second one is used to disable the fread wrapper, because we’re going to read the files from the disk.
• Then we have a SetMemory. Since I disabled the FreadWrapper it won’t return anything. This means that the value that is in EAX will be the one to be checked if is 1. Unfortunately EAX holds the compressed file size which is never 1, thus requiring modifying the jump.
• The following SetMemory changes the call to the decompression routine to my LoadFile.
• The next NopMemorys are used to disable the CRC checks and the delete call.
• Lastly, SetMemory was used to disable a internal CRC check that was causing some problems.

The actual file loading

File name setup

The snippet is from here.

__declspec(naked) PVOID LoadFile(PkrFile *pkrFile, PVOID loadBuf, DWORD one) {

	__asm {
		//get the directory
		mov eax, [esp + 0xA4]
		mov originalDirectory, eax

		//Setups the path
		push originalDirectory
		push filePathAdd
		call mystrcpy
		add esp, 8

		push 0x20
		push [esp+8]
		push eax //Its in eax from strcpy
		call strncat
		add esp, 0xC

		push [esp+4]
		call LoadStub
		add esp,4
		ret
	}

}

As I said in the beginning, the function can only load a file by knowing its name and the directory it is inside. This means that using some math I can access the other function’s stackframe and retrieve this information. After concatenating the directory with the file name it’s ready to be loaded.

Snippet from here.

PVOID *LoadStub(PkrFile *pkrFile) {

	HANDLE hFile = CreateFile(filePath, GENERIC_READ, 0, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL,
		NULL);

	if (hFile == INVALID_HANDLE_VALUE) {
		MessageBoxA(NULL, "Error opening the file", "dammit", 0);
		return NULL;
	}

	DWORD fileSize = GetFileSize(hFile, NULL);
	if (fileSize == INVALID_FILE_SIZE) {
		CloseHandle(hFile);
		MessageBoxA(NULL, "Error getting size", "dammit", 0);
		return NULL;
	}
	pkrFile->uncompressedSize = fileSize;

	UCHAR *buffer = malloc(fileSize);
	if (!buffer) {
		CloseHandle(hFile);
		MessageBoxA(NULL, "Error creating buffer", "dammit", 0);
		return NULL;
	}

	DWORD bytesRead = 0;
	if (!ReadFile(hFile, buffer, fileSize, &bytesRead, NULL)) {
		free(buffer);
		CloseHandle(hFile);
		MessageBoxA(NULL, "Error reading the file", "dammit", 0);
		return NULL;
	}

	if (bytesRead != fileSize) {
		free(buffer);
		CloseHandle(hFile);
		MessageBoxA(NULL, "Not all bytes were read", "dammit", 0);
		return NULL;
	}

	CloseHandle(hFile);
	
	printf("Loaded %s\n", filePath);
	return buffer;

}

Nothing special about this one, it’s a regular file loading routine. Until my latest version it would crash if you modified a file with a bigger one, that was caused due to not modifying the uncompressedSize. After adding the pkrFile->uncompressedSize = fileSize; line it still didn’t work, due to my PKRFile structure being incorrect. Thankfully I was able to fix this easily with the aid of x64dbg.

That’s all folks!

After a lot of blood, sweat and tears I got the custom file loader!

What is it?

It’s a container format used to store all the game’s files. It was developed by Neversoft and is most known for its use on “Tony Hawk Pro Skater” game series. Since it contains all game files in order to modify anything we need to know what is going on.

Structure

typedef struct{
	uint32_t magic;
	uint32_t dirOffset;
}PKR3File;

The files start with a magic number PKR3 followed by an offset to the directory tree header.

As you can see after the offset there are the words RIFF and WAVE which are also magic numbers of WAV files. Due to the fact that spidey.exe contains a lot of file names this mislead me. Initally I thought this files had no structure and were just a bunch of files merged together. Thankfully there are a lot of references in the code to the functions responsible to handle PKR files.

Dir structure

The header

typedef struct{
	uint32_t unk;
	uint32_t numDirs;
	uint32_t numFiles;
}PKRDirHeader;

Quick note: You’ll see a lot of variables/members called unk due to the fact that I have no ideia of what they do and their modification not having any sign of impact on the file interpretation and not existing any references to it in the cose.

It contains the number of directories and the total number of files that the PKR holds.

The body

typedef struct{
	char name[0x20];
	uint32_t unk;
	uint32_t numFiles;
}PKRDir;

After the header there’s an array of PKRDirs(its lenght is defined by numDirs). And right after this there’s a PKRFile array with the lenght of the total number of files.

typedef struct{
	union{
		uint8_t total[0x34];
		struct{
			char name[0x20];
			uint32_t crc;
			uint32_t compressed;
			uint32_t fileOffset;
			uint32_t uncompressedSize;
			uint32_t compressedSize;
		};
	};
}PKRFile;

As you can see the game files have integrity checks, in this case it’s actually CRC.

compressed can either hold 2(0x00000002) or -2(0xFFFFFFFE) which mean compressed and uncompressed respectively.

fileOffset contains the offset to file in the PKR file counting from the start.

compressedSize holds the size of the compressed files.

uncompressedSize holds the size of the uncompressed files OR the size of a compressed file when decompressed.

Here’s how all these structures look in the file:

Green is the PKRDirHeader, red the PKRDirs and blue the PKRFiles.

Compression method

The files are compressed using zlib. Finding it was a piece of cake, again there are a lot of references in the strings.

Implementing it was the hardest part, during this period I got so many crashes that I decided to simply go with uncompress(). I know it sucks because it wastes a lot of cpu power but hey, it works ¯\_(ツ)_/¯.

If you’re interested in reading my implementation you can checkout my github.

Quick recap

The last post ended with me being able to generate the correct checksum for the save slots. This means regardless of its contents being valid according to the game I’m now able to load my custom save files.

Current level

As I said this game becomes unbeatable at certain levels so this was the first thing I looked for. Looking at the available strings in the save slot there was one that caught my attention, l1a1_t, it really looked like “Level 1 Area 1”. So I decided to check in IDA if there were any references:

Awesome! Not only it is referenced but the level names are intercalated with the respective level codes(lXaX_t).

Time to test

I modified the string to l1a2_t and corrected the checksum. The save loaded correctly and it was displaying the correct string “Bank Approach” in the main menu, unfortunately it was still the first level… Since the levels don’t contain the same number of areas the developers took the easy way out and have 2 references to the current level in the save file, one is for the string the other is for the actual level.

Finding the real level

This one was actually easy. Instead of trying to find it through the disassembly I just played the first few areas and compared the saves. While I was progressing through each area, more and more bytes were turning from 0 to 1(one per level), starting at 0x56. Basically for each area that’s beaten the byte that corresponds to its index is set to 1. When you finish the game the byte at position 0x55 is also set to 1.

Since we’re at the start of the game all the bytes are set to 0

Costumes

Current costume

To find how they were stored I’d simply input the code save and check the differences in the file. At the offset 0x280(0x7C of the slot) that it saves the current ID of the costume the player is wearing:

1. SPIDERMAN
2. SPIDERMAN 2099
3. SYMBIOTE SPIDERMAN
4. CAPTAIN UNIVERSE
5. AMAZING BAG MAN
6. SCARLET SPIDEY
7. BEN REILLEY
8. QUICK CHANGE
9. PETER PARKER

Unlocked costumes

This one trickier to figure out. At the offset 0x284(0x80 of the slot) there’s a DWORD that seems to change whenever you unlock a costume. If you unlock them in order it will simply increase from 1 to 1023(0x3FF). It’s a bitmask and the first 9 bits represent the costumes. If it’s 0 then it’s locked, then it’s unlocked. That’s why it goes from 1(000000001) to 1023(111111111).

Storyboards, Comics..

This part doesn’t really matter to me and since I’ve found out about cheat codes I stopped working on this.

Make sure to check the source code of the save editor: HERE

And here’s a video of it in action: