ROM hacking resources

From Emulation General Wiki
Revision as of 23:22, 21 September 2016 by 109.163.234.9 (talk)
Jump to navigation Jump to search

This page lists tools and info on ROM Hacking. See Mods, Hacks and Fan-Translations for more general info.

General Tips

What Are Bytes

Our normal counting system uses the decimal base, that is base 10. We'll count this way 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... and then 10 = 1 * 10 + 0 * 1 = 1 * 10^1 + 0 * 10^0. So if we have something like 234 it actually means 2*100 + 3*10 + 4*1 = 2 * 10^2 + 3 * 10^1 + 4 * 10^0.

On the other hand, the hexadecimal base is base 16. So we'll count this way: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... and then A (a single "digit" the hexadecimal base uses for the value ten), and then B (eleven), C (twelve), D (thirteen), E (fourteen), F (fifteen)... and then 10 (sixteen, that is 1*16 + 0 = 1*16^1 + 0*16^0) and 11 (seventeen, 1*16+1*1) and 12 (eighteen, 1*16+2)... etc. Actually we should write this 0x10 or h10 (sixteen) so that it's not confused with the decimal 10 (ten). If this sounds too complicated, you can load your Calculator in Programmer mode and do all the converting between our decimal and the game's hexadecimal.

The binary base is base 2. We count this way: 0, 1... and then 10 (which is 1*2 + 0... two) and 11 (1*2+1... three) and 100 (1*2*2 + 0*2 + 0*1 ... four), and 101 (1*2*2 + 0*2 + 1*1... five)... and so on. As you can see, it gets long and impractical very quickly.

Everything in game ROMs, be it programming, graphics, sound, text, assets and anything else, is written in bits (zero or one), with each group of eight bits called a byte. Out of convenience, bytes are written using the hexadecimal base (any values using this we'll note with the prefix 0x), rather than an inconvenient succession of eight bits (using the binary system).

The value for each byte ranges from 0x00 (in bits: 0000 0000, in decimal: 0) to 0xFF (in bits: 1111 1111, in decimal 255).

Text Editing with Hex Editors

Hexadecimal editors, also called hex editors or binary data editors, can open ANY file, and display its contents as bytes written using the hexadecimal base. You can also edit said bytes.

Hex editors come with three elements usually: The part with the binary data, an address (also called offset) on the far-left which tells us the location of this byte in the file (and is useful), and...

The third area in the window is where "text data" is supposed to appear. It tries to interpret the hex data as text, by matching each byte value to a specific character in the ASCII set. So if there's the 0x41 byte for example, the matching character on the right side would be upper-case latin A.

Practice: Go and check the US version of Link's Awakening (either the GB or GBC version, doesn't matter), open it with any hex editor available online (the ROM of course, not the ZIP archive), and try modifying Marin's dialog at the very beginning. You'll have to search for a while, but you'll eventually find it.

However... Most of the times, the text data area is gibberish nonsensical symbols. Why? Often, it's because:

  • The specific portion of the file / ROM you're viewing isn't actually text data but something else. So you'll have to use the search feature or browse further down the file.
  • The text is encrypted (if the developers wanted on purpose to make it unreadable by hex editors: this is the case for 3DS ROMs when they're encrypted, or games with anti-modding measures like Gods Eater 2 PSP (JP) and Youkai Watch save files). Fortunately, this is impractical and mostly uncommon.
  • The text is compressed. Compression is a data transformation operation intended to save space. There are numerous schemes, and some games have their own unique flavors. Either by studying the file structure blindly, or reverse-engineering the game's programming during runtime as it makes use of this compressed file, you'll need to figure out the actual compression pattern, and decompress the text so that you can actually edit it (then recompress it and feed it back to the game).

But sometimes it can be none of the above reasons. But rather the fact it uses a custom character encoding that's not the ASCII standard.

Why is this the case? Because many of these games were made in Japanese, using JP characters, and as such the devs had no reason to respect the ASCII standard (where 0x21 means !, 0x31 means 1, 0x41 means A, 0x61 means a, etc...) but were more concerned about how to make the best out of the limited memory available. So you could have A at 0x00, x01, 0x0A, 0x10, 0x21, 0x41, 0x81 or anything really, depending on the game and what the devs felt like.

However this poses a problem. None of the mainstream hex editors really give a damn about this enough to account for cases other than the ASCII standard, so... You'll have to find a specialized hex editor geared more towards ROM hacking. The feature you'll need here is the ability to load custom character sets, stored in what's called table files (extension .TBL, though they're actually just renamed regular .txt files).

A table file for a game using the ASCII standard would look like:

41=A
42=B
43=C
44=D
45=E
46=F
47=G
48=H
49=I
4A=J
4B=K

And so on. For many games, you'll need to figure out their table files, using table file building tools also commonly referred to as "Relative Search" tools. Choice ones are RSEARCH and monkeymoore. You know for example Zelda 1 has "IT'S DANGEROUS TO GO ALONE" so you load the ROM in monkeymoore and search for the word "DANGEROUS". It will give you many options (assuming the text is uncompressed, and the programmers at least had the decency to respect alphabetic order -otherwise you'll need to use the "custom set" option, which you'll need in case your game is in Japanese). One of these options should guess the rest of the words of that sentence, with some unknown characters in between (like the spaces). Check that option and create your table with it.

Under WindHex, a hex editor with TBL support (there's also Crystaltile2 -which also had a few standard encodings for Japanese (Shift-JIS and UTF-8)- and Tinke), open the ROM, the load the table file you just created with the relative search tool, and then the text will be visible and editable if you scroll to that area of the ROM. You can see the byte equivalents for the missing characters (spaces, punctuation) and add them to the TBL file with Notepad (or WindHex's Table Editor).

Make sure also to identify control codes, that is bytes used to tell the game to do special stuff (like text color, speed, writing the hero's name, line breaks, telling the game where the text ends...) and add them to the TBL file too. Sometimes the game will use a dictionnary to make the game's text shorter in order to save space, so some byte values will replace letter combinations or entire words. Add those to the TBL file too.

Don't be afraid to experiment, even if this corrupts your ROM, so that you confirm any of your observations! Of course, you'll need to make sure to keep a safe backup copy of the unaltered ROM as well as the ROM revisions with your main hacking progress.

Text Dumping and Insertion Tools

Editing or translating a whole game with a hex editor is tiresome, and you can't even exceed the original length (since the extra text would just overwrite the next not-text data and corrupt the game).

The solution is to find the game's pointers. You know you can note down the address on the left side of the hex editor to know where you are in the ROM. The game does just like that, it has pointers telling it where the text is.

But... Pointers don't look the same way the address in your hex editor does. Each system has its quirks and rule for calculating the pointers, so look it up.

Practice: GBA pointers for a specific address are 4-byte. Replace the leftmost byte with 08, and invert the order of the bytes so that (byte1)(byte2)(byte3)(byte4) becomes (byte4)(byte3)(byte2)(byte1). Why invert? Because GBA is a Little Endian system. Open a GBA Pokémon game, find Professor Oak's dialog in the hex editor, find its starting address (you click on the first letter and then see its address for that byte in the sidebar of your hex editor) and calculate the pointer as detailed above. Use WindHex's feature for searching hex data to find that pointer. It should appear in the hex data and look just like what you calculated. You may modify it to an address in the very end of the ROM, in an empty area where you'll try writing new text. If you can pull out this, this means you could repoint that text pointer!

With Cartographer, a command-line tool, you can indicate a ROM, a TBL file, and then tell the program to start extracting text. You could tell it to extract in bulk text and garbage data alike from and until given addresses (RAW mode). But if you found a pointer, or even better, a succession of pointers (also called pointer tables), like in the Pokémon example above, you can extract text data in a more organized manner to a text file.

With Atlas, another command-line tool, assuming you went with the second (and more proper) method, you may re-insert that text file (after you modified it) in the ROM, and the tool will take care of updating the pointers. You can for example tell it to start inserting text in an empty yet comfortable (and more importantly, within reach - this depends on the pointer) location in the end of ROM. The possibilities are endless.

Command-line tools may sound scary, with their black MS-DOS dialog boxes flashing quickly and disappearing. They're actually easy to use. Just put all the needed files in the same folder, make a new txt file where you write the command detailed in the tool's readme, and save that file as one with the .bat extension. If you double-click on it, it will execute that tool the way you wanted it with little effort.

Graphical Editing with Tile Editors

Hex editors interpret binary data as raw bytes or text. Tile editors do something similar, but instead interpret binary data as graphics. Of course, this means if you're using the wrong mode or looking at an area that's not supposed to be graphical data or the graphics are compressed, you'll just see garbage.

Among the best tile editors out there are TileMolester and Crystaltile2. For newer 3D-based systems however, they may not help by much and then specialized graphical converters will be needed.

In older systems, in order to save space, graphics were usually stored in parts:

First there's the tile data: The actual drawings, and what you may edit with tile editors. They're small pieces that when assembled make a big picture.

The instructions to build the big picture are tile maps (in the case of backgrounds) or sprite attribute tables (for sprites).

The tile data uses indexed colors actually, so in Mario's example his shirt isn't colored "red", but rather colored with "color 1". NES Mario games, like most NES games, happen to use the NES 2BPP mode, that is the 2 bits per pixel mode. A bit is 0 or 1. With two bits we can write 00, 01, 10 and 11... so 4 possible colors in total. Or actually 3, since "color 0" is transparency, used around Mario's sprite so that the backgrounds behind him are visible unobstructed.

How can we tell what color Mario's shirt is, since it's just "color 1"? Palettes. Palettes are hex data consisting of 3 bytes, and each byte is the ID for a specific color: "red", "blue", "yellow", "purple", "light blue", and a few others (just under 60 valid choices in the NES, but way more in later systems)...

We know Mario's clothes change color after eating various items. But the tile data drawings are stored just once in the ROM. There's separate palettes telling the game to colorize the same drawing differently each time.

General Resources

  • ROMHacking.net - This is like a hub where the various hacking communities meet.  It hosts a large variety of major ROM hacks and translations.  Also hosts numerous FAQs and tools to help aspiring hackers get started.
  • Data Crystal - A wiki hosted by ROMHacking.net.  While it is a bit outdated at this point, it is still a good resource for information about different editors and links to a handful of prominent hacks.
  • FuSoYa's Niche - Site of the creator of the popular Super Mario World editor, Lunar Magic.  Also hosts a set of tools for ROM expansion, patching, compression, etc.
  • Zophar's Domain - A site that hosts lots of smaller patches, such as spoofs, as well as a significant amount of major ones.  Hosts a lot of content that can't be found on ROMHacking.net

Programs

General purpose 2D graphics/tile editor

Tile Molester. Works with Java, practically can edit any game.


Hex Editor

XVI32 or Translhextion. You can edit the bytes of the ROMs, i.e. everything. Some values are obvious and some text is already ASCII.

Game Specific

Super Mario World

MainLunarMagic-1-.png
  • Super Mario World Central - The largest site dedicated to SMW hacking.  Hosts hundreds of hacks and is a resource for SMW hacking utilities and knowledge.  Also has a very active community and forums.
  • X-Mario - Prominent Japanese site that hosts a variety of hacks.  Worth noting is that Japanese hacks are usually more reliant on creating challenging gameplay and are less flashy or graphically modified than their Western cousins.
  • Raocow's Talkhaus - The community for the most prominent let's player in SMW hacking.  Whether or not you enjoy his commentary, this site remains a great resource for discovering hacks, as the community here is generally focused on hacks that the SMW Central community doesn't cover.
  • VIP Wiki - Japanese wiki dedicated to the development of 2channel's series of popular hacks, the VIP & Wall Mix series.  Hosts the 5 current installments and news about the sixth, currently a work in progress.

Super Metroid

Fetch-1-.png
  • Metroid Construction - The most prominent and active Super Metroid hacking community currently.  Hosts a large variety of hacks, resources, FAQs, and an active community and forum.  Originated in m2k2 before becoming its own dedicated site.
  • Metroid Construction Wiki - As its name implies, a wiki created by the Metroid Construction community.
  • Metroid 2002 - The former most prominent and active Super Metroid hacking community previously.  While it has since been foregone in favor of Metroid Construction, it can still be a valuable resource for knowledge about the more advanced mechanics and inner workings of Super Metroid (as well as other games in the series).

The Legend of Zelda: A Link to the Past

  • Zelda Construction - Despite its huge popularity, the hacking community for ALttP is relatively small.  This is mainly due to the lack of a truly good editor, though one is in the works.  This community is a spin-off of Metroid Construction devoted to hacking of ALttP and other games in the series.

EarthBound

  • Starmen.net - Home of the hacking utility PK Hack, an editor for modifying the EarthBound/Mother 2 ROM.  Has an active hacking community in the PK Hack section of its forums, where links to completed hacks can also be found.

Yoshi's Island

  • Super Mario World Central - In addition to Super Mario World, smwcentral is a budding hub for Yoshi's Island hacking.  While it only hosts a few complete hacks, the community is very active, and new content is being produced at a consistent pace.
  • YI Hacking Wiki - A knowledge base for Yoshi's Island hacking.

Final Fantasy VI

  • FF6 Hacking - Home to a large community of Final Fantasy VI hackers, complete with active forums, links to patches, and FAQs and guides for those looking to get into hacking the game.

Final Fantasy Tactics

Fire Emblem

  • Fire Emblem Shrine - An active and prominent Fire Emblem hacking community.  Hosts a variety of completed hacks and FAQs to get you started.
  • Serenes Forest - Another active and prominent Fire Emblem hacking community, though perhaps a bit more active than Fire Emblem Shrine.

Pokémon

Sonic the Hedgehog

  • Sonic Retro - The largest resource for hacks and modifications of the Sonic series (primarily focused on the Genesis games).

Downloads

ROM Hacks