Editing Save disk space for ISOs

Jump to navigation Jump to search

Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision Your text
Line 1: Line 1:
Disc images (commonly known as ISOs, but ISO is actually a specific format) are faithful software recreations of game discs (when made correctly). However, with disc sizes ranging from 700 MB (CD), 900 MB (PSP single-layered UMD), 1 GB (DC GD), 1.46 GB (GCN Mini-DVD), 1.8 GB (PSP dual-layered UMD), 4.7 GB (single-layered DVD), 8.5 GB (dual-layered DVD), 25 GB (single-layered Blu-Ray), and 50 GB (dual-layered Blu-Ray), they can get pretty taxing for storage, especially when newer generations of consoles games are getting bigger in file sizes.  
+
Disc images (commonly known as ISOs, but ISO is actually a specific format) are faithful software recreations of game discs (when made correctly). However, with disc sizes ranging from 700 MB (CD), 1.4 GB (GC Mini-DVD), 4.7 GB (single-layered DVD), and 25 GB (Blu-Ray), they can get pretty taxing for storage, especially when newer generations of consoles games are getting bigger in file sizes.  
  
 
It wouldn't be so bad if not for the fact that the game data itself is often times only a fraction of the actual disc size - for instance, the ''Super Mario 25th Anniversary'' Wii disc itself is a 4.7GB, when really the actual game data is only a single SNES ROM (12 MB of useful data, to be precise) and nothing else. Naturally, one would want to trim this extra "fat" as much as possible, which is what this page aims to help to achieve.  Most of the information here is based partially on this [https://www.reddit.com/r/emulation/comments/3g933n/guide_reduce_the_size_of_your_ps2_gc_wii_x360_ds/ guide].
 
It wouldn't be so bad if not for the fact that the game data itself is often times only a fraction of the actual disc size - for instance, the ''Super Mario 25th Anniversary'' Wii disc itself is a 4.7GB, when really the actual game data is only a single SNES ROM (12 MB of useful data, to be precise) and nothing else. Naturally, one would want to trim this extra "fat" as much as possible, which is what this page aims to help to achieve.  Most of the information here is based partially on this [https://www.reddit.com/r/emulation/comments/3g933n/guide_reduce_the_size_of_your_ps2_gc_wii_x360_ds/ guide].
Line 17: Line 17:
 
:* BIN/ISO is the full disc data, including Audio-CD sound data and game data
 
:* BIN/ISO is the full disc data, including Audio-CD sound data and game data
 
:* CUE is the datasheet file
 
:* CUE is the datasheet file
: '''Light Dump:''' ISO + MP3/FLAC/WAV + CUE
+
: '''Light Dump:''' ISO + MP3/WAV + CUE
 
:* ISO is the disc data with only the game data
 
:* ISO is the disc data with only the game data
:* MP3/FLAC/WAV is the sound data from the Audio-CD, but these formats take much less disk space
+
:* MP3/WAV is the sound data from the Audio-CD, but these formats take much less disk space
 
:* CUE is the datasheet file
 
:* CUE is the datasheet file
  
Line 27: Line 27:
 
* '''Gain:''' Several hundreds of MBs to just a few dozens, depending on how much this specific game relies on the Audio-CD sound format
 
* '''Gain:''' Several hundreds of MBs to just a few dozens, depending on how much this specific game relies on the Audio-CD sound format
 
* '''Tools Used:''' Load the BIN+CUE using a virtual drive, then use a CD dumping tool
 
* '''Tools Used:''' Load the BIN+CUE using a virtual drive, then use a CD dumping tool
* '''Can be reverted?''' Yes, just burn the ISO+MP3/FLAC/WAV+CUE again using a CD burner tool (ImgBurner) either to a physical disk or as an ISO+BIN file. Lossy audio formats will result in data loss.
+
* '''Can be reverted?''' Yes, just burn the ISO+MP3/WAV+CUE again using a CD burner tool (ImgBurner) either to a physical disk or as an ISO+BIN file. Lossy audio formats will result in data loss.
 
* '''Playable on Hardware?''' No, but can be reverted to be
 
* '''Playable on Hardware?''' No, but can be reverted to be
* '''Playable on Emulators?''' Anything that supports loading from .cue files, or direct from the optical drive (if you use a virtual disc loader like CDEmu). Some aren't compatible with MP3, if that's the case, convert them to WAV with MP32WAV. You may need Sega Cue Maker.
+
* '''Playable on Emulators?''' Yes (use virtual drive if needed). Some aren't compatible with MP3, if that's the case, convert them to WAV with MP32WAV. You may need Sega Cue Maker.
  
 
Examples:
 
Examples:
Line 49: Line 49:
  
 
===CHD Compression===
 
===CHD Compression===
* '''Archive-quality dump?''' No (because of bugs and missing unimplemented directives<ref>http://problemkaputt.de/psxspx-cdrom-disk-images-chd-mame.htm#CHD_Notes</ref><ref>http://github.com/mamedev/mame/issues/10308</ref>)
+
* '''Archive-quality dump?''' Yes (with CHD version 5)
 
* '''Gain:''' Immediate
 
* '''Gain:''' Immediate
 
* '''Tools Used:'''  chdman (included with MAME)
 
* '''Tools Used:'''  chdman (included with MAME)
 
* '''Can be reverted?''' Yes, using extractcd (included with MAME)
 
* '''Can be reverted?''' Yes, using extractcd (included with MAME)
 
* '''Playable on Hardware?''' No.
 
* '''Playable on Hardware?''' No.
* '''Playable on Emulators?''' MAME, DuckStation, PCSX2 (since March of 2021), DEmul and NO$PSX. Some libretro cores for other emulators are starting to add support.
+
* '''Playable on Emulators?''' MAME, DuckStation, PCSX2 (since March of 2021) and DEmul. Some libretro cores for other emulators are starting to add support.
 
* '''Can process multi track bin files?''' Yes.
 
* '''Can process multi track bin files?''' Yes.
  
MAME uses the CHD format for disc images in general and includes tools to convert back and forth. Before MAME v145, CHD was in version 4 and it bumped to version 5 from MAME v146 and further. The CHDv5 uses 7zip's LZMA compression on the game data and lossless FLAC compression for the audio data to optimize compression even further than using BIN+CUE+MP3/WAV data separation alone. CHDv5 is lossy until all the bugs are resolved.
+
MAME uses the CHD format for disc images in general and includes tools to convert back and forth. Before MAME v145, CHD was in version 4 and it bumped to version 5 from MAME v146 and further. The CHD v5 uses 7zip's LZMA compression on the game data and lossless FLAC compression for the audio data to optimize compression even further than using BIN+CUE+MP3/WAV data separation alone. CHDv5 is truely lossless.
  
 
'''Instructions'''
 
'''Instructions'''
Place the chdman executable and extractcd in the same directory as the dumps you want to compress (dumps must be in BIN+CUE format or GDI).  Open Command Prompt or a terminal emulator and navigate to the directory where you placed the chdman executable and input one of the following:
+
Place chdman.exe and extractcd in the same directory as the dumps you want to compress (dumps must be in BIN+CUE format or GDI).  Open Command Prompt and navigate to the directory where you placed chdman.exe and input one of the following:
* BIN/CUE-GDI to CHD:
+
* BIN/CUE-GDI to CHD: <code>for /R %i in (*.cue, *.gdi) do chdman createcd -i "%i" -o "%~ni.chd"</code> (Windows)
for /R %i in (*.cue, *.gdi) do chdman createcd -i "%i" -o "%~ni.chd" (Windows)
+
* CHD to BIN/CUE: <code>for /R %i in (*.chd) do chdman extractcd -i "%i" -o "%~ni.cue"</code> (Windows)
for i in *.(cue|gdi); do chdman createcd -i "$i" -o "${i%.*}.chd"; done (Linux)
 
* CHD to BIN/CUE:  
 
for /R %i in (*.chd) do chdman extractcd -i "%i" -o "%~ni.cue" (Windows)
 
for i in *.chd; do chdman extractcd -i "$i" -o "${i%.*}.cue"; done (Linux)
 
 
Alternatively, if you only need to do one file you can use this: <code>chdman createcd -i "<FILENAME>.cue" -o "<FILENAME>.chd"</code>
 
Alternatively, if you only need to do one file you can use this: <code>chdman createcd -i "<FILENAME>.cue" -o "<FILENAME>.chd"</code>
  
 
If you have one of the European PSX games that features LibCrypt copy protection, then you will have a .sbi file in addition to the .bin/cue file.  You will still need to have the .sbi file in the same directory as the game file (in this case, the newly created CHD file) in order to run.
 
If you have one of the European PSX games that features LibCrypt copy protection, then you will have a .sbi file in addition to the .bin/cue file.  You will still need to have the .sbi file in the same directory as the game file (in this case, the newly created CHD file) in order to run.
 
Note that multi-track bin files will be combined and chdman will not split the files back when extracting.
 
  
 
==PlayStation 1==
 
==PlayStation 1==
Line 86: Line 80:
 
* '''Archive-quality dump?''' No
 
* '''Archive-quality dump?''' No
 
* '''Gain:''' Immediate (ISO size decreases a lot).
 
* '''Gain:''' Immediate (ISO size decreases a lot).
* '''Tools Used:''' [https://www.reddit.com/r/PSP/wiki/psx2psp PSX2PSP], Popstation MD GUI, [https://github.com/sahlberg/pop-fe POP-fe] (PS3 friendly), ...
+
* '''Tools Used:''' [https://www.reddit.com/r/PSP/wiki/psx2psp PSX2PSP], Popstation MD GUI, [https://www.psx-place.com/threads/w-i-p-utilities-cdda-enabler-for-psx-eboots-on-ps3.23539/ CDDA-ENABLER] (optimized for PS3) ...
 +
 
 
* '''Can be reverted?''' Yes. For homemade EBOOTs it can be done by using the same tool to generate the BIN+CUE files. For commercial releases from PSN you will have to use [https://github.com/xdotnano/PSXtract PSXtract]. There's data loss, but it's negligible in terms of functionality.
 
* '''Can be reverted?''' Yes. For homemade EBOOTs it can be done by using the same tool to generate the BIN+CUE files. For commercial releases from PSN you will have to use [https://github.com/xdotnano/PSXtract PSXtract]. There's data loss, but it's negligible in terms of functionality.
 
* '''Playable on Hardware?''' On PSP, not on PS1.
 
* '''Playable on Hardware?''' On PSP, not on PS1.
Line 94: Line 89:
  
 
===CHD===
 
===CHD===
* '''Archive-quality dump?''' No (because of bugs and unimplemented directives for TOC and CUE)
+
* '''Archive-quality dump?''' Yes
 
* '''Gain:''' Immediate (better than PBP).
 
* '''Gain:''' Immediate (better than PBP).
 
* '''Tools Used:'''  chdman
 
* '''Tools Used:'''  chdman
 
* '''Can be reverted?''' Yes, using the same tool (chdman).
 
* '''Can be reverted?''' Yes, using the same tool (chdman).
 
* '''Playable on Hardware?''' No.
 
* '''Playable on Hardware?''' No.
* '''Playable on Emulators?''' libretro Beetle PSX, Avocado, DuckStation, PCSX2 (since March of 2021 and also because PCSX2 now has PSOne emulation functionality as well, so two-in-one), PCSX ReARMed and NO$PSX.
+
* '''Playable on Emulators?''' libretro Beetle PSX, Avocado, DuckStation, PCSX2 (since March of 2021, and also because PCSX2 now has PSOne emulation functionality as well, so two-in-one) and PCSX ReARMed.
  
 
==PlayStation 2==
 
==PlayStation 2==
Line 122: Line 117:
  
 
===CSO (aka CISO)===
 
===CSO (aka CISO)===
* '''Archive-quality dump?''' Yes
+
* '''Archive-quality dump?''' No (missing data)
 
* '''Gain:''' Immediate (ISO dump size the same). Coupled with manual scrubbing, it can be bigger. Not as much of a gain as GZIP though.
 
* '''Gain:''' Immediate (ISO dump size the same). Coupled with manual scrubbing, it can be bigger. Not as much of a gain as GZIP though.
 
* '''Tools Used:''' maxcso
 
* '''Tools Used:''' maxcso
Line 128: Line 123:
 
* '''Playable on Hardware?''' NO
 
* '''Playable on Hardware?''' NO
 
* '''Playable on Emulators?''' Yes - PCSX2.
 
* '''Playable on Emulators?''' Yes - PCSX2.
 
===ZSO (aka ZISO)===
 
* '''Archive-quality dump?''' Yes
 
* '''Gain:''' Immediate (ISO dump size the same). Coupled with manual scrubbing, it can be bigger. Not as much of a gain as GZIP though.
 
* '''Tools Used:''' [https://github.com/unknownbrackets/maxcso#features maxcso], [https://github.com/ps2homebrew/Open-PS2-Loader/blob/master/pc/ziso.py ziso.py (requires python3 and pip install lz4)], [https://www.psx-place.com/resources/psxiso-compression-tool.1258/ PSXiSO (contains ziso CLI tool)]
 
* '''Can be reverted?''' Yes, using the same tool. No data loss.
 
* '''Playable on Hardware?''' Yes - Open PS2 Loader.
 
* '''Playable on Emulators?''' Yes - PCSX2 (since v1.7.5293)<ref>[https://github.com/PCSX2/pcsx2/releases/tag/v1.7.5293 v1.7.5293]</ref>
 
  
 
===CHD===
 
===CHD===
Line 143: Line 130:
 
* '''Can be reverted?''' Yes, using the same tool (chdman).
 
* '''Can be reverted?''' Yes, using the same tool (chdman).
 
* '''Playable on Hardware?''' No.
 
* '''Playable on Hardware?''' No.
* '''Playable on Emulators?''' Yes - PCSX2 (since v1.7.0-dev-1156)<ref>Added in 1150 but wasn't built. 1156 is the first release with CHD support. [https://web.archive.org/web/20210416020418/https://buildbot.orphis.net/pcsx2/index.php?m=fulllist Automated Pcsx2 builds]</ref>
+
* '''Playable on Emulators?''' PCSX2 (since v1.7.0-dev-1156<ref>Added in 1150 but wasn't built. 1156 is the first release with CHD support. https://buildbot.orphis.net/pcsx2/index.php?m=fulllist</ref>).
  
 
==PlayStation 3==
 
==PlayStation 3==
Line 155: Line 142:
  
 
For a long time (before the availability of Cobra and Mamba) the only playable format for PS3 backups (relying on little more than "peek and poke" CFW-provided syscalls to mount the game's folder over the currently inserted disc) but also the least accurate one, with significant compatibility issues (varying on whether the backup is stored on an internal or external drive, whether an original game disc is currently inserted, various optional hacks such as "BDMirror" moving the files to the root of the external drive on demand, ...)
 
For a long time (before the availability of Cobra and Mamba) the only playable format for PS3 backups (relying on little more than "peek and poke" CFW-provided syscalls to mount the game's folder over the currently inserted disc) but also the least accurate one, with significant compatibility issues (varying on whether the backup is stored on an internal or external drive, whether an original game disc is currently inserted, various optional hacks such as "BDMirror" moving the files to the root of the external drive on demand, ...)
 
===ISO support for RPCS3===
 
Currently [[Save_disk_space_for_ISOs#External_links|there is no direct ISO or any compressed file support for RPCS3]] (your only option is mount ISO image to boot the game with RPCS3).
 
 
*You can either use [https://github.com/RPCS3/rpcs3/issues/4021#issuecomment-1650682400 this tool], or [https://github.com/ptmorris1/RPCS3-ISOLauncher-Launchbox this one] for playing directly your ISO dumps with RPCS3.
 
 
*or use [https://www.mediafire.com/file/v5yslivyurr5luy/Make_sure_edit_path_locations_first.bat/file this simple batch script] ('''edit path locations first of course''') which mounts your ISO using powershell commands and launching it using RPCS3 CLI commands and ejecting it when you done with RPCS3
 
  
 
==PlayStation Portable==
 
==PlayStation Portable==
Line 188: Line 168:
  
 
Official format for digitally distributed PSP/minis (partially related to the one used for PS1 games), containing a compressed disc image and optionally a custom boot logo.
 
Official format for digitally distributed PSP/minis (partially related to the one used for PS1 games), containing a compressed disc image and optionally a custom boot logo.
 
===CHD===
 
* '''Archive-quality dump?''' Yes
 
* '''Gain:''' Immediate (Less than PBP).
 
* '''Tools Used:'''  chdman
 
* '''Can be reverted?''' Yes, using the same tool (chdman).
 
* '''Playable on Hardware?''' No.
 
* '''Playable on Emulators?''' Yes - PPSSPP.
 
  
 
==PlayStation Vita==
 
==PlayStation Vita==
Line 217: Line 189:
 
* '''Can be reverted?''' No.
 
* '''Can be reverted?''' No.
 
* '''Playable on Hardware?''' Yes - with the NoNpDRM plugin.
 
* '''Playable on Hardware?''' Yes - with the NoNpDRM plugin.
* '''Playable on Emulators?''' Yes - Vita3K
+
* '''Playable on Emulators?''' ?
  
 
NoNpDRM is a Vita plugin that generates decrypted licenses (valid for any console) when running an original Vita title, as well as allowing the system to accept those licenses, allowing for playing the encrypted files directly copied from a game card or memory card (by extension the name is therefore also used for such files, or the combination of the game's files and a decrypted license).
 
NoNpDRM is a Vita plugin that generates decrypted licenses (valid for any console) when running an original Vita title, as well as allowing the system to accept those licenses, allowing for playing the encrypted files directly copied from a game card or memory card (by extension the name is therefore also used for such files, or the combination of the game's files and a decrypted license).
Line 241: Line 213:
  
 
However compressed formats incorporating the "padding zeroing" part were made since then, and Dolphin supports them! These are the WBFS and GCZ formats.
 
However compressed formats incorporating the "padding zeroing" part were made since then, and Dolphin supports them! These are the WBFS and GCZ formats.
 
Dolphin can't play games directly from compressed archives (7zip/zip/gzip/rar...). Instead, it utilizes its own compression method.
 
  
 
===GCZ (Dolphin native archive format) - GC/Wii===
 
===GCZ (Dolphin native archive format) - GC/Wii===
  
'''This format has been deprecated in favor of [[#RVZ (Modern Dolphin format) - GC/Wii|RVZ]].'''
+
Dolphin can't play games directly from compressed archives (7zip/zip/gzip/rar...). Instead, it utilizes its own compression method. '''This format has been deprecated in favor of [[#RVZ (Modern Dolphin format) - GC/Wii|RVZ]].'''
  
* '''Archive-quality dump?''' Yes (Not directly if scrubbed; see reverting notes)
+
* '''Archive-quality dump?''' Not directly (can be restored with NKit).
* '''Gain:''' Immediate. Depends on the game. It is very minimal if junk data is not removed (scrubbed).
+
* '''Gain:''' Immediate (ISO dump size lowers). Considerable, depending on the game.
* '''Tools Used:''' Dolphin (right-click the game(s) in the games list and select "Convert File..." (or "Convert Selected Files..." if more than one is selected), and select GCZ), or [https://wit.wiimm.de/ wit (Wiimms ISO Tools)], Nkit (for indirect recovery)
+
* '''Tools Used:''' Dolphin (right-click the game(s) in the games list and select "Compress ISO..." (or "Compressed selected ISOs..." if more than one is selected)), [https://wiki.gbatemp.net/wiki/NKit NKit], or [https://wit.wiimm.de/ wit (Wiimms ISO Tools)]
* '''Can be reverted?''' Yes (Directly for non-scrubbed files. For scrubbed files, first convert to ISO, then use Nkit tool's "Recover to ISO" option)
+
* '''Can be reverted?''' Yes. You can right-click on the GCZ-compressed (in blue) ISOs in Dolphin and choose "Uncompress selected ISOs". Garbage data for Wii titles will be zeroed out and thus different from the official dump but will have no effect on gameplay in most cases - if desired can be unscrubbed [https://gbatemp.net/threads/new-app-nkit-restore-shrink-and-preserve-disc-images-in-playable-formerly-swiit.533402/ NKitRestore].
 
* '''Playable on Hardware?''' No
 
* '''Playable on Hardware?''' No
* '''Playable on Emulators?''' Yes (Dolphin)
+
* '''Playable on Emulators?''' Only Dolphin.
 +
 
 +
Some examples:
 +
* Super Mario Anniversary (Wii): 4.7 GB (original) > 12 MB (GCZ) (!!)
 +
* Super Smash Bros Brawl (Wii): 7.8 GB (original) > 7.1 GB (GCZ) (main cause is FMV's low compression)
 +
* Xenoblade PAL (Wii): 7.8 GB (original) > 6.3 GB (GCZ)
 +
* Super Mario Galaxy 2 (Wii): 4.7 GB (original) > 1.8 GB (GCZ)
 +
* Tales of Graces (Wii): 4.7 GB (original) > 4.2 GB (GCZ) (game data already fills the disk)
 +
* Animal Crossing (GC): 1.4 GB (original) > 26 MB (GCZ)
 +
* Zelda Four Swords Plus Japan (GC): 1.4 GB (original) > 480 MB (GCZ)
 +
* Zelda Twilight Princess (GC): 1.4 GB (original) > 1.4 GB (GCZ) (game data already fills the disk)
 +
* Megaman Collection (GC): 1.4 GB (original) > 1.1 GB (GCZ) (sound data is stored as uncompressed stream to fill disk)
  
 
===RVZ (Modern Dolphin format) - GC/Wii===
 
===RVZ (Modern Dolphin format) - GC/Wii===
  
The Dolphin team developed a new compression format based on WIA called RVZ. Usage is very similar to GCZ in Dolphin itself, but it only works on newer Dolphin versions. It can efficiently compress both the actual data and junk data, without altering the file/directory structure.
+
The Dolphin team developed a new compression format based on WIA called RVZ. Unlike all the previous formats, RVZ is lossless and can preserve the padding data on Wii discs as well as the necessary files needed by the Wii's IOS. Usage is very similar to GCZ in Dolphin itself, but it only works on newer Dolphin versions.
  
* '''Archive-quality dump?''' Yes
+
* '''Archive-quality dump?''' Yes.
* '''Gain:''' Immediate. Considerable, depending on the game.
+
* '''Gain:''' Immediate (ISO dump size lowers). Considerable, depending on the game.
* '''Tools Used:''' Dolphin 5.0-12188+ (right-click the game(s) in the game list and select "Convert File..." (or "Convert Selected Files..." if more than one is selected), and select RVZ)
+
* '''Tools Used:''' Dolphin 5.0-12188+ (right-click the game(s) in the games list and select "Compress ISO..." (or "Compressed selected ISOs..." if more than one is selected))
* '''Can be reverted?''' Yes (In Dolphin, right-click the game, select "Convert File..." and select ISO)
+
* '''Can be reverted?''' Yes.
* '''Playable on Hardware?''' No
+
* '''Playable on Hardware?''' No.
* '''Playable on Emulators?''' Yes (Dolphin 5.0-12188+)
+
* '''Playable on Emulators?''' Only Dolphin.
  
 
===Scrubbing and Trimming - GC/Wii===
 
===Scrubbing and Trimming - GC/Wii===
Line 271: Line 252:
 
* '''Tools Used:''' GC-Tool ("wipe garbage data..." - Scrubbing for GameCube), GameCube ISO Trimmer (scrubbing and trimming for GameCube), WiiScrubber (scrubbing/trimming for Wii, although compressing with Dolphin will scrub it anyways).
 
* '''Tools Used:''' GC-Tool ("wipe garbage data..." - Scrubbing for GameCube), GameCube ISO Trimmer (scrubbing and trimming for GameCube), WiiScrubber (scrubbing/trimming for Wii, although compressing with Dolphin will scrub it anyways).
 
* '''Can be reverted?''' Yes, with WiiScrubber's undo file or NKit.
 
* '''Can be reverted?''' Yes, with WiiScrubber's undo file or NKit.
* '''Playable on Hardware?''' Yes for scrubbed; some games won't work trimmed.
+
* '''Playable on Hardware?''' Yes for scrubbing; some games won't work trimmed.
* '''Playable on Emulators?''' Yes for scrubbed; some games won't work trimmed.
+
* '''Playable on Emulators?''' Yes for scrubbing; some games won't work trimmed.
  
 
Scrubbing games zeros out garbage/dummy data in the ISO file. The resulting file will be the same size, but it will be able to compress better than unmodified ISOs. The difference can be huge depending on the game. Animal Crossing, for instance, will compress to just a 26 MB .gcz file after being scrubbed!
 
Scrubbing games zeros out garbage/dummy data in the ISO file. The resulting file will be the same size, but it will be able to compress better than unmodified ISOs. The difference can be huge depending on the game. Animal Crossing, for instance, will compress to just a 26 MB .gcz file after being scrubbed!
Line 279: Line 260:
  
 
===WBFS - Wii only===
 
===WBFS - Wii only===
* '''Archive-quality dump?''' Not directly; see reverting notes.
+
* '''Archive-quality dump?''' Not directly, see note about reversing.
 
* '''Gain:''' Immediate (ISO dump size lowers). Considerable, depending on the game.
 
* '''Gain:''' Immediate (ISO dump size lowers). Considerable, depending on the game.
 
* '''Tools Used:''' [http://www.wiibackupmanager.co.uk/downloads.html Wii Backup Manager] (after adding your game, go to "Transfer" and select "WBFS") or wit (Wiimms ISO Tools).
 
* '''Tools Used:''' [http://www.wiibackupmanager.co.uk/downloads.html Wii Backup Manager] (after adding your game, go to "Transfer" and select "WBFS") or wit (Wiimms ISO Tools).
 
* '''Can be reverted?''' Maybe. WBFS files can always be converted back to ISO, but they will remain scrubbed (can be unscrubbed with NKitRestore) and, depending on settings at the time of conversion to WBFS, may be missing update/extras partitions (can be readded with wit or NKitRestore and a copy of the missing data).
 
* '''Can be reverted?''' Maybe. WBFS files can always be converted back to ISO, but they will remain scrubbed (can be unscrubbed with NKitRestore) and, depending on settings at the time of conversion to WBFS, may be missing update/extras partitions (can be readded with wit or NKitRestore and a copy of the missing data).
 
* '''Playable on Hardware?''' Yes - Preferred format by most SD/USB loaders
 
* '''Playable on Hardware?''' Yes - Preferred format by most SD/USB loaders
* '''Playable on Emulators?''' Yes (Dolphin)
+
* '''Playable on Emulators?''' Yes - Dolphin
  
 
A scrubbed and sparse (gap-dropping) format directly derived from the one used in the obsolete WBFS file system.
 
A scrubbed and sparse (gap-dropping) format directly derived from the one used in the obsolete WBFS file system.
Line 296: Line 277:
 
* '''Can be reverted?''' Maybe (same caveats as WBFS).
 
* '''Can be reverted?''' Maybe (same caveats as WBFS).
 
* '''Playable on Hardware?''' Yes (on Wii/Vwii only) - Nintendont for Gamecube games, uLoader for Wii games
 
* '''Playable on Hardware?''' Yes (on Wii/Vwii only) - Nintendont for Gamecube games, uLoader for Wii games
* '''Playable on Emulators?''' Yes (Dolphin)
+
* '''Playable on Emulators?''' Yes - Dolphin
  
 
A scrubbed and sparse (gap-dropping) format. Unrelated to the PSP's CSO format.
 
A scrubbed and sparse (gap-dropping) format. Unrelated to the PSP's CSO format.
Line 304: Line 285:
 
===FST (extracted File System) - GC/Wii===
 
===FST (extracted File System) - GC/Wii===
  
* '''Archive-quality dump?''' No (Yes, if not scrubbed)
+
* '''Archive-quality dump?''' No
 
* '''Gain:''' Very low with GC ISOs, much better with Wii ones. Can also be combined with classic archives compression (ZIP/RAR/7z etc.) to match WIA and NKit compression ratios.
 
* '''Gain:''' Very low with GC ISOs, much better with Wii ones. Can also be combined with classic archives compression (ZIP/RAR/7z etc.) to match WIA and NKit compression ratios.
 
* '''Tools Used:''' [https://wit.wiimm.de/info/composing.html wit]
 
* '''Tools Used:''' [https://wit.wiimm.de/info/composing.html wit]
 
* '''Can be reverted?''' Partially via the align-files.txt created by wit when extracting.
 
* '''Can be reverted?''' Partially via the align-files.txt created by wit when extracting.
* '''Playable on Hardware?''' GC games on Wii/Vwii: Yes ([https://github.com/FIX94/Nintendont/ Nintendont]) / Other combinations: No
+
* '''Playable on Hardware?''' (GC games on Wii/Vwii): Yes - [https://github.com/FIX94/Nintendont/ Nintendont]. (Other combinations): No
* '''Playable on Emulators?''' Yes (Dolphin)
+
* '''Playable on Emulators?''' Yes - Dolphin (both GC and Wii)
  
 
This has the advantage of easy experimenting with [[#Destructive Modification]]. Includes all disadvantages of trimming.
 
This has the advantage of easy experimenting with [[#Destructive Modification]]. Includes all disadvantages of trimming.
  
===WIA (Wii ISO Archive) - GC/Wii===
+
===WIA (Wii ISO Archive) - Wii only?===
* '''Archive-quality dump?''' Not directly; see reverting notes (Yes, if not scrubbed)
+
* '''Archive-quality dump?''' Not directly unless --raw option is given, effectively preventing any space savings.
* '''Gain:''' Immediate (ISO dump size lowers); more than WBFS. (No gain if not scrubbed and compression not used)
+
* '''Gain:''' Immediate (ISO dump size lowers). More than WBFS.
* '''Tools Used:''' wit, Dolphin 5.0-12188+, Nkit (for indirect recovery)
+
* '''Tools Used:''' wit
* '''Can be reverted?''' Yes (Directly, if not scrubbed. Not directly, if scrubbed, by converting it to ISO first, and then using Nkit tool's "Recover to ISO" option)
+
* '''Can be reverted?''' Maybe (same caveats as WBFS).
 
* '''Playable on Hardware?''' No
 
* '''Playable on Hardware?''' No
* '''Playable on Emulators?''' Yes (Dolphin 5.0-12188+)
+
* '''Playable on Emulators?''' Yes
  
A scrubbed and compressed format designed for maximum space savings without cutting corners on corruption detection, but it's not directly editable. Never really caught on and may be considered de facto deprecated by NKit.
+
A scrubbed and compressed format designed for maximum space savings without cutting corners on corruption detection, but it's not directly playable or editable. Never really caught on and may be considered de facto deprecated by NKit.
  
 
===NKit formats - GC/Wii===
 
===NKit formats - GC/Wii===
'''This format has been deprecated in favor of [[#RVZ (Modern Dolphin format) - GC/Wii|RVZ]]. ([https://github.com/Nanook/NKit/wiki Developer's Wiki])'''
+
* '''Archive-quality dump?''' Not directly, can be restored with NKit
* '''Archive-quality dump?''' Yes (Not for Wii games, if the Wii update partition is removed and the recovery file not preserved)
+
* '''Gain:''' Immediate - (Wii) only for GCZ format.
* '''Gain:''' Immediate
+
* '''Tools Used:''' NKit.
* '''Tools Used:''' NKit
+
* '''Can be reverted?''' Yes.
* '''Can be reverted?''' Yes (unless the Wii update partition was removed and the recovery file not preserved)
+
* '''Playable on Hardware?''' (GC) Yes - ISO-based variant only (Wii) No
* '''Playable on Hardware?''' GC: Yes (ISO variant only) / Wii: No
+
* '''Playable on Emulators?''' Yes - Dolphin
* '''Playable on Emulators?''' Yes (Dolphin)
 
  
 
Apart from its previously mentioned unscrubbing/untrimming features mentioned in the above formats' descriptions, NKit is able to convert any ISO (clean dump or otherwise) to and from an [https://wiki.gbatemp.net/wiki/NKit/NKitFormat intermediate format applying various reversible changes] that optimize the image for lossless compression, including full decryption and optionally splitting the often non-unique update partitions to separate files.
 
Apart from its previously mentioned unscrubbing/untrimming features mentioned in the above formats' descriptions, NKit is able to convert any ISO (clean dump or otherwise) to and from an [https://wiki.gbatemp.net/wiki/NKit/NKitFormat intermediate format applying various reversible changes] that optimize the image for lossless compression, including full decryption and optionally splitting the often non-unique update partitions to separate files.
  
NKit.GCZ is less space efficient than NKit.ISO compressed with a powerful general purpose archiver (7zip), however it is directly playable in Dolphin.
+
NKit-GCZ (as well as plain GCZ) is less space efficient than NKit-ISO compressed with a powerful general purpose archiver (7zip), however it is directly playable in Dolphin.
  
It has problems with emulation, such as slower emulated load times, breaking netplay if load times aren't the same, and crashing a few games.
+
But using it for emulation has problems, such as slower emulated load times, breaking netplay if load times aren't the same, and crashing a few games.
  
 
Compressing NKit.ISO with a strong compression format gives the smallest size while being reversible.
 
Compressing NKit.ISO with a strong compression format gives the smallest size while being reversible.
Line 352: Line 332:
 
* '''Can be reverted?''' Yes. The wud tool includes a decompression utility, which recreates the original file.
 
* '''Can be reverted?''' Yes. The wud tool includes a decompression utility, which recreates the original file.
 
* '''Playable on Hardware?''' No (can be converted to an installable format using [https://github.com/FIX94/wud2app wud2app])
 
* '''Playable on Hardware?''' No (can be converted to an installable format using [https://github.com/FIX94/wud2app wud2app])
* '''Playable on Emulators?''' Yes - Cemu (and potentially other [[Wii U emulators]])
+
* '''Playable on Emulators?''' Yes - Future releases of Cemu (and potentially other [[Wii U emulators]])
  
 
By Exzap, Cemu's author ([http://gbatemp.net/threads/wii-u-image-wud-compression-tool.397901/ release thread]). The tool detects duplicate sectors and only stores them once: all the empty ranges end up removed, storing only ranges which contain file or filesystem data. Of course, the original ISO can be reconstituted.
 
By Exzap, Cemu's author ([http://gbatemp.net/threads/wii-u-image-wud-compression-tool.397901/ release thread]). The tool detects duplicate sectors and only stores them once: all the empty ranges end up removed, storing only ranges which contain file or filesystem data. Of course, the original ISO can be reconstituted.
Line 380: Line 360:
 
Ideal format for game modding, less so for playing on console due to Loadiine's intrinsic modus operandi (appearing to the OS as the host title, with potentially different permissions) leading to poor compatibility.
 
Ideal format for game modding, less so for playing on console due to Loadiine's intrinsic modus operandi (appearing to the OS as the host title, with potentially different permissions) leading to poor compatibility.
  
===WUA (Wii U Archive)===
+
==Switch==
* '''Archive-quality dump?''' No (based on extracted format, see above)
 
* '''Gain:''' Immediate (Total size decreases, adds compression to the extracted format).
 
* '''Tools Used:''' Cemu (introduced in v1.27)
 
* '''Can be reverted?''' No (based on extracted format, see above)
 
* '''Playable on Hardware?''' No
 
* '''Playable on Emulators?''' Yes - Cemu (since v1.27)
 
  
Cemu (since 1.27) can convert installed titles to WUA, combining  the title itself, update (if available) and DLC (if available) into one file. The conversion tool is available via the Cemu title manager.
+
See overview of formats [https://yuzu-emu.org/wiki/overview-of-switch-game-formats/ here].
Based on a small testset of 44 titles with updates and dlc, WUA reduced the overall file size by roughly 16%. It also reduced the number of files per title from 3,500 on average to 1.
 
  
==Switch==
 
 
===NSZ===
 
===NSZ===
 
* '''Archive-quality dump?''' Yes
 
* '''Archive-quality dump?''' Yes
Line 399: Line 371:
 
* '''Playable on Hardware?''' No
 
* '''Playable on Hardware?''' No
 
* '''Playable on Emulators?''' No
 
* '''Playable on Emulators?''' No
 
It is possible to merge the game base, update and DLCs on a single NSP with the tool [https://github.com/julesontheroad/NSC_BUILDER NSC_Builder] and compress it to NSZ
 
  
 
==Game Boy Advance / Nintendo DS / Nintendo 3DS==
 
==Game Boy Advance / Nintendo DS / Nintendo 3DS==
Line 455: Line 425:
 
* '''Playable on Hardware?''' Yes. If the rebuilding process didn't damage anything vital.
 
* '''Playable on Hardware?''' Yes. If the rebuilding process didn't damage anything vital.
 
* '''Playable on Emulators?''' Yes - Xenia.
 
* '''Playable on Emulators?''' Yes - Xenia.
 
===ZAR===
 
[https://github.com/xenia-canary/xenia-canary/pull/165 This Pull Request] adds support to [[xenia]] emulator for loading ZAR (ZArchive files) of a decrypted and extracted dump of XBOX 360 game.
 
*The process is basically to dump your disc, extract files using your favorite tool, and create a ZAR file from it using ZArchive tool or [https://github.com/xenia-canary/xenia-canary/pull/299 xenia-canary->create Zar package option].
 
*For digital titles (Xbox 360 Demos, Video DLC and XBLA games etc) you need to extract the contents from its containers using "[https://digiex.net/threads/wxpirs-extract-content-from-xbox-360-demos-video-dlc-and-arcade-game-containers.9464/ wxPirs]" and then create a ZAR file from it using the "[https://github.com/Exzap/ZArchive/releases ZArchive]" tool or xenia-canary->create Zar package option[https://github.com/xenia-canary/xenia-canary/pull/165#issuecomment-1641099753].
 
  
 
=Other Tricks=
 
=Other Tricks=
Line 466: Line 431:
  
 
===NTFS Compression===
 
===NTFS Compression===
You can enable filesystem-level compression (like "NTFS Compression" in Windows) for the directory containing your ISOs/ROMs. This has a very noticeable space gain and doesn't affect the emulator's functionality. It's surprisingly effective.
+
You can enable filesystem-level compression (like "NTFS Compression" in Windows) for the directory containing your ISOs/ROMs. This has a very noticeable space gain and doesn't affect the emulator's functionality. It's surprisingly more effective than many people would like to give this credit.
  
 
* '''Archive-quality dump?''' Yes
 
* '''Archive-quality dump?''' Yes
Line 504: Line 469:
  
 
* '''Xfs'''
 
* '''Xfs'''
* '''Note:''' Without de-duplication it is already quite efficient at storing files.
+
* '''Note:''' Without de-duplication it is already quiet efficient at storing files.
 
** '''ReadWriteable?''' Yes.
 
** '''ReadWriteable?''' Yes.
 
** '''Deduplication?''' Yes, file level via duperemove.
 
** '''Deduplication?''' Yes, file level via duperemove.
Line 528: Line 493:
 
Since this results, in most cases, in very noticeable detrimental effects in gameplay (if the game doesn't crash outright), this is nothing short of mutilating the game image. Sadly enough, some of these dumps make it to sharing sites.
 
Since this results, in most cases, in very noticeable detrimental effects in gameplay (if the game doesn't crash outright), this is nothing short of mutilating the game image. Sadly enough, some of these dumps make it to sharing sites.
  
Avoid resorting to destructive modifications since it can lead to random crashes and unexpected behavior, especially in games with lots of shared assets. [[File hashes|Verify your dumps]] to make sure you don't have these, and if you still want to reduce size, prefer using other methods or uses decompression on demand. One reason to use a destructively modified dump is for burning your own Dreamcast games as GD-ROMs were over a gigabyte in size and CD-Rs top out at 850MB.
+
Avoid resorting to destructive modifications since it can lead to random crashes and unexpected behavior, especially in games with lots of shared assets. [[http://emulation.gametechwiki.com/index.php/File_Hashes Verify your dumps]] to make sure you don't have these, and if you still want to reduce size, prefer using other methods or uses decompression on demand. One reason to use a destructively modified dump is for burning your own Dreamcast games as GD-ROMs were over a gigabyte in size and CD-Rs top out at 850MB.
  
 
* '''Archive-quality dump?''' No (Removes data)
 
* '''Archive-quality dump?''' No (Removes data)
 
=See also=
 
*[[List of filetypes]]
 
 
=External links=
 
*[https://github.com/citra-emu/citra/issues/6013 CHD or ZArchive feature requests for citra]
 
*[https://github.com/RPCS3/rpcs3/issues/4021 CHD or ZArchive feature requests for RPCS3]
 
*[http://web.archive.org/web/20231127115239/https://github.com/yuzu-emu/yuzu/issues/10025#issuecomment-1493873239 CHD or ZArchive feature requests for yuzu and Ryujinx]
 
*[https://github.com/xemu-project/xemu/issues/129 CHD or ZArchive feature requests for xemu]
 
*[https://github.com/86Box/86Box/issues?q=is%3Aissue+is%3Aopen+format+label%3Afeature+ Various image format feature requests for 86Box]
 
*[https://github.com/joncampbell123/dosbox-x/issues/1822 CHD feature request for DOSBox-X]
 
*[https://github.com/schellingb/dosbox-pure/issues/40 CHD feature request for DOSBox Pure]
 
*[https://yuzu-emu.org/entry/yuzu-progress-report-aug-2023/#project-terra yuzu now supports sparse and compressed NCA games]. This means a long list of titles whose eShop releases couldn’t previously boot are now either running or perfectly playable, and their RomFS can be fully dumped.
 
  
 
=References=
 
=References=
  
 
[[Category:FAQs]]
 
[[Category:FAQs]]

Please note that all contributions to Emulation General Wiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see Emulation General Wiki:Copyrights for details). Do not submit copyrighted work without permission!

To edit this page, please answer the question that appears below (more info):

Cancel Editing help (opens in new window)