Changes

Jump to navigation Jump to search

Save disk space for ISOs

5,679 bytes added, 19:34, 31 October 2020
CHD Archive Format
Disc images (commonly known as ISOs , but ISO is actually a specific format) are faithful software recreations of game discs(when made correctly). However, with disc sizes ranging from 700 MB (CD), 1.4 GB (GC Mini-DVD), 4.7 GB (single-layered DVD), and 25 GB (Blu-Ray), they can get pretty taxing for storage, especially when newer generations of consoles games are getting bigger in file sizes.
It wouldn't be so bad if not for the fact that the game data itself is often times only a fraction of the actual disc size - for instance, the ''Super Mario 25th Anniversary'' Wii disc itself is a 4.7GB, when really the actual game data is only a single SNES ROM (12 MB of useful data, to be precise) and nothing else. Naturally, one would want to trim this extra "fat" as much as possible, which is what this page aims to help to achieve. Most of the information here is based partially on this [https://www.reddit.com/r/emulation/comments/3g933n/guide_reduce_the_size_of_your_ps2_gc_wii_x360_ds/ guide].
==Applicable to All Platforms==
===Audio-CD===
Sega-CD, PC-Engine, PlayStation, Sega Saturn... what do these all have in common? They all use a regular CD format! Game developers often stored music and other sounds using the Audio-CD format, but it was terribly inefficient when it comes to disc storage as it also had to store the actual game along with the sound files. To put it in perspective, a 700 MB CD containing nothing but Audio-CD data can hold at most around 80 minutes worth of sound data, meaning games that used a lot of sounds where were limited in size.
Since then developers no longer use Audio-CD format and instead prefer custom audio formats that come included in the "game data" part of the disc. By the time the PS1 generation came, the Audio-CD part was just used for messages like "Don't put this in a CD player!" and little else (exceptions exist, of course!)
* '''Can be reverted?''' Yes, using extractcd (included with MAME)
* '''Playable on Hardware?''' No.
* '''Playable on Emulators?''' Only MAME , DuckStation and DEmul. Some libretro cores for other emulators are starting to add support.
* '''Can process multi track bin files?''' Yes.
'''Instructions'''
Place chdman.exe and extractcd in the same directory as the dumps you want to compress (dumps must be in BIN+CUE format). Open Command Prompt and navigate to the directory where you placed chdman.exe and input one of the following:
* BIN/CUE to CHD: <code>for /R %i in (*.cue) do chdman createcd -i "%i" -o "%~ni.chd"</code> (Windows)* CHD to BIN/CUE: <code>for /R %i in (*.chd) do chdman extractcd -i "%i" -o "%~ni.cue"</code> (Windows)
Alternatively, if you only need to do one file you can use this: <code>chdman createcd -i "<FILENAME>.cue" -o "<FILENAME>.chd"</code>
PBP is the official format used by Sony for the PS1 Classics on PSP and PS3. Audio tracks may be compressed in ATRAC3 or as raw PCM (unsupported on PS3's ps1_netemu, supported on PSP only by the earliest versions of POPS or the newest ones with the [http://wololo.net/talk/viewtopic.php?f=28&t=41330&start=60 cdda_enabler plugin])
 
===CHD===
* '''Archive-quality dump?''' Yes
* '''Gain:''' Immediate (better than PBP).
* '''Tools Used:''' chdman
* '''Can be reverted?''' Yes, using the same tool (chdman).
* '''Playable on Hardware?''' No.
* '''Playable on Emulators?''' libretro Beetle PSX, Avocado, DuckStation and PCSX ReARMed.
==PlayStation 2==
* '''Archive-quality dump?''' No (missing data)
* '''Gain:''' Not Immediate (ISO dump size the same), however when used with compression the gain is really noticeable.
* '''Tools Used:''' UltraISO, USBUtil, ExPERT/Xpert ([http://www.ps2-home.com/forum/viewtopic.php?t=3419 guides here])
* '''Can be reverted?''' Not really. Garbage data here is zeroed out, it's not important and you still have the same disk data structure.
* '''Playable on Hardware?''' No.Yes (after GZIP-decompression) (some games may break)* '''Playable on Emulators?''' Yes.- PCSX2 (some games may break)
You may forego the manual scrubbing part entirely. If you feel adventurous you might want to do it to enhance compression though. Let's get to the meaty part though...
* '''Playable on Hardware?''' NO
* '''Playable on Emulators?''' Yes - PCSX2.
 
==PlayStation 3==
===Extracted files (aka JB format/GAMES)===
* '''Archive-quality dump?''' Not directly (BD filesystem metadata lost).
* '''Gain:''' Immediate (any padding between files is removed).
* '''Tools Used:''' Most on-console file managers and *man homebrews, 7-Zip and other standard ISO extractors on PC.
* '''Can be reverted?''' Yes, using PS3 ISO Rebuilder and an [http://jonnysp.bplaced.net/ IRD file] representing the original file locations. Generic inaccurate JB to ISO conversions possible with makeps3iso (preferred) or genps3iso.
* '''Playable on Hardware?''' Yes - on CFW/HEN
* '''Playable on Emulators?''' Yes - RPCS3
 
For a long time (before the availability of Cobra and Mamba) the only playable format for PS3 backups (relying on little more than "peek and poke" CFW-provided syscalls to mount the game's folder over the currently inserted disc) but also the least accurate one, with significant compatibility issues (varying on whether the backup is stored on an internal or external drive, whether an original game disc is currently inserted, various optional hacks such as "BDMirror" moving the files to the root of the external drive on demand, ...)
==PlayStation Portable==
* '''Gain:''' Immediate (ISO dump size lowers). Substantial, depending on game. Higher compression levels generally produce decreasing space gains while potentially increasing load times. Some tools allow for leaving audio/video content uncompressed.
* '''Tools Used:''' CISO, CISO GUI, maxcso, UMDGen, PSP ISO Compressor, CISO Multi Compressor, CISO XP, ...
<br />CISO GUI (CISO is command line-based): Drag-and-drop ISO file on program, Select compression level (1 to 9 (Max)), "Compress", and choose directory for saving the new CSO file.
<br />UMDGen is a PSP ISO editor. You can save to either the CSO format with this.
* '''Can be reverted?''' Yes, using the same tools and saving in ISO (uncompressed) format. No data loss.
* '''Playable on Hardware?''' Yes - most CFWs (Possibly longer load times though).
* '''Playable on Emulators?''' Yes - PPSSPP.
 
CISO GUI and CISO XP: Drag-and-drop ISO file on program, Select compression level (1 to 9 (Max)), "Compress", and choose directory for saving the new CSO file.
 
UMDGen is a PSP ISO editor. You can save to either the CSO format with this.
===DAX, JSO, ZSO===
Official format for digitally distributed PSP/minis (partially related to the one used for PS1 games), containing a compressed disc image and optionally a custom boot logo.
 
==PlayStation Vita==
===Trimming===
* '''Archive-quality dump?''' No, but trivially undoable.
* '''Gain:''' Immediate (ROM size lowers).
* '''Tools Used:''' None - [https://github.com/motoharu-gosuto/psvgamesd#trimming-zeroes manual process]
* '''Can be reverted?''' Yes (see above).
* '''Playable on Hardware?''' Yes - with psvgamesd.
* '''Playable on Emulators?''' ?
 
The psvgamesd gamecard backup creation and mounting tools supports mounting a trimmed backup (although no tools exist for dumping directly in trimmed format, or converting between full and trimmed.
 
Neither variant of the .psv format (unrelated to the Cobra Blackfin's .psv format) has caught on, despite being the most accurate options for physical game backups.
 
 
===Extracted files (NoNpDRM format)===
* '''Archive-quality dump?''' No for physical titles, close enough (and generally accepted as such) but no cigar for digital ones.
* '''Gain:''' Immediate (compared to ROM).
* '''Tools Used:''' NoNpDRM plugin + a file manager like VitaShell.
* '''Can be reverted?''' No.
* '''Playable on Hardware?''' Yes - with the NoNpDRM plugin.
* '''Playable on Emulators?''' ?
 
NoNpDRM is a Vita plugin that generates decrypted licenses (valid for any console) when running an original Vita title, as well as allowing the system to accept those licenses, allowing for playing the encrypted files directly copied from a game card or memory card (by extension the name is therefore also used for such files, or the combination of the game's files and a decrypted license).
 
NoPsmDRM is the equivalent for PlayStation Mobile titles.
 
An hypothetical NoPspEmuDRM, allowing for PSP/PS1 eboots to be natively runnable and manageable from the LiveArea/Content Manager, is currently not known to exist nor believed to be in development.
==Dreamcast==
===CHD Archive Format===
* '''Archive-quality dump?''' CHD v4: No (missing datadue to lossy compression, no gameplay issues). CHD v5: Possibly Archive-quality.
* '''Gain:''' Immediate (ROM dump size lowers).
* '''Tools Used:''' GDI to CHD converter.
* '''Can be reverted?''' NoYes with CHD v5.
* '''Playable on Hardware?''' No
* '''Playable on Emulators?''' Yes - DEmul, Reicast , Flycast, and Redream
==GameCube / Wii==
 
People used to resort to WiiScrubber (Wii) and GCM Utility (GC) to scrub/trim games to end up with dumps that while they had no immediate size change, their randomized garbage data (like "dummy", "padding" or "znull") was still there but zeroed out making archived dumps using zip/7zip/rar formats have stunning gains (from 1.4GB uncompressed to 26MB zipped for Animal Crossing for example!). Of course, the file still needed to be uncompressed to its full size everytime you wanted to play it.
 
Trimming and scrubbing (in Wiiscrubber terms) aren't the same! While they both are terms for "zeroing garbage data" to make it more compression-friendly, trimming does not just that like scrubbing but takes the extra step of relocating the garbage data to the end of the file, hence altering its TOC and requiring the disc to be fakesigned, for a not-so-big compression gain. Hence why scrubbing is by far the most authentic and safe way to solve the garbage data problem.
 
However compressed formats incorporating the "padding zeroing" part were made since then, and Dolphin supports them! These are the WBFS and GCZ formats.
 
===GCZ (Dolphin native archive format) - GC/Wii===
 
Dolphin can't play games directly from compressed archives (7zip/zip/gzip/rar...). Instead, it utilizes its own compression method. '''This format has been deprecated in favor of [[#RVZ (Modern Dolphin format) - GC/Wii|RVZ]].'''
 
* '''Archive-quality dump?''' Not directly (can be restored with NKit).
* '''Gain:''' Immediate (ISO dump size lowers). Considerable, depending on the game.
* '''Tools Used:''' Dolphin (right-click the game(s) in the games list and select "Compress ISO..." (or "Compressed selected ISOs..." if more than one is selected)), [https://wiki.gbatemp.net/wiki/NKit NKit], or [https://wit.wiimm.de/ wit (Wiimms ISO Tools)]
* '''Can be reverted?''' Yes. You can right-click on the GCZ-compressed (in blue) ISOs in Dolphin and choose "Uncompress selected ISOs". Garbage data for Wii titles will be zeroed out and thus different from the official dump but will have no effect on gameplay in most cases - if desired can be unscrubbed [https://gbatemp.net/threads/new-app-nkit-restore-shrink-and-preserve-disc-images-in-playable-formerly-swiit.533402/ NKitRestore].
* '''Playable on Hardware?''' No
* '''Playable on Emulators?''' Only Dolphin.
Some examples:
* Megaman Collection (GC): 1.4 GB (original) > 1.1 GB (GCZ) (sound data is stored as uncompressed stream to fill disk)
People used to resort to WiiScrubber ===RVZ (WiiModern Dolphin format) and GCM Utility (- GC) to scrub/trim games to end up with dumps that while they had no immediate size change, their randomized garbage data (like "dummy", "padding" or "znull") was still there but zeroed out making archived dumps using zip/7zip/rar formats have stunning gains (from 1.4GB uncompressed to 26MB zipped for Animal Crossing for example!). Of course, the file still needed to be uncompressed to its full size everytime you wanted to play it.Wii===
Trimming and scrubbing (in Wiiscrubber terms) aren't The Dolphin team developed a new compression format based on WIA called RVZ. Unlike all the same! While they both are terms for "zeroing garbage data" to make it more compression-friendlyprevious formats, trimming does not just that like scrubbing but takes the extra step of relocating RVZ is lossless and can preserve the garbage padding data to on Wii discs as well as the end of necessary files needed by the file, hence altering its TOC and requiring the disc to be fakesigned, for a not-so-big compression gainWii's IOS. Hence why scrubbing Usage is by far the most authentic and safe way very similar to solve the garbage data problemGCZ in Dolphin itself, but it only works on newer Dolphin versions.
However compressed formats incorporating the "padding zeroing" part were made since then, and Dolphin supports them! These are the WBFS and GCZ formats. ===GCZ (Dolphin native archive format) - GC/Wii=== Dolphin can't play games directly from compressed archives (7zip/zip/gzip/rar...). Instead, it utilizes its own compression method. * '''Archive-quality dump?''' Yes for GC games, Not directly for Wii games (Can be restored with NKit).
* '''Gain:''' Immediate (ISO dump size lowers). Considerable, depending on the game.
* '''Tools Used:''' Dolphin 5.0-12188+ (right-click the game(s) in the games list and select "Compress ISO..." (or "Compressed selected ISOs..." if more than one is selected)), [https://wiki.gbatemp.net/wiki/NKit NKit], or [https://wit.wiimm.de/ wit (Wiimms ISO Tools)]* '''Can be reverted?''' Yes. You can right-click on the GCZ-compressed (in blue) ISOs in Dolphin and choose "Uncompress selected ISOs". Garbage data for Wii titles will be zeroed out and thus different from the official dump but will have no effect on gameplay in most cases - if desired can be unscrubbed [https://gbatemp.net/threads/new-app-nkit-restore-shrink-and-preserve-disc-images-in-playable-formerly-swiit.533402/ NKitRestore].* '''Playable on Hardware?''' No.
* '''Playable on Emulators?''' Only Dolphin.
* '''Archive-quality dump?''' Not directly, see note about reversing.
* '''Gain:''' Not Immediate for just scrubbing (ISO dump size the same); Immediate for trimming (ISO dump size lowers).
* '''Tools Used:''' GC-Tool ("wipe garbage data..." - Scrubbing for GameCube), GameCube ISO Trimmer (scrubbing and trimming for GameCube), WiiScrubber (scrubbing /trimming for Wii, although compressing with Dolphin will scrub it anyways).
* '''Can be reverted?''' Yes, with WiiScrubber's undo file or NKit.
* '''Playable on Hardware?''' Yes for scrubbing; some games won't work trimmed.
* '''Gain:''' Immediate (ISO dump size lowers). Considerable, depending on the game.
* '''Tools Used:''' [http://www.wiibackupmanager.co.uk/downloads.html Wii Backup Manager] (after adding your game, go to "Transfer" and select "WBFS") or wit (Wiimms ISO Tools).
* '''Can be reverted?''' Maybe. WBFS files can always be converted back to ISO, but they will remain scrubbed (can be unscrubbed with NKitRestore) and, depending on settings at the time of conversion to WBFS, may be missing update/extras partitions (can be laboriously readded with wit or NKitRestore and a copy of the missing data).* '''Playable on Hardware?''' '''Yes - Preferred format by most SD/USB loaders'''
* '''Playable on Emulators?''' Yes - Dolphin
A scrubbed and sparse (gap-dropping) format directly derived from the one used in the obsolete WBFS file system.
 
Ideal space-saving option for Wii games to be played via an USB loader.
===CSO (aka CISO) - GC/Wii===
* '''Archive-quality dump?''' (Gamecube) Not clear; (Wii) Not directly, see note about reversing.
* '''Gain:''' Immediate (ISO dump size lowers). Less than WBFS.
* '''Tools Used:''' Wii Backup Manager (same as above, but select "CISO" - possibly Wii only), wit
* '''Can be reverted?''' Maybe. CISO files can always be converted back to ISO, but they will remain scrubbed (can be unscrubbed with NKitRestoresame caveats as WBFS).
* '''Playable on Hardware?''' Yes (on Wii/Vwii only) - Nintendont for Gamecube games, uLoader for Wii games
* '''Playable on Emulators?''' Yes - Dolphin
A scrubbed and sparse (gap-dropping) format. Unrelated to the PSP's CSO format.
 
Great space-saving option for Gamecube games to be played on Nintendont.
 
===FST (extracted File System) - GC/Wii===
 
* '''Archive-quality dump?''' No
* '''Gain:''' Very low with GC ISOs, much better with Wii ones. Can also be combined with classic archives compression (ZIP/RAR/7z etc.) to match WIA and NKit compression ratios.
* '''Tools Used:''' [https://wit.wiimm.de/info/composing.html wit]
* '''Can be reverted?''' Partially via the align-files.txt created by wit when extracting.
* '''Playable on Hardware?''' (GC games on Wii/Vwii): Yes - [https://github.com/FIX94/Nintendont/ Nintendont]. (Other combinations): No
* '''Playable on Emulators?''' Yes - Dolphin (both GC and Wii)
 
This has the advantage of easy experimenting with [[#Destructive Modification]]. Includes all disadvantages of trimming.
===WIA (Wii ISO Archive) - Wii only?===
* '''Archive-quality dump?''' Not directly unless --raw option is given, effectively preventing any space savings.
* '''Gain:''' Immediate (ISO dump size lowers). Less More than WBFS.
* '''Tools Used:''' wit
* '''Can be reverted?''' Maybe. Just (same caveats as the previous formats, WIA files can always be converted back to ISO, but the conversion may be lossy (reversible with added workWBFS) depending on conversion settings.
* '''Playable on Hardware?''' No
* '''Playable on Emulators?''' No
===NKit formats - GC/Wii===
* '''Archive-quality dump?''' Yes - by explicit design choice!
* '''Gain:''' Immediate or not depending on choice of - (Wii) only for GCZ- or ISO-based outputformat.
* '''Tools Used:''' NKit.
* '''Can be reverted?''' Yes!
* '''Playable on Hardware?''' (GC) Yes? - ISO-based variant only (Wii) No
* '''Playable on Emulators?''' Yes - Dolphin
Apart from its previously mentioned unscrubbing/untrimming features mentioned in the above formats' descriptions, NKit is able to convert any ISO (clean dump or otherwise) to and from an [https://wiki.gbatemp.net/wiki/NKit/NKitFormat intermediate format applying various reversible changes] that optimize the image for lossless compression, including various options such as full decryption and optionally splitting the often non-unique update partitions to separate files. NKit-GCZ (as well as plain GCZ) is less space efficient than NKit-ISO compressed with a powerful general purpose archiver, however it is directly playable in Dolphin. NKit ISO is the ideal space-saving option for Gamecube games.
==Wii U==
===APP (aka NUS/Installable Format)===
* '''Archive-quality dump?''' No, with respect in regard to a disc image (update partition and padding dropped)discs; Most accurate format for Partially in regard to CDN dumps (digital titles)data.
* '''Gain:''' Immediate (Total size decreases, compared to disc image)
* '''Tools Used:''' wud2app (disc image to APP), disc2app (original disc to APP), NUSPacker (extracted format to APP), Teconmoon's WiiVC Injector (Wii/Gamecube disc image to Enhanced Vwii APP), ...
* '''Can be reverted?''' No, if starting from a WiiU disc image; Yes (see below) if starting from extracted files
* '''Playable on Hardware?''' Yes (after installation)
* '''Playable on Emulators?''' ?No
Consisting of a folder containing *.app, *.h3, title.cert, title.tik, and title.tmd files, this official Nintendo format intended for development use (is how tiles are stored on discs and some system titles are stored on the console and closely related is very similar to how digital titles are sold instored on the CDN (on the CDN the files are named differently, .app files are encrypted with the title key and common keys and the tmd has a certificate chain on the end that is same for every tmd) . This is equivalent to the files contained within a 3DS's CIA formatfile; however unlike a 3DS applicationtitle, converting a disc title to digital does not require modifying the title itself, thus leaving the title's signatures valid.
===RPX/RPL (aka Installed/Extracted/Loadiine Format)===
* '''Gain:''' Immediate (Total size decreases). Can be less than a WUX depending on the game.
* '''Tools Used:''' see below
* '''Can be reverted?''' No, any padding/file order/signature information is lost.
* '''Playable on Hardware?''' Yes, use [https://github.com/dimok789/loadiine_gx2/releases Loadiine GX2]
* '''Playable on Emulators?''' Yes
===CIA (CTR Importable Archive, 3DS and DSiWare only)===
* '''Archive-quality dump?''' No (data removed and edited), compared in regard to a gamecard dump. Potentially most accurate format for dumps; Partially in regard to CDN (digitally distributed) titlesdata.
* '''Gain:''' Immediate (smaller than original ROM - not applicable to digital titles).
* '''Tools Used:''' GodMode9 (can directly dump 3DS cards or installed titles to CIA, as well as converting 3DS to CIA), 3dsconv or 3DS Simple CIA Converter 5.0+ (3DS rom to CIA); makerom (3DS to/from CIA, NCCHs/DSiWare/ELF+RSF to 3DS/CIA), make_cia (DSiWare to CIA)
Official Nintendo format for developers to install digital titles, later became the most popular format for installable homebrew and game backups owing to the relatively low marketshare of flashcards and emulators (and, for the former, strong impopularity) in favor of CFW.
Equivalent of the Wii's WAD (for digital titles only) and the DSi's TAD (again for digital titles only, and with basically no popular support due to the lack of any homebrew title installers supporting the format - not to be confused with an homonymous unofficial format nor with the files produced by the official export-DSiWare-to-SD feature)
===NCCHs (CXIs and CFAs, 3DS only)===
==Destructive Modification==
Of courseSometimes, there are also the devs who don't merely use garbage data for padding but also bloat most bloated part of the game is the game data partitself, either intentional or because of poor design choices. For instance, many PSP JRPGs have a specific FMV movie duplicated a dozen times, and the Megaman Collection on GC GCN stores its sound data using an uncompressed format bringing the size of that portion alone to 1GB. In many of these cases, there's just not much you can do about it without destructively altering the game 's data.
So what do some people do? Note when I say , "deletedeleting" it often means replacing the file with a 1KB dummy file to prevent the ISO file structure from collapsing on itself, but sometimes such care isn't even put into ensuring it's still in a functional state. Some examples:
* delete all videos: this brings Super Smash Bros Brawl to 4.7GB(mainly due to Subspace Emissary).
* delete all voice acting and occasionally sound and music: this brings Xenoblade PAL to 4.2 GB with even only one of both dubs removed.
* delete unused content you could get with emulator cheats like rooms and stuff.* delete "extra" languages other than Englishthe language you need.* If a game has multiple quests, they delete some then you could try deleting characters/quests they side-quest you don't like. Especially seen in compilations and stuff with demos includedas much.
Tools used for this are modding tools for rebuilding file trees, like UMDGen (PSP), Tinke (DS) and also regular ISO tools (PS1, Saturn)...
Since this results , in most of the case cases, in very noticeable detrimental effects in gameplay (if the game doesn't crash outright), this is nothing short of mutilating the game image. Some Sadly enough, some of these dumps make it to sharing sites sadly enough.
Avoid resorting to destructive modifications since it can lead to random crashes and unexpected behavior, especially in games with lots of shared assets. [[http://emulation.gametechwiki.com/index.php/File_Hashes Verifyyour dumps]] your dumps to make sure you don't have these, and if you still want to compress those reduce size, prefer using other methods or using uses decompression on demand. One reason to use a destructively modified dump is for burning your own Dreamcast games, as GD-ROMs were over a gigabyte in size and CD-Rs top out at 850MB.
* '''Archive-quality dump?''' No (destructive modRemoves data)
[[Category:FAQs]]
Anonymous user

Navigation menu