Emulation issues and roadblocks: Difference between revisions

Line 263:

* PCSX2: [https://pcsx2.net/blog Blog]

PlayStation consoles were always notorious for system complexity (e.g. [[PlayStation_emulators#Emulation_issues|PlayStation]], [[PlayStation_2_emulators#Emulation_issues|PlayStation 2]]). Sony's technology being developer-unfriendly makes it emulator-unfriendly as well, and RPCS3's steep system requirements prove it. Even if done properly, [[High/Low_level_emulation|an LLE approach]] would be performance suicide, as some things just have to be abstracted enough to get high framerates in games.

Sony has never released an [[Official_emulators|official PS3 emulator]] or native backwards compatibility for the PS4 or PS5. Instead, it offers select PS3 titles through cloud-based streaming on PlayStation Plus Premium. The primary technical challenge is the PS3's Cell Broadband Engine architecture, which differs significantly from the x86-64 architecture used in the PS4 and PS5 (especially considering the PS4's very weak hardware even at the time of its announcement, let alone its release). This makes accurate and performant software emulation considerably more difficult. While the PS5's hardware seems sufficient (as some independent experiments have shown some success running PS3 code through custom emulation efforts)<ref>{{YouTubeVideo|_zbw_A9dIWM|I Ran PS3 Games on PS5 Silicon to Prove Sony Wrong}}</ref><ref>[https://overclock3d.net/news/software/ps3-emulation-tested-on-ps5-under-linux-with-rpcs3-does-it-work/ PS3 Emulation Tested on PS5 under Linux with RPCS3]</ref>, the prevailing narrative still falls back on the old assumption that achieving broad compatibility would just be too big of an engineering hurdle. Sony has opted for cloud streaming (which runs games on actual or virtualized PS3 hardware in data centers) rather than investing in a full native emulator for consumer devices. Additional factors likely include development and long-term maintenance costs, although these maintenance costs are often claimed by some to be a major issue when they are not actually a huge factor. Other potential reasons may stem from the difficulty in justifying the complex copyright and licensing situation for hundreds of old titles, diminishing returns for a relatively niche audience (based on Sony's experience with previous generations' backwards compatibility through PSN titles),<ref>{{YouTubeVideo|6WkpaJkB2M8|Why PS4 Doesn't Have Backwards Compatibility and Xbox One Does}}</ref><ref>{{YouTubeVideo|6rzLZ9Bkhwo|MVG: Sony's complicated history with Backwards Compatibility}}</ref> and potential political or investor pressure to avoid re-releasing older games that could be perceived as controversial under today's Overton window. On top of that, there is a strong monetary appetite for remakes and remasters (which often leads to poor releases, heavy censorship of the original games, and turning them into soulless re-releases aimed at younger audiences, delisting the original and unaltered versions of the game from digital storefronts) sold at full retail price instead of affordable offering emulation software. Despite these hurdles, the PlayStation 3 emulation scene (much like the [[Switch]] emulation scene) possesses two major advantages. First, unlike many proprietary "black box" systems, the Cell architecture was thoroughly documented by IBM. Furthermore, the [[RPCS3]] project has benefited from over a decade of continuous development; since its early stages, it has maintained a dedicated donor base that provides the consistent financial support.

There are two major bottlenecks at play:

* '''[[wikipedia:Cell_(microprocessor)|Cell Broadband Engine]]''' - consists of two architectures that developers have to program for; [[POS_(Pong_Consoles)_CPUs_and_Other_Chips#PPC_CPUs|PowerPC]], and... whatever the SPEs really are; and you have a great formula for high system requirements, SPU hardware environment is the furthest thing from a [[POS_(Pong_Consoles)_CPUs_and_Other_Chips#x86_CPUs|x86 PC processor]]. "Cell Broadband Engine" (i.e. the PS3 CPU) leaned into SIMD with the SPUs recklessly, ease of programming be damned, but it broke records in 2006. Four pipelines of 512-bit wide ALUs per core is an absolute bonkers amount of computational capability. The main issue is that SPU code is stored as data blobs anywhere the game shoved it into: main executable constants, resource files, generated on the fly...anything goes, and it’s impossible to extract before it is submitted to SPEs for processing (same with the RSX shaders). Another half is that a way SPU are utilised in later games is a sort of microservices with context switching and load balancing (i.e several microtasks code can be uploaded/replaced/switched into single SPE memory at runtime during one frame cycle of game logic). Like SPURS for example. Old style SPE programming from early games with fixed systems bound to specific cores and not changed since works well with static recompilation. Unfortunately, this way was suboptimal for games and developers quickly went into "microtasks" design. SPEs are closer to [[PlayStation_2_emulators#Emulation_issues|VUs from the PS2]] than a GPU. They're a SIMD cluster, don't be fooled by their apparent similarity to GPUs. GPU are low throughput but very wide. SPEs are also wide, but not nearly as much so. However, their individual thread throughput is insane even compared to a something like an RTX 4090. It's not possible to beat SPUs 1:1 (it's ALL vector, it doesn't have scalar registers at least not general-purpose ones), they have 128 registers and very low latency SRAM, almost no memory fetches and everything is async. Although AVX-512 comes close in performance (at 5+ GHz). GPUs on the other hand are around 10-100x slower, but that's just because of how their cores are designed and the fact that SPU kernels are optimized to run in much smaller groups. [https://discord.com/channels/272035812277878785/272083901277667342/1197648306084524202 There's too much working against emulation at 3.2 GHz rate since most SPU instructions need many instructions on PC (like dozens in some cases), so the PC side needs to be an order of magnitude faster.]

* '''[[wikipedia:RSX_Reality_Synthesizer|RSX (Reality Synthesizer)]]''': The PlayStation 3 GPU went unemulated for a long time, simply because of how many components were undocumented; the [https://www.psdevwiki.com/ps3/RSX RSX Reality Synthesizer] is a custom-designed chip similar to the GeForce 7800 GTX developed by Nvidia specifically for the PlayStation 3. It's not well-documented, and developers have to figure out how it displays graphics and graphical effects. Without access to Nvidia's resources, which would normally be included with an SDK, this is very difficult.

:Something of note is that this GPU was also managed by two different memory units with very disparate frequency speeds; 1) 256 MB of GDDR3 RAM clocked at 650 MHz with an effective transmission rate of 1.4 GHz, and 2) up to 224 MB of the 3.2 GHz XDR main memory via the CPU (480 MB max).

In practice, smaller devs use the PPE most of the time, but AAA use everything what PS3 had. It took an eternity for AVX-512 to arrive, with Intel releasing their first implementation only in 2016; but wide availability in consumer grade PCs only started with AMD's Ryzen 7000 series in late 2022. Fortunately, most of the [[High_and_low-level_emulation|optimizations]] in RPCS3 still apply for AVX2, e.g., the 512bit checksum is still much faster than a full comparison, and the LTO, rotated checksum, and the other optimizations added also apply for machines without AVX-512. The RPCS3 developers use ahead-of-time recompilation via an LLVM. [https://discord.com/channels/272035812277878785/272083901277667342/1200171144603512892 RPCS3 is not just compiling the code it runs, it also needs to compile a real-time environment simulator e.g. the 128x128bit register file, LS SRAM, the very weird memory flow controller, etc.]

For more information about PlayStation 3 hardware and its reverse engineering;

* MVG: {{YouTubeVideo|lLebZyha74o|Why is the Sony PlayStation PS3 so hard to emulate?}}, {{YouTubeVideo|zW3XawAsaeU|Why was the Sony PlayStation 3 so hard to develop games for ?}}

* Zygal Studios: {{YouTubeVideo|W0-NEwh1m2c|Why was the PS3 Difficult to Develop Games For?}}

* Modded Warfare: {{YouTubePlaylist|PLn7ji3VsPy3HtSY6rB8yCCRQpWQOc-uJS|PS3 Jailbreak Tutorials}}

* {{YouTubeVideo|4joCMfTPP4M|Alexandro Sanchez: FOSDEM'22 - PlayStation 3 Emulation}}

* [https://www.copetti.org/writings/consoles/playstation-3/ Rodrigo Copetti: PLAYSTATION 3 Architecture Practical Analysis]

* [https://www.psdevwiki.com/ps3/ psdevwiki: PlayStation 3]

* Whatcookie: {{YouTubeVideo|2Ngo-tnE6zQ|How does Static Recompilation differ from Emulation?}}, {{YouTubeVideo|3dkN-6TJNHs|whatcookie: The most efficient way to do nothing}}, {{YouTubeVideo|19ae5Mq2lJE|Why is PS3 emulation so fast: RPCS3 optimizations explained}}, {{YouTubeVideo|0HWOmEjlpMs|It took 5 years to make this code 11.8 times faster}}, [https://whatcookie.github.io/posts/why-is-avx-512-useful-for-rpcs3/ Why Is AVX 512 Useful for RPCS3?], {{YouTubeVideo|VR_HZMMMOX4|How to uncap framerates}}, {{YouTubeVideo|hCuT_GXagz0|Why Intel and AMD couldn't play LittleBigPlanet together}} and {{YouTubeVideo|vki1VFlpupI|RPCS3 developer optimizing code in RPCS3}}.

* RPCS3's [https://rpcs3.net/blog/2020/08/21/hardware-performance-scaling/ progress report August 2020]: Since intel dropped TSX due to security reasons, RPCS3 will use TSX-FA/TSX Force Abort on CPUs with new microcode and [https://github.com/RPCS3/rpcs3/issues/6028 this will result potential regressions], good news is RPCS3 has improved non-TSX performance, because of this, even if your CPU supports TSX it will be disabled for RPCS3 by default.

* [https://github.com/clienthax/Ps3GhidraScripts Ps3GhidraScripts] - A collection of scripts for parsing PS3 executables with Ghidra.

* [https://discord.com/channels/644252595329957915/668417976554225674 T2 SDE discord - ppc-aim-ps3 channel]

* RPCS3: [https://rpcs3.net/blog/ Blog], [https://discord.gg/rpcs3 Discord], {{YouTubeChannelHandle|RPCS3_emu|YouTube}}.

* [[Emulation books and articles]]

* [[Console-specific development wikis]]

* [[Wikipedia:PSGL|PSGL API]]

[[Category:FAQs]]

@@ Line 263: / Line 263: @@
 * PCSX2: [https://pcsx2.net/blog Blog]
+PlayStation consoles were always notorious for system complexity (e.g. [[PlayStation_emulators#Emulation_issues|PlayStation]], [[PlayStation_2_emulators#Emulation_issues|PlayStation 2]]). Sony's technology being developer-unfriendly makes it emulator-unfriendly as well, and RPCS3's steep system requirements prove it. Even if done properly, [[High/Low_level_emulation|an LLE approach]] would be performance suicide, as some things just have to be abstracted enough to get high framerates in games.
+Sony has never released an [[Official_emulators|official PS3 emulator]] or native backwards compatibility for the PS4 or PS5. Instead, it offers select PS3 titles through cloud-based streaming on PlayStation Plus Premium. The primary technical challenge is the PS3's Cell Broadband Engine architecture, which differs significantly from the x86-64 architecture used in the PS4 and PS5 (especially considering the PS4's very weak hardware even at the time of its announcement, let alone its release). This makes accurate and performant software emulation considerably more difficult. While the PS5's hardware seems sufficient (as some independent experiments have shown some success running PS3 code through custom emulation efforts)<ref>{{YouTubeVideo|_zbw_A9dIWM|I Ran PS3 Games on PS5 Silicon to Prove Sony Wrong}}</ref><ref>[https://overclock3d.net/news/software/ps3-emulation-tested-on-ps5-under-linux-with-rpcs3-does-it-work/ PS3 Emulation Tested on PS5 under Linux with RPCS3]</ref>, the prevailing narrative still falls back on the old assumption that achieving broad compatibility would just be too big of an engineering hurdle. Sony has opted for cloud streaming (which runs games on actual or virtualized PS3 hardware in data centers) rather than investing in a full native emulator for consumer devices. Additional factors likely include development and long-term maintenance costs, although these maintenance costs are often claimed by some to be a major issue when they are not actually a huge factor. Other potential reasons may stem from the difficulty in justifying the complex copyright and licensing situation for hundreds of old titles, diminishing returns for a relatively niche audience (based on Sony's experience with previous generations' backwards compatibility through PSN titles),<ref>{{YouTubeVideo|6WkpaJkB2M8|Why PS4 Doesn't Have Backwards Compatibility and Xbox One Does}}</ref><ref>{{YouTubeVideo|6rzLZ9Bkhwo|MVG: Sony's complicated history with Backwards Compatibility}}</ref> and potential political or investor pressure to avoid re-releasing older games that could be perceived as controversial under today's Overton window. On top of that, there is a strong monetary appetite for remakes and remasters (which often leads to poor releases, heavy censorship of the original games, and turning them into soulless re-releases aimed at younger audiences, delisting the original and unaltered versions of the game from digital storefronts) sold at full retail price instead of affordable offering emulation software. Despite these hurdles, the PlayStation 3 emulation scene (much like the [[Switch]] emulation scene) possesses two major advantages. First, unlike many proprietary "black box" systems, the Cell architecture was thoroughly documented by IBM. Furthermore, the [[RPCS3]] project has benefited from over a decade of continuous development; since its early stages, it has maintained a dedicated donor base that provides the consistent financial support.
+There are two major bottlenecks at play:
+* '''[[wikipedia:Cell_(microprocessor)|Cell Broadband Engine]]''' - consists of two architectures that developers have to program for; [[POS_(Pong_Consoles)_CPUs_and_Other_Chips#PPC_CPUs|PowerPC]], and... whatever the SPEs really are; and you have a great formula for high system requirements, SPU hardware environment is the furthest thing from a [[POS_(Pong_Consoles)_CPUs_and_Other_Chips#x86_CPUs|x86 PC processor]]. "Cell Broadband Engine" (i.e. the PS3 CPU) leaned into SIMD with the SPUs recklessly, ease of programming be damned, but it broke records in 2006. Four pipelines of 512-bit wide ALUs per core is an absolute bonkers amount of computational capability. The main issue is that SPU code is stored as data blobs anywhere the game shoved it into: main executable constants, resource files, generated on the fly...anything goes, and it’s impossible to extract before it is submitted to SPEs for processing (same with the RSX shaders). Another half is that a way SPU are utilised in later games is a sort of microservices with context switching and load balancing (i.e several microtasks code can be uploaded/replaced/switched into single SPE memory at runtime during one frame cycle of game logic). Like SPURS for example. Old style SPE programming from early games with fixed systems bound to specific cores and not changed since works well with static recompilation. Unfortunately, this way was suboptimal for games and developers quickly went into "microtasks" design. SPEs are closer to [[PlayStation_2_emulators#Emulation_issues|VUs from the PS2]] than a GPU. They're a SIMD cluster, don't be fooled by their apparent similarity to GPUs. GPU are low throughput but very wide. SPEs are also wide, but not nearly as much so. However, their individual thread throughput is insane even compared to a something like an RTX 4090. It's not possible to beat SPUs 1:1 (it's ALL vector, it doesn't have scalar registers at least not general-purpose ones), they have 128 registers and very low latency SRAM, almost no memory fetches and everything is async. Although AVX-512 comes close in performance (at 5+ GHz). GPUs on the other hand are around 10-100x slower, but that's just because of how their cores are designed and the fact that SPU kernels are optimized to run in much smaller groups. [https://discord.com/channels/272035812277878785/272083901277667342/1197648306084524202 There's too much working against emulation at 3.2 GHz rate since most SPU instructions need many instructions on PC (like dozens in some cases), so the PC side needs to be an order of magnitude faster.]
+* '''[[wikipedia:RSX_Reality_Synthesizer|RSX (Reality Synthesizer)]]''': The PlayStation 3 GPU went unemulated for a long time, simply because of how many components were undocumented; the [https://www.psdevwiki.com/ps3/RSX RSX Reality Synthesizer] is a custom-designed chip similar to the GeForce 7800 GTX developed by Nvidia specifically for the PlayStation 3. It's not well-documented, and developers have to figure out how it displays graphics and graphical effects. Without access to Nvidia's resources, which would normally be included with an SDK, this is very difficult.
+:Something of note is that this GPU was also managed by two different memory units with very disparate frequency speeds; 1) 256 MB of GDDR3 RAM clocked at 650 MHz with an effective transmission rate of 1.4 GHz, and 2) up to 224 MB of the 3.2 GHz XDR main memory via the CPU (480 MB max).
+In practice, smaller devs use the PPE most of the time, but AAA use everything what PS3 had. It took an eternity for AVX-512 to arrive, with Intel releasing their first implementation only in 2016; but wide availability in consumer grade PCs only started with AMD's Ryzen 7000 series in late 2022. Fortunately, most of the [[High_and_low-level_emulation|optimizations]] in RPCS3 still apply for AVX2, e.g., the 512bit checksum is still much faster than a full comparison, and the LTO, rotated checksum, and the other optimizations added also apply for machines without AVX-512. The RPCS3 developers use ahead-of-time recompilation via an LLVM. [https://discord.com/channels/272035812277878785/272083901277667342/1200171144603512892 RPCS3 is not just compiling the code it runs, it also needs to compile a real-time environment simulator e.g. the 128x128bit register file, LS SRAM, the very weird memory flow controller, etc.]
+For more information about PlayStation 3 hardware and its reverse engineering;
+* MVG: {{YouTubeVideo|lLebZyha74o|Why is the Sony PlayStation PS3 so hard to emulate?}}, {{YouTubeVideo|zW3XawAsaeU|Why was the Sony PlayStation 3 so hard to develop games for ?}}
+* Zygal Studios: {{YouTubeVideo|W0-NEwh1m2c|Why was the PS3 Difficult to Develop Games For?}}
+* Modded Warfare: {{YouTubePlaylist|PLn7ji3VsPy3HtSY6rB8yCCRQpWQOc-uJS|PS3 Jailbreak Tutorials}}
+* {{YouTubeVideo|4joCMfTPP4M|Alexandro Sanchez: FOSDEM'22 - PlayStation 3 Emulation}}
+* [https://www.copetti.org/writings/consoles/playstation-3/ Rodrigo Copetti: PLAYSTATION 3 Architecture Practical Analysis]
+* [https://www.psdevwiki.com/ps3/ psdevwiki: PlayStation 3]
+* Whatcookie: {{YouTubeVideo|2Ngo-tnE6zQ|How does Static Recompilation differ from Emulation?}}, {{YouTubeVideo|3dkN-6TJNHs|whatcookie: The most efficient way to do nothing}}, {{YouTubeVideo|19ae5Mq2lJE|Why is PS3 emulation so fast: RPCS3 optimizations explained}}, {{YouTubeVideo|0HWOmEjlpMs|It took 5 years to make this code 11.8 times faster}}, [https://whatcookie.github.io/posts/why-is-avx-512-useful-for-rpcs3/ Why Is AVX 512 Useful for RPCS3?], {{YouTubeVideo|VR_HZMMMOX4|How to uncap framerates}}, {{YouTubeVideo|hCuT_GXagz0|Why Intel and AMD couldn't play LittleBigPlanet together}} and {{YouTubeVideo|vki1VFlpupI|RPCS3 developer optimizing code in RPCS3}}.
+* RPCS3's [https://rpcs3.net/blog/2020/08/21/hardware-performance-scaling/ progress report August 2020]: Since intel dropped TSX due to security reasons, RPCS3 will use TSX-FA/TSX Force Abort on CPUs with new microcode and [https://github.com/RPCS3/rpcs3/issues/6028 this will result potential regressions], good news is RPCS3 has improved non-TSX performance, because of this, even if your CPU supports TSX it will be disabled for RPCS3 by default.
+* [https://github.com/clienthax/Ps3GhidraScripts Ps3GhidraScripts] - A collection of scripts for parsing PS3 executables with Ghidra.
+* [https://discord.com/channels/644252595329957915/668417976554225674 T2 SDE discord - ppc-aim-ps3 channel]
+* RPCS3: [https://rpcs3.net/blog/ Blog], [https://discord.gg/rpcs3 Discord], {{YouTubeChannelHandle|RPCS3_emu|YouTube}}.
+* [[Emulation books and articles]]
+* [[Console-specific development wikis]]
+* [[Wikipedia:PSGL|PSGL API]]
 [[Category:FAQs]]