Difference between revisions of "Hypervisors"

From Emulation General Wiki
Jump to navigation Jump to search
m (Type 2 hypervisors: great for winxp or win9x retro gaming without passthrough gpu.)
m
 
(42 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
{{WIP}}
 
{{WIP}}
  
A '''[[wikipedia:x86_virtualization|hypervisor]]''' is a program that takes advantage of an architectural quirk in [[Intel CPUs]] to run emulated software at near-bare-metal speeds, running modern operating systems faster than emulators, but running video games slower than native hardware. This is possible because the processor is designed to execute code in a hierarchy that allows less-privileged software to run almost unmodified. For any instruction in a program that requests more privileges, such as those used by drivers, the hypervisor steps in to emulate the hardware response. There are two types of hypervisors;
+
A '''[[wikipedia:x86_virtualization|hypervisor]]''' is a program that takes advantage of an architectural quirk in hardware to run emulated software at near-bare-metal speeds, '''running operating systems faster than [[POS_(Pong_Consoles)_CPUs_and_Other_Chips#x86_CPUs|emulators]] especially with [[Wikipedia:X86_virtualization#Hardware-assisted_virtualization|hardware-assisting]]''', this is possible because the processor is designed to execute code in a hierarchy that allows less-privileged software to run almost unmodified. For any instruction in a program that requests more privileges, such as those used by drivers, the hypervisor steps in to emulate the hardware response. But even so running video games still slower than native hardware naturally. There are two types of hypervisors;
 
# A '''bare-metal hypervisor''', which is the operating system.
 
# A '''bare-metal hypervisor''', which is the operating system.
 
# A '''software hypervisor''', the ones many are familiar with. They hook into the kernel of an existing operating system (such as Linux or Windows NT) and run alongside the rest of the software on the system.
 
# A '''software hypervisor''', the ones many are familiar with. They hook into the kernel of an existing operating system (such as Linux or Windows NT) and run alongside the rest of the software on the system.
  
 
Hypervisors are the source of the "host" and "guest" terminology since they help clarify the specific "machine" one refers to; the host is the physical hardware and it manages the guests. Most hypervisors try and support the Open Virtualization Format (OVF) to facilitate virtual appliances which are built for a specific purpose (such as running common server software like WordPress or MediaWiki).
 
Hypervisors are the source of the "host" and "guest" terminology since they help clarify the specific "machine" one refers to; the host is the physical hardware and it manages the guests. Most hypervisors try and support the Open Virtualization Format (OVF) to facilitate virtual appliances which are built for a specific purpose (such as running common server software like WordPress or MediaWiki).
 +
 +
See [[Emulation Accuracy]] page for more information about terms like "[[Hypervisors|hypervisors]]", "[[Simulators|simulators]]", "[[Compatibility_layer|compatibility layers]]", "[[Wrappers|wrappers]]", "[[FPGA|FPGA-based hardware cloning]]" and "[[:Category:Emulators|software emulators]]".
  
 
==Type 2 hypervisors==
 
==Type 2 hypervisors==
Line 21: Line 23:
 
|align=left|{{Icon|Windows|Linux|macOS|FreeBSD}}
 
|align=left|{{Icon|Windows|Linux|macOS|FreeBSD}}
 
|[https://www.qemu.org/download/ {{QEMUVer}}]<br/>[https://github.com/kjliew/qemu-3dfx qemu-3dfx ($)]
 
|[https://www.qemu.org/download/ {{QEMUVer}}]<br/>[https://github.com/kjliew/qemu-3dfx qemu-3dfx ($)]
|{{✓}} ||{{✓}} ||{{✓}}
 
|-
 
|[[VirtualBox]]
 
|align=left|{{Icon|Windows|Linux|macOS|FreeBSD|Solaris}}
 
|[https://www.virtualbox.org/wiki/Downloads {{VirtualBoxVer}}]
 
 
|{{✓}} ||{{✓}} ||{{✓}}
 
|{{✓}} ||{{✓}} ||{{✓}}
 
|-
 
|-
 
|[[VMware Workstation]]
 
|[[VMware Workstation]]
 
|align=left|{{Icon|Windows|Linux}}
 
|align=left|{{Icon|Windows|Linux}}
|{{VMwareWorkstationVer}}
+
|[https://customerconnect.vmware.com/en/downloads/#all_products {{VMwareWorkstationVer}}]
 
|{{✗}} ||{{✓}} ||{{✓}}
 
|{{✗}} ||{{✓}} ||{{✓}}
 
|-
 
|-
Line 42: Line 39:
 
|18
 
|18
 
|{{✗}} ||{{✓}} ||{{✓}}
 
|{{✗}} ||{{✓}} ||{{✓}}
 +
|-
 +
|[[VirtualBox]]
 +
|align=left|{{Icon|Windows|Linux|macOS|FreeBSD|Solaris}}
 +
|[https://www.virtualbox.org/wiki/Downloads {{VirtualBoxVer}}]
 +
|{{✓}} ||{{✓}} ||{{~}}
 
|-
 
|-
 
!colspan="6"|Mobile / ARM
 
!colspan="6"|Mobile / ARM
 
|-
 
|-
 
|UTM
 
|UTM
|align=left|{{icon|macOS|iOS}}
+
|align=left|{{icon|macOS}}
 
|[https://github.com/utmapp/UTM/releases {{UTMVer}}]
 
|[https://github.com/utmapp/UTM/releases {{UTMVer}}]
 +
|{{✗}} ||{{✓}} ||{{?}}
 +
|-
 +
|VMOS
 +
|align=left|{{Icon|Android}}
 +
|[https://www.vmos.com/ 1.0.63]
 
|{{✗}} ||{{✓}} ||{{?}}
 
|{{✗}} ||{{✓}} ||{{?}}
 
|}
 
|}
Line 64: Line 71:
 
;Virtual PC
 
;Virtual PC
 
:Microsoft's original offering for consumers, originally made by Connectix as commercial software until Microsoft made it a free download in 2006. Support ended with the introduction of Hyper-V but the last version can be downloaded [http://www.win3x.org/win3board/viewtopic.php?t=19227&language=en here]. Beware that it may cause issues on newer Windows so it's recommended that it's ran on older versions of Windows. As of modern updates of Windows 10 and Windows 11, it is completely impossible to use it. Hyper-V is the official successor, although it lacks support for guests before Windows XP or Windows Server 2003.
 
:Microsoft's original offering for consumers, originally made by Connectix as commercial software until Microsoft made it a free download in 2006. Support ended with the introduction of Hyper-V but the last version can be downloaded [http://www.win3x.org/win3board/viewtopic.php?t=19227&language=en here]. Beware that it may cause issues on newer Windows so it's recommended that it's ran on older versions of Windows. As of modern updates of Windows 10 and Windows 11, it is completely impossible to use it. Hyper-V is the official successor, although it lacks support for guests before Windows XP or Windows Server 2003.
 +
 +
;VMOS
 +
:A program that runs another copy of Android on the current Android system, which would be useful for running multiple copies of the same application simultaneously, running applications that rely on older/newer versions of Android, testing out different Android system variations, or safely executing operations/applications that are likely to harm the system and/or user data.
  
 
==Type 1 hypervisors==
 
==Type 1 hypervisors==
Line 70: Line 80:
 
* ESXi (commercial hypervisor by VMware)
 
* ESXi (commercial hypervisor by VMware)
 
* Xen (open-source hypervisor developed for the Linux kernel)
 
* Xen (open-source hypervisor developed for the Linux kernel)
 +
* [https://serverfault.com/questions/855094/is-kvm-a-type-1-or-type-2-hypervisor KVM]
 
* UnRAID (commercial NAS distribution with an integrated hypervisor)
 
* UnRAID (commercial NAS distribution with an integrated hypervisor)
 
* Hyper-V (commercial hypervisor Microsoft developed for Windows. Requires a Pro, Enterprise, or Education version of Windows 8, 8.1, or 10)
 
* Hyper-V (commercial hypervisor Microsoft developed for Windows. Requires a Pro, Enterprise, or Education version of Windows 8, 8.1, or 10)
Line 87: Line 98:
 
If you're using an NVIDIA card, you also need to configure how the graphics card appears to the VM so that NVIDIA's driver doesn't reject the card.
 
If you're using an NVIDIA card, you also need to configure how the graphics card appears to the VM so that NVIDIA's driver doesn't reject the card.
  
==Emulation issues==
+
If you're using Windows 10/11 for host OS you can use Hyper-V and GPU-P just like Linux GPU passthrough. [https://github.com/jamesstringerparsec/Easy-GPU-PV#prerequisites jamesstringerparsec's "Easy-GPU-PV" batch script] for making GPU Paravirtualization on Windows Hyper-V easier.
 +
 
 +
==Hyper-V Subsets==
 +
In May 2019, Windows Subsystem for Linux 2/WSL 2 was announced, introducing important changes such as a real Linux kernel, through a subset of Hyper-V features. WSL 2 differs from [[Compatibility_layer|WSL 1]] in that WSL 2 runs inside a managed virtual machine that implements the full Linux kernel (similar to [[Android_emulators#Emulators|Windows Subsystem for Android/WSA]]). As a result, WSL 2 is compatible with more Linux binaries than WSL 1, as not all syscalls were implemented in WSL 1. There are even feature requests such as [https://github.com/microsoft/WSL/issues/7747 Gamepad support].
 +
 
 +
==Hypervisor issues==
 
===Graphics===
 
===Graphics===
 
Hypervisors don't have direct access to the graphics hardware with few exceptions including Virgil/Virtio GPU available in QEMU (only works with Linux guests) and VMWare ESXi GPU passthrough, which means graphics card emulation is less than ideal for video games that use 3D APIs. However, 3D acceleration in hypervisors today is capable of running fairly demanding video games or other 3D applications with a few drawbacks such as limited DirectX API versions. Every hypervisor has a different approach to handling 3D graphics, and most of the time companion programs such as VMware Tools have to be installed in virtualized operating systems. Supports for OpenGL are slightly better than Direct3D since it doesn't have to be reverse engineered, but most Windows games (including ports) use Direct3D.
 
Hypervisors don't have direct access to the graphics hardware with few exceptions including Virgil/Virtio GPU available in QEMU (only works with Linux guests) and VMWare ESXi GPU passthrough, which means graphics card emulation is less than ideal for video games that use 3D APIs. However, 3D acceleration in hypervisors today is capable of running fairly demanding video games or other 3D applications with a few drawbacks such as limited DirectX API versions. Every hypervisor has a different approach to handling 3D graphics, and most of the time companion programs such as VMware Tools have to be installed in virtualized operating systems. Supports for OpenGL are slightly better than Direct3D since it doesn't have to be reverse engineered, but most Windows games (including ports) use Direct3D.
  
===[[Macintosh line#x86|macOS]]===
+
===[[macOS]]===
 
There's an ongoing effort to get macOS installations working on hypervisors that run on non-Apple hardware. Such support has historically been low or non-existent since Apple makes no effort to support standard PC hardware. Some people have gotten this to successfully work on KVM/QEMU, with an example/setup guide for Debian/Ubuntu hosts being available [https://github.com/kholia/OSX-KVM here].
 
There's an ongoing effort to get macOS installations working on hypervisors that run on non-Apple hardware. Such support has historically been low or non-existent since Apple makes no effort to support standard PC hardware. Some people have gotten this to successfully work on KVM/QEMU, with an example/setup guide for Debian/Ubuntu hosts being available [https://github.com/kholia/OSX-KVM here].
  
Line 99: Line 115:
 
Despite many of the common operating systems including legacy ones such as MS-DOS, OS/2, Novell Netware, etc. are being supported by modern hypervisors, some obscure (e.g. [https://templeos.org/ TempleOS]), abandoned (e.g. [https://ja.wikipedia.org/wiki/%E8%B6%85%E6%BC%A2%E5%AD%97 Chokanji 1~4]), beta or experimental (e.g. [https://betawiki.net/wiki/Windows_95 Windows "Chicago" Beta]) operating systems may still fall out of the range and having issues, especially when relying on specific hardware configuration or old quirks that no longer presents in modern PC hardware.
 
Despite many of the common operating systems including legacy ones such as MS-DOS, OS/2, Novell Netware, etc. are being supported by modern hypervisors, some obscure (e.g. [https://templeos.org/ TempleOS]), abandoned (e.g. [https://ja.wikipedia.org/wiki/%E8%B6%85%E6%BC%A2%E5%AD%97 Chokanji 1~4]), beta or experimental (e.g. [https://betawiki.net/wiki/Windows_95 Windows "Chicago" Beta]) operating systems may still fall out of the range and having issues, especially when relying on specific hardware configuration or old quirks that no longer presents in modern PC hardware.
  
Using a low-level [[Intel CPUs]] emulator to run these operating systems may ease out such problems compared to hypervisors.
+
Using a low-level [[POS_(Pong_Consoles)_CPUs_and_Other_Chips#x86_CPUs|x86 CPUs]] emulator to run these operating systems may ease out such problems but its significantly slower performance compared to [[Wikipedia:X86_virtualization#Hardware-assisted_virtualization|hardware-assisted]] [[Hypervisors]].
 +
 
 +
===Sound issues===
 +
VMware (especially newer versions) has sound related issues with some of the legacy guest operating systems like [[Windows_2000/XP/Vista_emulators|Windows XP]]. According to user reports, sometimes sound stops working OR it become laggy on guest operating system by launching some applications/games. Sometimes audio gets distorted/crackling noise popping. Also you may experience audio related application/game crashes. Usually restarting guest operating system fixes these kind of issues.
 +
 
 +
:Reddit thread: [https://old.reddit.com/r/vmware/comments/sb4cml/windows_xp_guest_sound_laggy_no_matter_what_in/ Windows XP guest sound laggy]
 +
 
 +
===ISBRendererComm error with 3D Acceleration===
 +
[[File:VMware ISBRenderer crash.jpeg|thumb|298px|ISBRendererComm crash]]
 +
There was a major change in VMware 16.x in how virtual graphics were handled (sandboxed) and it also added support for DX11 in Windows VMs. But due to these changes users reported lots of crashes lately especially with "3D Acceleration". For more information about these type of errors see [https://communities.vmware.com/t5/VMware-Workstation-Pro/ISBRendererComm-Lost-connection-to-mksSandbox-and-MKS/td-p/2838888 this thread].
 +
 
 +
===Hyper-V feature conflict issue===
 +
[https://learn.microsoft.com/en-us/troubleshoot/windows-client/application-management/virtualization-apps-not-work-with-hyper-v Many third-party Type 2 virtualization applications don't work together with Hyper-V feature from 'Control Panel/Windows Features']. Affected applications include "VMware Workstation" and "VirtualBox". These applications might not start virtual machines, or they may fall back to a slower, emulated mode ('''no''' VT-x and nested paging etc.)
 +
 
 +
Even though some reports consider this issue fixed via [https://www.ubackup.com/enterprise-backup/vmware-workstation-and-hyper-v-are-not-compatible.html recent updates (with VMware 15.5.6 and Windows 10 20H1) and WHP windows feature], user reports shows that it's still there. See [https://old.reddit.com/r/vmware/comments/swjp4t/running_vmware_workstation_on_a_hyperv_enabled/ this thread] for more information about this issue. Latest VirtualBox and BlueStacks versions have no problem with Hyper-V feature anymore.
  
 
==See also==
 
==See also==
* [[Intel CPUs]]
+
;Related to Hypervisors;
 +
* [[Windows_95/98/ME_emulators]]
 +
* [[Windows_2000/XP/Vista_emulators]]
 +
* [[MacOS]]
 +
* [[Android_emulators]]
 +
;CPUs and Other chips emulation instead of virtualization;
 +
*[[POS_(Pong_Consoles)_CPUs_and_Other_Chips]]
  
[[Category:Computers]]
+
[[Category:Computers|*]]
 
[[Category:Not really emulators]]
 
[[Category:Not really emulators]]

Latest revision as of 13:10, 1 February 2024

A hypervisor is a program that takes advantage of an architectural quirk in hardware to run emulated software at near-bare-metal speeds, running operating systems faster than emulators especially with hardware-assisting, this is possible because the processor is designed to execute code in a hierarchy that allows less-privileged software to run almost unmodified. For any instruction in a program that requests more privileges, such as those used by drivers, the hypervisor steps in to emulate the hardware response. But even so running video games still slower than native hardware naturally. There are two types of hypervisors;

  1. A bare-metal hypervisor, which is the operating system.
  2. A software hypervisor, the ones many are familiar with. They hook into the kernel of an existing operating system (such as Linux or Windows NT) and run alongside the rest of the software on the system.

Hypervisors are the source of the "host" and "guest" terminology since they help clarify the specific "machine" one refers to; the host is the physical hardware and it manages the guests. Most hypervisors try and support the Open Virtualization Format (OVF) to facilitate virtual appliances which are built for a specific purpose (such as running common server software like WordPress or MediaWiki).

See Emulation Accuracy page for more information about terms like "hypervisors", "simulators", "compatibility layers", "wrappers", "FPGA-based hardware cloning" and "software emulators".

Type 2 hypervisors[edit]

Name Operating System(s) Latest Version FLOSS Active Recommended
PC / x86
QEMU Windows Linux macOS FreeBSD 8.2.0
qemu-3dfx ($)
VMware Workstation Windows Linux 17.5.1
VMware Fusion macOS 11.5
Parallels Desktop macOS 18
VirtualBox Windows Linux macOS FreeBSD Solaris 7.0.14 ~
Mobile / ARM
UTM macOS 4.2.5 ?
VMOS Android 1.0.63 ?
QEMU
A general-purpose emulator that supports a wide range of hardware for use on a multitude of platforms. Used to be a hypervisor by way of KQEMU but has been deprecated in favor of other systems like KVM (a kernel module for Linux and FreeBSD) and HAXM (which is developed by Intel). QEMU has been used to emulate several non-PC x86 machines like XQEMU/Xemu for the Xbox, and Orbital for the PlayStation 4. The non-specific application of QEMU allows emulators like XQEMU and Orbital to function as hypervisors without having to write custom-made drivers.
VirtualBox
Is open source but it has an extension pack that adds features common on standard PCs like USB 2.0, 3.0, PXE boot, and hard drive encryption. It is generally the easiest to set up and get running.
VMware Workstation Pro
Is also available for GNU/Linux, though the Windows version has better 3D support because of its native Direct3D implementation.
VMware Workstation Player
Formerly VMware Player, it is VMware's free offering but lacks a few features like snapshots.
Virtual PC
Microsoft's original offering for consumers, originally made by Connectix as commercial software until Microsoft made it a free download in 2006. Support ended with the introduction of Hyper-V but the last version can be downloaded here. Beware that it may cause issues on newer Windows so it's recommended that it's ran on older versions of Windows. As of modern updates of Windows 10 and Windows 11, it is completely impossible to use it. Hyper-V is the official successor, although it lacks support for guests before Windows XP or Windows Server 2003.
VMOS
A program that runs another copy of Android on the current Android system, which would be useful for running multiple copies of the same application simultaneously, running applications that rely on older/newer versions of Android, testing out different Android system variations, or safely executing operations/applications that are likely to harm the system and/or user data.

Type 1 hypervisors[edit]

Rather than being installed as a standard program in an operating system, Type 1 hypervisors form the operating system used by a computer. Examples include;

  • ESXi (commercial hypervisor by VMware)
  • Xen (open-source hypervisor developed for the Linux kernel)
  • KVM
  • UnRAID (commercial NAS distribution with an integrated hypervisor)
  • Hyper-V (commercial hypervisor Microsoft developed for Windows. Requires a Pro, Enterprise, or Education version of Windows 8, 8.1, or 10)

Integrations[edit]

Many type 2 hypervisors include some sort of additional software that can be installed in a guest. This software listens to the hypervisor and facilitates communication between the guest and host, allowing for features like automatic screen resizing (where the host measures the height and width of the window and sends an optimal size for the guest to resize the display to), clipboard sharing (where anything copied on the host can be pasted in the guest), shared folders (where a directory on the host can be made available for access from a guest using a built-in file server and a direct network connection to the guest), and mouse integration (where the hypervisor emulates an absolute pointing device like a drawing tablet to give the impression that moving the cursor in and out of the machine's display is seamless). Three hypervisors are known to have them;

  • VirtualBox Guest Additions
  • VMware Tools (and its open-source counterpart, open-vm-tools)
  • SPICE Guest Additions (for use with libvirt and/or QEMU on UNIX/Linux hosts)

VFIO[edit]

VFIO (short for Virtual Function I/O) is a Linux kernel driver for PCI passthrough. Its appeal is that it works with graphics cards, meaning hypervisors on Linux can bypass graphics emulation for modern Windows games as opposed to having to use compatibility layers and wrappers. The catch is that the system needs;

  • An IOMMU (outlined in the VT-d and AMD-Vi specs) with processor and motherboard support

And because the graphics card doesn't get utilized until the guest powers on, that also means;

  • A dedicated monitor, a monitor with multiple inputs, and/or a KVM switch (not to be confused with the KVM hypervisor)

If you're using an NVIDIA card, you also need to configure how the graphics card appears to the VM so that NVIDIA's driver doesn't reject the card.

If you're using Windows 10/11 for host OS you can use Hyper-V and GPU-P just like Linux GPU passthrough. jamesstringerparsec's "Easy-GPU-PV" batch script for making GPU Paravirtualization on Windows Hyper-V easier.

Hyper-V Subsets[edit]

In May 2019, Windows Subsystem for Linux 2/WSL 2 was announced, introducing important changes such as a real Linux kernel, through a subset of Hyper-V features. WSL 2 differs from WSL 1 in that WSL 2 runs inside a managed virtual machine that implements the full Linux kernel (similar to Windows Subsystem for Android/WSA). As a result, WSL 2 is compatible with more Linux binaries than WSL 1, as not all syscalls were implemented in WSL 1. There are even feature requests such as Gamepad support.

Hypervisor issues[edit]

Graphics[edit]

Hypervisors don't have direct access to the graphics hardware with few exceptions including Virgil/Virtio GPU available in QEMU (only works with Linux guests) and VMWare ESXi GPU passthrough, which means graphics card emulation is less than ideal for video games that use 3D APIs. However, 3D acceleration in hypervisors today is capable of running fairly demanding video games or other 3D applications with a few drawbacks such as limited DirectX API versions. Every hypervisor has a different approach to handling 3D graphics, and most of the time companion programs such as VMware Tools have to be installed in virtualized operating systems. Supports for OpenGL are slightly better than Direct3D since it doesn't have to be reverse engineered, but most Windows games (including ports) use Direct3D.

macOS[edit]

There's an ongoing effort to get macOS installations working on hypervisors that run on non-Apple hardware. Such support has historically been low or non-existent since Apple makes no effort to support standard PC hardware. Some people have gotten this to successfully work on KVM/QEMU, with an example/setup guide for Debian/Ubuntu hosts being available here.

However, running macOS on hypervisors running on actual Mac hardware is natively supported by hypervisor software like Parallels Desktop or VMware Fusion.

Obscure, Abandoned, Beta or Experimental Operating Systems[edit]

Despite many of the common operating systems including legacy ones such as MS-DOS, OS/2, Novell Netware, etc. are being supported by modern hypervisors, some obscure (e.g. TempleOS), abandoned (e.g. Chokanji 1~4), beta or experimental (e.g. Windows "Chicago" Beta) operating systems may still fall out of the range and having issues, especially when relying on specific hardware configuration or old quirks that no longer presents in modern PC hardware.

Using a low-level x86 CPUs emulator to run these operating systems may ease out such problems but its significantly slower performance compared to hardware-assisted Hypervisors.

Sound issues[edit]

VMware (especially newer versions) has sound related issues with some of the legacy guest operating systems like Windows XP. According to user reports, sometimes sound stops working OR it become laggy on guest operating system by launching some applications/games. Sometimes audio gets distorted/crackling noise popping. Also you may experience audio related application/game crashes. Usually restarting guest operating system fixes these kind of issues.

Reddit thread: Windows XP guest sound laggy

ISBRendererComm error with 3D Acceleration[edit]

ISBRendererComm crash

There was a major change in VMware 16.x in how virtual graphics were handled (sandboxed) and it also added support for DX11 in Windows VMs. But due to these changes users reported lots of crashes lately especially with "3D Acceleration". For more information about these type of errors see this thread.

Hyper-V feature conflict issue[edit]

Many third-party Type 2 virtualization applications don't work together with Hyper-V feature from 'Control Panel/Windows Features'. Affected applications include "VMware Workstation" and "VirtualBox". These applications might not start virtual machines, or they may fall back to a slower, emulated mode (no VT-x and nested paging etc.)

Even though some reports consider this issue fixed via recent updates (with VMware 15.5.6 and Windows 10 20H1) and WHP windows feature, user reports shows that it's still there. See this thread for more information about this issue. Latest VirtualBox and BlueStacks versions have no problem with Hyper-V feature anymore.

See also[edit]

Related to Hypervisors;
CPUs and Other chips emulation instead of virtualization;