Well how do I start?
"Huston, the Eagle has landed"
I think this is the first time in the last 12 month since I "felt the urge" to do this, that I feel that I've done it, that VGA Passthrough is not only possible, but it also stable enough (99% of the time, although a reboot is sometime necessary) for daily use:
- USB passthrough is working fine
- the sound controller passthrough is also working fine
- VGA passthrough is alright and well I just played Crysis 1280x1024 all setting Ultra High without any problems.
Once you have the hardware, the knowledge is out there (and here), you just need to play with it and make it work for you.
Howto
The latest version of the guide is at: https://docs.google.com/document/d/1ef_nfl652L0HLn_wGvnpgjsBJd9LZzaV_-rIcEEoK8Y/edit
Benchmarks
Proper benchmarking is hard, and I might have done something wrong, because almost all the benchmarks are showing >90% of native Windows performance, except Crysis (I played it in the vm with all setting on ultra high and it was fine, at least in my humble opinion, so i don’t know why I got so low result in the benchmark ) and Stawker Call of Prypiat (this is a total mistery).
You can find the results at: https://docs.google.com/spreadsheet/ccc?key=0Aryg5nO-kBebdFdOLU1Yc0trT29JdUxac21QcXg3Qnc#gid=6
You can always do your own ;)
Q&A
From the last post about VGA passthrough.
Anonim1
Xen benchmarks, well I think I've had enough of using 'sombody's else code' for the time beeing, but I wouldn't exclude a Xen comparison sometime in the future, if I have the time.
Anonim2
https://help.ubuntu.com/community/KVM/Installation this + intel_iommu=on to grub (grub2-update), and not installing ubuntu with uefi should be the same thing as fedora, I don't have the time to test ubuntu right now.
Evan Fraser
Check how many cpus does the VM see? If you get the same bad performance no mather what video card you use, maybe the game is cpu bound that's where I would start.
Hi!
ReplyDeleteI've tried to passthrough my HD6930 inside Win7 guest on KVM, and so far it seems that its working. But, when i trying to turn off guest, the host freeze completely. Do u know how to solve this problem?
My hardware CPU->Phenom2 1090T+MB->GA990FX-D3 (IOMMU enabld in BIOS)
OS: Fedora 17
Hi, have you looked at the logs? Is there something strange there?
ReplyDeleteHi!
ReplyDeleteOk, i tried to look at logs after freeze in recovery mode, so far it seems nothing suspicious in /var/log/messages.
Also i cant pass to the guest my pci ati audio device (01:00.1). When i'm add device in virt-manager and launch vm, virt-manager hangs, dunno why. Only 01:00.0 HD6930 passed to the guest.
Also i didn't blacklist my radeon driver cause i do have 2 videocards inside(X1950PRO and HD6930).
My kernel 3.4.0-1.fc17.x86_64
Libvirt 0.9.11.3
qemu-kvm 1.0
virt-manager 0.9.11
Any guesses?
Hi, can you blacklist the radeon module, or temporarily unload it mdeprobe -r radeon before starting the VM or trying to add the audio, just for testing and see if you have the same problems. I'm afraid I haven't had this sort of problems.
ReplyDeletei tried to passthrough HD4850 to my win7 vm on fedora 17 KVM, and got BSOD in guest win7 OS.
ReplyDeleteWell, I just installed Fedora Desktop 17 and trying to do so.
ReplyDeleteI've just disabled Xorg by switching from runlevel 5 to 3:
http://superuser.com/questions/364260/how-to-disable-x-window-on-fedora-15
now I'm trying to do:
sudo modprobe -r radeon
FATAL: Module radeon is in use.
however, it is already blacklisted:
cat /etc/modprobe.d/blacklist.conf | grep radeon
blacklist radeonfb
blacklist radeon
I've managed to remove driver from memory with:
ReplyDelete/sbin/new-kernel-pkg --package kernel --mkinitrd --dracut --depmod --install $(uname -r)
after blacklisting it, now I'm fighting with problem how to unbind card from the host:
błąd: internal error process exited while connecting to monitor: char device redirected to /dev/pts/1
do_spice_init: starting 0.10.1
spice_server_add_interface: SPICE_INTERFACE_MIGRATION
spice_server_add_interface: SPICE_INTERFACE_KEYBOARD
spice_server_add_interface: SPICE_INTERFACE_MOUSE
spice_server_add_interface: SPICE_INTERFACE_QXL
red_worker_main: begin
display_channel_create: create display channel
cursor_channel_create: create cursor channel
spice_server_add_interface: SPICE_INTERFACE_PLAYBACK
spice_server_add_interface: SPICE_INTERFACE_RECORD
Failed to assign device "hostdev1" : Device or resource busy
*** The driver 'pci-stub' is occupying your device 0000:01:00.0.
***
*** You can try the following commands to free it:
***
*** $ echo "1002 9460" > /sys/bus/pci/drivers/pci-stub/new_id
*** $ echo "0000:01:00.0" > /sys/bus/pci/drivers/pci-stub/unbind
*** $ echo "0000:01:00.0" > /sys/bus/pci/drivers/pci-stub/bind
*** $ echo "1002 9460" > /sys/bus/pci/drivers/pci-stub/remove_id
***
qemu-kvm: -device pci-assign,host=01:00.0,
Ok, I missed some things from:
ReplyDeletehttp://fedoraproject.org/wiki/How_to_debug_Virtualization_problems#PCI_device_assignment
Now, according to Your entry:
If you get “Connection reset by peers” when booting up the VM whit a pci device attached IOMMU might not be enabled in the BIOS, grub, or you installed using UEFI (dmesg | grep -e DMAR -e IOMMU shows only IOMMU).
But IOMMU is enabled in BIOS (it's AWARD), also Virtualization is Enabled.
I'm getting:
[ 0.000000] Please enable the IOMMU option in the BIOS setup
[ 1.166451] AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
Well, I've tried with Xen and it also failed.
ReplyDeleteHave no idea what next.. thx for great how-to anyway :(
Sorry for the late response, If you do dmesg | grep AMD-Vi do you get AMD-Vi: Initialized for Passthrough Mode ? AMD uses different kernel parameter than Intel, on AMD you need to pass "iommu=pt iommu=1" - from the kvm wiki, you have to put this on grub command line, if you haven't already done this.
ReplyDeleteAMD Machine
dmesg | grep AMD-Vi
...
AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
AMD-Vi: Lazy IO/TLB flushing enabled
AMD-Vi: Initialized for Passthrough Mode
...
I had only:
DeleteAMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
AMD-Vi: Initialized for Passthrough Mode
via DMESG..
That is exactly what I get and I'm having the same problem. It seems the "AMD-Vi: Lazy IO/TLB flushing enabled" line is missing. What is required to enable that line. I'm not above compiling a custom kernel build but I don't like going that route unless necessary.
DeleteI got it to work on my Fedora 16 KVM host! I needed to pass an argument to the KVM kernel module "options kvm allow_unsafe_assigned_interrupts=1" I created the file "/etc/modprobe.d/kvm.conf" and inserted the above quoted line and it made all the device redirection start working. I hope this helps someone else as I was stumped on this for days.
DeleteI've tried again, because IT WORKS on ESXi 5.0 (won't work with 5.1!! important)
DeleteHowever, I don't know GPU assign is failing with Fedora 17 all-the-time..
New posts at:
http://toomeek.waw.pl/blog/?p=2296
http://toomeek.waw.pl/blog/?p=2285
and for ESXi 5.0
http://toomeek.waw.pl/blog/?p=2260
.. but I'm also trying to assign 2 x GPU cards to 2 x VMs ..
Deletewithout luck.
Thanks for sharing your how-to!!! I wish I had seen it earlier, but now I have everything setup on Xen and it works nicely. Is there a reason you chose KVM over Xen? By the way, I agree with you on the state of KVM documentation, which was one reason I chose Xen. Still, I'd like to give KVM a try and perhaps compare the two.
ReplyDeleteI chose KVM because I couldn't figure out how to make VGA passthrough work with XEN and because KVM was quite easy to set up, and there are other applications I use that benefit from KVM and Qemu.
ReplyDeleteI managed to blacklist the radeon driver but it seems impossible to assign my graphic card.
ReplyDeleteWhen I use virt-manager and try to add the radeon 5850 with its hdmi module, it ends with:
"Error starting domain: Unable to read from monitor: Connection rest by peer." And some python errors.
If I try this:
qemu-kvm -m 512 -boot c -net none -hda /dev/fedora/WinVM -device pci-assign,host=01:00.0 -device pci-assign,host=01:00.1
Failed to assign device "(null)" : Device or resource busy
*** The driver 'pci-stub' is occupying your device 0000:01:00.0.
***
*** You can try the following commands to free it:
***
*** $ echo "1002 68a1" > /sys/bus/pci/drivers/pci-stub/new_id
*** $ echo "0000:01:00.0" > /sys/bus/pci/drivers/pci-stub/unbind
*** $ echo "0000:01:00.0" > /sys/bus/pci/drivers/pci-stub/bind
*** $ echo "1002 68a1" > /sys/bus/pci/drivers/pci-stub/remove_id
***
qemu-kvm: -device pci-assign,host=01:00.0: Device 'pci-assign' could not be initialized
And if I blacklist pci-stub:
qemu-kvm -m 512 -boot c -net none -hda /dev/fedora/WinVM -device pci-assign,host=01:00.0 -device pci-assign,host=01:00.1
Failed to assign device "(null)" : Device or resource busy
Couldn't find out why.
qemu-kvm: -device pci-assign,host=01:00.0: Device 'pci-assign' could not be initialized
Do you have any idea whats wrong?
Any plans on trying this with Fedora 19/20 and kernel 3.12+ with vfio, yet?
ReplyDeleteLike here: https://bbs.archlinux.org/viewtopic.php?id=162768
I'm using your howto on F19, but I can't figure out how to use vfio instead^^