VGA passthrough with KVM on Fedora 16

I've succesfully managed to pass my Radeon HD6770 graphics card to a virtual machine using KVM on Fedora 16.

UPDATE: Please check newer post for updated informations. (See KVM page ->)

http://www.youtube.com/watch?v=E4rF9JRVZuY


I've been trying to do this for a long while now and this is the first time it worked well enough to start doing interesting stuff (eg Heaven Benchmark, Age of Empire, Halo, I'll try Battlefield 3 in the next few days). I've tryed both with kvm and xen, but xen is in my oppinion too difficult to setup, al least for me and I haven't managed to get it working reliable (al least not yet).

My "compute" PC:

• Intel i7-2600 non-K processor, this one has VT-x and VT-d - any processor with VT-d Intel / IOMMU AMD should work though. This is very important!
• ASRock H67M-GE motherboard with VT-d support, I went with ASRock because it was written in black and white on their website, in the motherboard bios manual that it explicity supported VT-d , not a generic Intel virtualization option, or VT-x as other motherboars. Again this is very important!
• 8GB of DDR3 RAM, the more the better
• VTX3D Radeon HD6770 1GB DDR5 128-bit
• 120GB A-DATA S510 SSD
• Sirtec 500W power supply

This is how I made it work (I used Fedora 16, so I don't guarante this will work on other distros too) (Please read a lot more before you try this, and read this post until the last . before you try anything, I won't be responsible if you're system breaks, BACK UP, if you're doing this on a machine with valuable data):

1. I installed Fedora 16, the 64 bit kind, from  DVD, minimal setup, no desktop, as this will also serve as my "compute" (eg programming, testing, dev, server, all in one torture)  pc, as well as my gaming rig. I went with no swap partition and a LVM setup where I reserved a 50GB for the vm and the rest to /. Notice I didn't create any user accounts so I used root.
2. I updated Fedora to have the latest packeges and kernel and rebooted.
3. I setup ssh acces and installed the virtualization related packeges:
1. # yum install @virtualization
2. # service libvirtd start
4. For Intel cpu's you have to specifiy a boot parameter;
1.  Append "intel_iommu=on" to /etc/defaut/grub GRUB_CMDLINE_LINUX.
2. Updated grub 
1. # grub2-mkconfig -o /boot/grub2/grub.cfg
3. Reboot
5. Follow these instructions, for pci passthrough troubleshooting: https://fedoraproject.org/wiki/How_to_debug_Virtualization_problems#PCI_device_assignment 
6. I also blacklisted the radeon driver - requiers reboot or 
1. # modeprobe -r radeon
7. Installing the vm
1. I used virt-manager new virtual machine dialog, and used the 50GB lvm as raw storage and pointed the way to the Windows 7 install DVD. Do NOT try to add/modify any other options, unless you really know what you are doing. ( in my case adding USB devices eg keyboard and mice would result in system frezes -  you should try to add keyboard and mouse support to the VM if possible, just keep in mind that if windows is loading slow it might be from that; or when I activated APIC or ACPI,  after I instaled graphics card driver, Windows wouldn't start because of BSOD)
2. I installed Windows 7 x64
3. I added the graphics card and HDMI audio of the card using virt-manager to my vm, I let Windows install the driver, rebooted and well benchmarked it.

I've played some games using the vm, and well it was ok, I couldn't say that it wasn't a physical machine, but it's stil not perfect. What I'd like to do is to remote acces the vm and play the games, a bit like OnLive, but sofar SPICE/VNC from virt-manager go dark when I attach the graphic card, I cand stil send keyboard and mouse input to the vm but the image frezes - and I think that this is just how they were supposed to work; Windows RDP no go, AOE3 won't even start; standard VNC and TeamViewer aren't fast enough for high quality, skipped frames and things like that. If I won't find something suitable, I'll code it myself, this is probably be my 2012 big summer project.

I've experieced some bugs, like the fact that sometimes, when I'd shut down the vm, the amount of ram allocated to it wouldn't be freed, i had either to restart libvirtd, kill some programs, or reboot. Sometimes virt-manager would crash.

And i'm a bit dissapointed by the networking capabilities of KVM, I did try the latest virtIo drivers for Windows ( http://alt.fedoraproject.org/pub/alt/virtio-win/latest/images/bin/ ) but they weren't as fast at NAT (x10-100 slower!!!), and I'd like/need to make the vm a part of my home network, this is something i'll have to investigate more.

THERE IS NOT A UNIFIED UPTODATE KVM USER MANUAL. The best I could find was a 2010 German wikibooks website, this is where I really miss Virtual Box, it has the best documentation out there (at least comparing with KVM and XEN) for virtualization programs.