Linux – Prevent Radeon driver from attaching to specific PCI devices

amd radeondebiandebian-wheezylinuxlinux-kvm

I have two Radeon cards in this machine, a Radeon HD 6570 and a Radeon HD 6950:

lspci | grep VGA

01:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Turks [Radeon HD 6570]
02:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Cayman PRO [Radeon HD 6950]

I'm trying to get VGA passthrough to work with KVM on Debian 7 (Wheezy), passing through the 6950 as a secondary video card to a Windows 7 guest. This works fine if I blacklist the radeon kernel module via /etc/modprobe.d/.

If I remove the blacklist to run X11 (or even just a KMS console) on the 6570 the radeon module seems to attach to both cards:

dmesg | egrep "01:00.0|02:00.0|radeon"

pci 0000:01:00.0: [1002:6759] type 0 class 0x000300
pci 0000:01:00.0: reg 10: [mem 0xe0000000-0xefffffff 64bit pref]
pci 0000:01:00.0: reg 18: [mem 0xf7e20000-0xf7e3ffff 64bit]
pci 0000:01:00.0: reg 20: [io 0xe000-0xe0ff]
pci 0000:01:00.0: reg 30: [mem 0xf7e00000-0xf7e1ffff pref]
pci 0000:01:00.0: supports D1 D2
pci 0000:02:00.0: [1002:6719] type 0 class 0x000300
pci 0000:02:00.0: reg 10: [mem 0xd0000000-0xdfffffff 64bit pref]
pci 0000:02:00.0: reg 18: [mem 0xf7d20000-0xf7d3ffff 64bit]
pci 0000:02:00.0: reg 20: [io 0xd000-0xd0ff]
pci 0000:02:00.0: reg 30: [mem 0xf7d00000-0xf7d1ffff pref]
pci 0000:02:00.0: supports D1 D2
vgaarb: device added: PCI:0000:01:00.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: device added: PCI:0000:02:00.0,decodes=io+mem,owns=none,locks=none
vgaarb: bridge control possible 0000:02:00.0
vgaarb: bridge control possible 0000:01:00.0
pci 0000:01:00.0: Boot video device
[drm] radeon kernel modesetting enabled.
radeon 0000:01:00.0: setting latency timer to 64
radeon 0000:01:00.0: VRAM: 1024M 0x0000000000000000 – 0x000000003FFFFFFF (1024M used)
radeon 0000:01:00.0: GTT: 512M 0x0000000040000000 – 0x000000005FFFFFFF
[drm] radeon: 1024M of VRAM memory ready
[drm] radeon: 512M of GTT memory ready.
radeon 0000:01:00.0: irq 46 for MSI/MSI-X
radeon 0000:01:00.0: radeon: using MSI.
[drm] radeon: irq initialized.
radeon 0000:01:00.0: WB enabled
[drm] radeon: ib pool ready.
[drm] radeon: power management initialized
fbcon: radeondrmfb (fb0) is primary device
fb0: radeondrmfb frame buffer device
[drm] Initialized radeon 2.12.0 20080528 for 0000:01:00.0 on minor 0
radeon 0000:02:00.0: enabling device (0000 -> 0003)
radeon 0000:02:00.0: setting latency timer to 64
radeon 0000:02:00.0: VRAM: 2048M 0x0000000000000000 – 0x000000007FFFFFFF (2048M used)
radeon 0000:02:00.0: GTT: 512M 0x0000000080000000 – 0x000000009FFFFFFF
[drm] radeon: 2048M of VRAM memory ready
[drm] radeon: 512M of GTT memory ready.
radeon 0000:02:00.0: irq 49 for MSI/MSI-X
radeon 0000:02:00.0: radeon: using MSI.
[drm] radeon: irq initialized.
radeon 0000:02:00.0: WB enabled
[drm] radeon: ib pool ready.
[drm] radeon: power management initialized
fb1: radeondrmfb frame buffer device
[drm] Initialized radeon 2.12.0 20080528 for 0000:02:00.0 on minor 1
[drm] radeon: finishing device.
radeon 0000:02:00.0: ffff88041a941800 unpin not necessary
[drm] radeon: ttm finalized
pci-stub 0000:02:00.0: claimed by stub
pci-stub 0000:02:00.0: irq 49 for MSI/MSI-X

This causes the Windows 7 VM to bluescreen on boot.

How can I configure things so that module radeon only attaches to the 6570 and not the 6950?

Best Answer

I had had some pci_stub-related lines in my rc.local but this was apparently too late in the boot process to help, running after radeon started fiddling with things.

I fixed things by moving the pci_stub kernel module stuff to the initramfs:

  1. lspci -nn | grep Cayman to find the vendor:device numbers (at the very end of the line) for the 6950 and its HDMI port:

    02:00.0 VGA compatible controller [0300]: Advanced Micro Devices [AMD] nee ATI Cayman PRO [Radeon HD 6950] [1002:6719]
    02:00.1 Audio device [0403]: Advanced Micro Devices [AMD] nee ATI Cayman/Antilles HDMI Audio [Radeon HD 6900 Series] [1002:aa80]
    
  2. Pop open /etc/initramfs-tools/modules and add a new line:

    pci_stub ids=1002:6719,1002:aa80
    
  3. Rebuild the initramfs:

    sudo update-initramfs -u
    
  4. Reboot and observe pci_stub beating out radeon:

    dmesg | egrep "pci-stub|radeon"
    
    pci-stub: add 1002:6719 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
    pci-stub 0000:02:00.0: claimed by stub
    pci-stub: add 1002:AA80 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
    pci-stub 0000:02:00.1: claimed by stub
    [drm] radeon kernel modesetting enabled.
    radeon 0000:01:00.0: setting latency timer to 64
    radeon 0000:01:00.0: VRAM: 1024M 0x0000000000000000 - 0x000000003FFFFFFF (1024M used)
    radeon 0000:01:00.0: GTT: 512M 0x0000000040000000 - 0x000000005FFFFFFF
    
  5. Enjoy bluescreen-free VGA passthrough!

Related Question