GPU Passthrough shows black screen when VM running

Questions about virtualization software
Forum rules
Before you post read how to get help. Topics in this forum are automatically closed 6 months after creation.
Locked
daslightlybiggit
Level 1
Level 1
Posts: 3
Joined: Wed Mar 17, 2021 12:17 pm

GPU Passthrough shows black screen when VM running

Post by daslightlybiggit »

Hey folks,

I'm relatively new to virtualization and I tried to search for the topic and couldn't find it; sorry if this is a repeat. However, with everything I google, there's no straight answer and the cards are another nuance in and of themselves.
I've been trying to passthrough my GPU for the past 3 days and I've finally gotten it. Only now my VM screen is black with the terminal line just blinking. It doesn't go past that.

Here's my situation:
  • I have two GPUs, an RX 570 (first pci slot) and an Nvidia 1650 (last pci slot; I only have two on my mobo)
  • I want to passthrough (and have managed to passthrough) the RX 570 since the 1650 is in an IOMMU group that is linked with my ethernet, SATA controller, USB controller, and a whole host of other other things; so it's not worth passing.
  • I've installed Win 10 using the basic VM setup in virt manager and then tried to add the PCI host devices for my RX570 - whether I install with or without the RX570 makes no difference. The second it enters the scene: black screen.
I read that when you're using the GPU in the first pci slot, the motherboard "reserves" it but I haven't found any answer on how to "unreserve" it.
If more info is needed, please let me know, because I'm not entirely sure what will be relevant here.

Regardless, any help would be greatly appreciated!
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 1 time in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
Moonstone Man
Level 16
Level 16
Posts: 6078
Joined: Mon Aug 27, 2012 10:17 pm

Re: GPU Passthrough shows black screen when VM running

Post by Moonstone Man »

daslightlybiggit wrote: Wed Mar 17, 2021 12:28 pm Regardless, any help would be greatly appreciated!
You provide no usable technical information, not even the name of the virtual machine manager, so until you fix all that, you'll get nowhere. Please read this link: how to get help, especially section 5. It will tell you what we need to be told so that we can assist you.

If you post output from a terminal, enclose it in code tags [code]output.here[/code]. You'll see the code tags icon </> when you reply. This is necessary to make the text readable.

Finally, with this type of issue, your question may be better posted on the website or mailing list devoted solely to the unnamed VM manager.
daslightlybiggit
Level 1
Level 1
Posts: 3
Joined: Wed Mar 17, 2021 12:17 pm

Re: GPU Passthrough shows black screen when VM running

Post by daslightlybiggit »

The VM is called Virt manager or Virtual Machine Manager, I mentioned as much in my post but near the end, so I guess it's hard to see. I don't know if it goes by any other name, sorry. It runs QEMU/KVM if that's any help.

This is what I get if I follow section 5 with inxi -Fxxxrz:

Code: Select all

System:
  Kernel: 5.4.0-67-generic x86_64 bits: 64 compiler: gcc v: 9.3.0 
  Desktop: Cinnamon 4.8.6 wm: muffin 4.8.1 dm: LightDM 1.30.0 
  Distro: Linux Mint 20.1 Ulyssa base: Ubuntu 20.04 focal 
Machine:
  Type: Desktop Mobo: Micro-Star model: B450M MORTAR MAX (MS-7B89) v: 1.0 
  serial: <filter> UEFI [Legacy]: American Megatrends v: 2.40 
  date: 10/27/2019 
CPU:
  Topology: 8-Core model: AMD Ryzen 7 2700X bits: 64 type: MT MCP arch: Zen+ 
  rev: 2 L1 cache: 768 KiB L2 cache: 4096 KiB L3 cache: 16.0 MiB 
  flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm 
  bogomips: 118383 
  Speed: 1891 MHz min/max: 2200/3700 MHz boost: enabled Core speeds (MHz): 
  1: 1889 2: 1893 3: 2118 4: 2073 5: 2068 6: 2077 7: 2064 8: 2199 9: 2193 
  10: 1893 11: 1893 12: 1893 13: 1894 14: 2199 15: 2191 16: 2064 
Graphics:
  Device-1: NVIDIA vendor: ASUSTeK driver: nvidia v: 450.102.04 
  bus ID: 25:00.0 chip ID: 10de:1f0a 
  Device-2: AMD Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] 
  vendor: Sapphire Limited driver: vfio-pci v: 0.2 bus ID: 26:00.0 
  chip ID: 1002:67df 
  Display: server: X.Org 1.20.9 driver: nvidia 
  unloaded: fbdev,modesetting,nouveau,vesa tty: N/A 
  OpenGL: renderer: GeForce GTX 1650/PCIe/SSE2 v: 4.6.0 NVIDIA 450.102.04 
  direct render: Yes 
Audio:
  Device-1: NVIDIA TU106 High Definition Audio vendor: ASUSTeK 
  driver: snd_hda_intel v: kernel bus ID: 25:00.1 chip ID: 10de:10f9 
  Device-2: AMD Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] 
  vendor: Sapphire Limited driver: vfio-pci v: 0.2 bus ID: 26:00.1 
  chip ID: 1002:aaf0 
  Device-3: AMD Family 17h HD Audio vendor: Micro-Star MSI 
  driver: snd_hda_intel v: kernel bus ID: 28:00.3 chip ID: 1022:1457 
  Device-4: Texas Instruments PCM2900B Audio CODEC type: USB 
  driver: hid-generic,snd-usb-audio,usbhid bus ID: 3-3:3 chip ID: 08bb:29b0 
  Device-5: C-Media Blue Snowball type: USB 
  driver: hid-generic,snd-usb-audio,usbhid bus ID: 3-4:4 chip ID: 0d8c:0005 
  serial: <filter> 
  Sound Server: ALSA v: k5.4.0-67-generic 
Network:
  Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet 
  vendor: Micro-Star MSI driver: r8169 v: kernel port: e000 bus ID: 22:00.0 
  chip ID: 10ec:8168 
  IF: enp34s0 state: up speed: 1000 Mbps duplex: full mac: <filter> 
  IF-ID-1: virbr0 state: down mac: <filter> 
  IF-ID-2: virbr0-nic state: down mac: <filter> 
Drives:
  Local Storage: total: 6.37 TiB used: 1.22 TiB (19.2%) 
  ID-1: /dev/nvme0n1 vendor: Western Digital model: WDS100T2B0C-00PXH0 
  size: 931.51 GiB speed: 31.6 Gb/s lanes: 4 serial: <filter> rev: 211070WD 
  scheme: MBR 
  ID-2: /dev/sda type: USB model: External USB 3.0 size: 2.73 TiB 
  serial: <filter> rev: 0104 scheme: MBR 
  ID-3: /dev/sdb vendor: Crucial model: CT1000MX500SSD1 size: 931.51 GiB 
  speed: 6.0 Gb/s serial: <filter> rev: 032 temp: 31 C scheme: GPT 
  ID-4: /dev/sdc vendor: Crucial model: CT1000MX500SSD1 size: 931.51 GiB 
  speed: 6.0 Gb/s serial: <filter> rev: 032 temp: 37 C scheme: GPT 
  ID-5: /dev/sdd vendor: Crucial model: CT1000MX500SSD1 size: 931.51 GiB 
  speed: 6.0 Gb/s serial: <filter> rev: 023 temp: 35 C scheme: GPT 
Partition:
  ID-1: / size: 915.40 GiB used: 40.03 GiB (4.4%) fs: ext4 
  dev: /dev/nvme0n1p5 
Sensors:
  System Temperatures: cpu: 39.2 C mobo: N/A gpu: nvidia temp: 31 C 
  Fan Speeds (RPM): N/A gpu: nvidia fan: 27% 
Repos:
  No active apt repos in: /etc/apt/sources.list 
  Active apt repos in: /etc/apt/sources.list.d/additional-repositories.list 
  1: deb https://dl.winehq.org/wine-builds/ubuntu/ bionic main
  Active apt repos in: /etc/apt/sources.list.d/graphics-drivers-ppa-focal.list 
  1: deb http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu focal main
  Active apt repos in: /etc/apt/sources.list.d/kisak-kisak-mesa-focal.list 
  1: deb http://ppa.launchpad.net/kisak/kisak-mesa/ubuntu focal main
  Active apt repos in: /etc/apt/sources.list.d/lutris-team-lutris-focal.list 
  1: deb http://ppa.launchpad.net/lutris-team/lutris/ubuntu focal main
  Active apt repos in: /etc/apt/sources.list.d/official-package-repositories.list 
  1: deb http://packages.linuxmint.com ulyssa main upstream import backport #id:linuxmint_main
  2: deb http://archive.ubuntu.com/ubuntu focal main restricted universe multiverse
  3: deb http://archive.ubuntu.com/ubuntu focal-updates main restricted universe multiverse
  4: deb http://archive.ubuntu.com/ubuntu focal-backports main restricted universe multiverse
  5: deb http://security.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse
  6: deb http://archive.canonical.com/ubuntu/ focal partner
  Active apt repos in: /etc/apt/sources.list.d/vivaldi.list 
  1: deb http://repo.vivaldi.com/stable/deb/ stable main
Info:
  Processes: 350 Uptime: 3h 52m Memory: 31.37 GiB used: 2.80 GiB (8.9%) 
  Init: systemd v: 245 runlevel: 5 Compilers: gcc: 9.3.0 alt: 9 
  Shell: bash (sudo) v: 5.0.17 running in: gnome-terminal inxi: 3.0.38 
Before I attempted setting up a VM I enabled IOMMU, SVM, and NX in my BIOS - NX is the only I don't know if I should've enabled because I only saw it mentioned once that it needs to be enabled. Every other passthrough guide doesn't mention it. After enabling all of that, I would find my GPU's ID and ensure it's alone in its IOMMU group. There were no changes made in this step, so I don't know if the script is really needed here - purely an informational step.

The only other technical part I can think of is what changes I made in grub to have vfio control my graphics card. Particularly I made changes to "GRUB_CMDLINE_LINUX_DEFAULT"

Code: Select all

  GNU nano 4.8                   /etc/default/grub                              
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=on kvm.ignore_msrs=1 vfio-pci.ids=1002:67df,1002:aaf0
GRUB_CMDLINE_LINUX=""

# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"
After that I create a new Virtual Machine in Virtual Machine Manager add the Video card in as a PCI Host device, remove all the virtual graphics and tablet and all the other parts I don't need (I'll list them individually if it's necessary). I Press "Begin installation" and I get past the "Loading Files" part of Windows 10 installation and then black screen in the VM - can't go past that.

Sorry for the confusion, but I didn't want to vomit out information and make things more confusing. Clearly that wasn't the case.
So far, this is all I know and/or am able to explain thoroughly. Hope this clarifies things a bit.
User avatar
SMG
Level 25
Level 25
Posts: 31333
Joined: Sun Jul 26, 2020 6:15 pm
Location: USA

Re: GPU Passthrough shows black screen when VM running

Post by SMG »

On the issue of VM's I defer to those with more experience. However, I want to point out on your current Mint install only Nvidia is running the graphics.

Graphics:
Device-1: NVIDIA vendor: ASUSTeK driver: nvidia v: 450.102.04
bus ID: 25:00.0 chip ID: 10de:1f0a
Device-2: AMD Ellesmere [Radeon RX 470/480/570/570X/580/580X/590]
vendor: Sapphire Limited driver: vfio-pci v: 0.2 bus ID: 26:00.0
chip ID: 1002:67df
Display: server: X.Org 1.20.9 driver: nvidia
unloaded: fbdev,modesetting,nouveau,vesa tty: N/A
OpenGL: renderer: GeForce GTX 1650/PCIe/SSE2 v: 4.6.0 NVIDIA 450.102.04
direct render: Yes

Many desktops have a setting in BIOS/UEFI which determines the graphics card is use. Based on the data above, it seems to me yours is currently set up to only run Nvidia. That is a guess on my part as I am not familiar with your mobo and I could be wrong. However, there are zero drivers (display or renderer) for the AMD card.

So if the AMD card is not set up to render/display anything in Mint (ie. no drivers), is it possible for it to pass anything through to the VM?
Image
A woman typing on a laptop with LM20.3 Cinnamon.
daslightlybiggit
Level 1
Level 1
Posts: 3
Joined: Wed Mar 17, 2021 12:17 pm

Re: GPU Passthrough shows black screen when VM running

Post by daslightlybiggit »

SMG wrote: Fri Mar 19, 2021 6:41 pm On the issue of VM's I defer to those with more experience. However, I want to point out on your current Mint install only Nvidia is running the graphics.

Graphics:
Device-1: NVIDIA vendor: ASUSTeK driver: nvidia v: 450.102.04
bus ID: 25:00.0 chip ID: 10de:1f0a
Device-2: AMD Ellesmere [Radeon RX 470/480/570/570X/580/580X/590]
vendor: Sapphire Limited driver: vfio-pci v: 0.2 bus ID: 26:00.0
chip ID: 1002:67df
Display: server: X.Org 1.20.9 driver: nvidia
unloaded: fbdev,modesetting,nouveau,vesa tty: N/A
OpenGL: renderer: GeForce GTX 1650/PCIe/SSE2 v: 4.6.0 NVIDIA 450.102.04
direct render: Yes

Many desktops have a setting in BIOS/UEFI which determines the graphics card is use. Based on the data above, it seems to me yours is currently set up to only run Nvidia. That is a guess on my part as I am not familiar with your mobo and I could be wrong. However, there are zero drivers (display or renderer) for the AMD card.

So if the AMD card is not set up to render/display anything in Mint (ie. no drivers), is it possible for it to pass anything through to the VM?
Both of my GPUs have their drivers installed. I'm running two GPUs on one monitor; my AMD is on DisplayPort, my Nvidia is on HDMI.
Since I gave my AMD card to vfio to manage, my Displayport input just gives me a black screen (indicating the AMD GPU is being managed by software, aka vfio) and I can only use my HDMI input, which the Nvidia card is connected to.

I don't know how to better explain it, so forgive me if it all sounds confusing. Just to clarify, I've installed drivers for Nvidia, and AMD drivers were already present when I initially installed Mint (apparently they're already in the kernel?). The reason you only see Nvidia drivers is because I ran the -inxi command with my display input set to HDMI (which again, is what my Nvidia card is connected to). Had I run the command on my Displayport (the input that my AMD GPU was connected to) you'd probably see only AMD drivers. I don't know for certain.
What piqued my interest is this:
Many desktops have a setting in BIOS/UEFI which determines the graphics card is use. Based on the data above, it seems to me yours is currently set up to only run Nvidia. That is a guess on my part as I am not familiar with your mobo and I could be wrong. However, there are zero drivers (display or renderer) for the AMD card.

So if the AMD card is not set up to render/display anything in Mint (ie. no drivers), is it possible for it to pass anything through to the VM?
How would I do that? What option would I look for? I would assume an option like that would be universal so if I just know what the option is called, a quick google search will most likely point me to where the option is in my BIOS. Most MSI mobos are pretty bog-standard with how they're setup.

Outside of following your tip with the BIOS, it seems to me I'm gonna have to post further in the Virtual Manager forum (if it exists). Although, I can't possibly be the only person using Mint and passing a GPU over to a VM, am I? :?
User avatar
SMG
Level 25
Level 25
Posts: 31333
Joined: Sun Jul 26, 2020 6:15 pm
Location: USA

Re: GPU Passthrough shows black screen when VM running

Post by SMG »

daslightlybiggit wrote: Mon Mar 22, 2021 12:28 pm and AMD drivers were already present when I initially installed Mint (apparently they're already in the kernel?).
This is correct. The AMD drivers are included in the kernel and install when the
daslightlybiggit wrote: Mon Mar 22, 2021 12:28 pmThe reason you only see Nvidia drivers is because I ran the -inxi command with my display input set to HDMI (which again, is what my Nvidia card is connected to). Had I run the command on my Displayport (the input that my AMD GPU was connected to) you'd probably see only AMD drivers.
I am not aware of being able to run the inxi command on one gpu versus a different one, but there's a lot of things of which I am not aware. It's your thread and not mine, so we can ignore my confusion and not clutter the thread with it.
daslightlybiggit wrote: Mon Mar 22, 2021 12:28 pmHow would I do that? What option would I look for? I would assume an option like that would be universal so if I just know what the option is called, a quick google search will most likely point me to where the option is in my BIOS.
I was thinking of the options of discrete graphics versus an apu (onboard the cpu chip), but yours does not have the latter, therefore there may not be a setting for it.
daslightlybiggit wrote: Mon Mar 22, 2021 12:28 pmOutside of following your tip with the BIOS, it seems to me I'm gonna have to post further in the Virtual Manager forum (if it exists). Although, I can't possibly be the only person using Mint and passing a GPU over to a VM, am I? :?
I found this description of Ubuntu documentation: KVM/VirtManager . I have helped people using VirtualBox and VMWare, but this is the first of heard of Virt Manager, so I will let others help you.
Image
A woman typing on a laptop with LM20.3 Cinnamon.
Locked

Return to “Virtual Machines”