Tech

Proxmox, Dockered Jellyfin & a Nvidia 3070TI

Timothy Duong

Timothy Duong

— 7 min read

This is a post for anyone that has Ubuntu VM running on a Proxmox Baremetal Host with an Nvidia GPU and would like to pass-through the PCI through to your Docker Container.

I found myself waddling through online 'documentation' (more like end-user duct-tape guides) and found that it was not as complicated as I thought only after going through the pain of trying to understand which of the configurations actually applied to me. I fortunately actually didn't need to check IOMMU tables and add driver blacklists to my Proxmox host.

What was my aim? To pass-through my GPU to my VM running Docker and into a container that hosted Jellyfin and then enable a configuration called 'Hardware accelerated encoding/transcoding'

Why hardware accelerated? Because I'm not longer using my GPU for desktop gaming and I want to speed up trickplay image creation in Jellyfin alongside freeing up CPU usage as I onboard more users to my Jellyfin server 😄

The below image with the green boxes is what I'm trying to achieve. If it does not make sense to you, I would stop reading this article here!

Green = what I'll be walking through in configuration.

Prerequisites

Finding out what your hardware supports is a key step to enabling this successfully, since everyone would have a different set-up, ranging from the Motherboard/CPU combo to the GPU itself.

  • Enable SVM in Motherboard CPU Configuration: For myself, I had a Asus x570 Motherboard AMD, paired with an AMD 5700X3D. The only thing I needed to enable was 'SVM' (Secure Virtual Machine) which is required for Proxmox to boot the Virtual Machines anyway. I did not require to enable SR-IOV (My GPU does not support vGPU and I don't want to achieve Bifurcation)
  • Working Proxmox & Ubuntu VM deployed: This article will not go through deployment of a Ubuntu VM and/or Docker on that VM.
    • If you need to learn how to set-up Promox check out Proxmox's VE Guide
    • If you need to learn how to create a Ubuntu VM in Proxmox, check out this guide by SelvarajK

Limitations

  • Enabling GPU Pass-through apparently impacts server back-ups & restore. I've yet to test this.

Now if you got up to here and just want the instructions, follow on and note, the article is broken into 3 Parts:

  • Part 1: Creating a GPU PCI Device on Proxmox Host
  • Part 2: Enabling the PCI Device in VM & Docker
  • Part 3 - Enabling Hardware Transcoding in Jellyfin

Part 1: Creating a GPU PCI Device on Proxmox Host

The following section walks us through creating a PCI Device from a pre-existing GPU that's installed physically to the Proxmox Host (e.g. Baremetal)

  1. Log into your Proxmox environment as administrator and navigate to Datacenter > Resource Mappings > PCI Devices and select 'Add'
  2. A pop-up screen will appear as seen below. It will be your 'IOMMU' Table, you will need to find your card. In my case, I selected the GeForce RTX 3070 Ti card and not 'Pass through all functions as one device' as I did not care for the HD Audio Controller. Select the appropriate device and name it too then select 'Create'
    1. Name: host-gpu-nvidia-3070ti
    2. Selection: select the GPU from the table
    3. Comment: Add a comment if you want to, e.g. a note if you will
  1. Your GPU / PCI Device should appear now, as seen below in my example as 'Host-GPU-3070Ti'
PCI Device
  1. The next step is to assign the GPU to your Docker Host VM, in my example, I am using Ubuntu. Navigate to your Proxmox Node and locate your VM, select its 'Hardware' > add 'PCI Device' and select the GPU we added earlier.
Adding PCI device to VM
Adding PCI device to VM and selecting the GPU
  1. Select 'Add' and the GPU should be added as 'Green' to the VM which means it's attached but not yet initialised. Reboot the VM.
  2. Once rebooted, log into the Linux VM and run the command
    lspci | grep -e VGA
    This will grep output all 'VGA' devices on PCI:
Image shows the default proxmox vga virtual controller alongside the nvidia controller
  1. Take a breather, make a tea/coffee, the next steps now are enabling the Nvidia drivers and runtimes to allow Docker & Jellyfin to run-things.

Part 2: Enabling the PCI Device in VM & Docker

The following section outlines the steps to allow the VM/Docker Host to use the GPU in-addition to passing it onto the docker container (Jellyfin in my case).

  1. By default, the VM host (Ubuntu) should be able to see the PCI Device, after SSH'ing into your VM Host, run lspci | grep -e VGA the output should be similar to step 7 from Part 1.
  2. Run ubuntu-drivers devices this command will out available drivers for the PCI devices.
  1. Install the Nvidia Driver - Choose either Option A or Option B. I recommend B.
    1. Simple / automation Option: Run sudo ubuntu-drivers autoinstall to install the 'recommended' version automatically
    2. Custom / Choose the version Option: Run sudo apt install nvidia-driver-XXX-server-open replacing XXX with the version you'd like if you want to server open-source version.
  2. After the installation, run sudo reboot to reboot the VM/Docker Host
  3. After reboot, run nvidia-smi to validate if the nvidia drivers were installed successfully and the GPU has been passed through to your Docker Host
3070Ti is now usable by the VM Host, next stop is the Docker Container
  1. To get the GPU/Driver working with Containers, we need to first add the Nvidia Container Runtime repositories to your VM/Docker Host
    Run the following command to add the open source repo:
    curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
  2. then run sudo apt-get update to update all repos including our newly added one
  3. then run sudo apt-get install -y nvidia-container-toolkit to install the nvidia-container-toolkit to the docker host
  4. Reboot VM/Docker-host with sudo reboot
  5. Check the run time is installed with test -f /usr/bin/nvidia-container-runtime && echo "file exists."
  6. The runtime is now installed but it is not running and needs to be enabled for Docker, use the following commands
    1. sudo nvidia-ctk runtime configure --runtime=docker
    2. sudo systemctl restart docker
    3. sudo nvidia-ctk runtime configure --runtime=containerd
    4. sudo systemctl restart containerd
  7. The nvidia container toolkit runtime should now be running, lets head to Jellyfin to test! Or of course, if you're using another application, you're good from here.

Part 3 - Enabling Hardware Transcoding in Jellyfin

  1. Your Jellyfin should currently be working but Hardware Acceleration for Transcoding is disabled. Even if you did enable 'Nvidia NVENC' it would still not work and any video should you try would error with:
Generic Error occurs when Jellyfin tries to transcode an non-working file/codec or in our case, Nvidia NVENC is not yet set-up
  1. We will need to update our Docker Compose file and re-deploy the stack/containers. Append this to your Docker Compose File.
runtime: nvidia
deploy:
  resources:
    reservations:
      devices:
        - driver: nvidia
          count: all
          capabilities: [gpu]
  1. My docker file now looks like this:
version: "3.2"
services:
  jellyfin:
    image: 'jellyfin/jellyfin:latest'
    container_name: jellyfin
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Australia/Sydney
    volumes:
      - '/path/to/jellyfin-config:/config' # Config folder
      - '/mnt/media-nfsmount:/media' # Media-mount
    ports:
      - '8096:8096'
    restart: unless-stopped
    # Nvidia runtime below
    runtime: nvidia
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

Full docker-compose file, update with your variables accordingly

  1. Log into your Jellyfin as administrator and go to 'Dashboard'
  2. Select 'Playback' > Transcoding
  3. Select 'Nvidia NVENC' from the dropdown menu
  4. Enable any/all codecs that apply
  1. Select 'Save' at the bottom
  2. Go back to your library and select any media to play.
  3. Voila, you should be able to play without that error "Playback Error
    - Playback failed because the media is not supported by this client.

Happy Watching!

Thanks to the following resources on helping me achieve this:

How to Install NVIDIA Drivers on Ubuntu 22.04 | 20.04
The Nvidia Graphics Card Drivers can often improve the performance of Ubuntu systems and often improve performance for gaming or digital editing. In the following tutorial, you will learn how to…
MediumComputingPost
Installing the NVIDIA Container Toolkit — NVIDIA Container Toolkit
NVIDIA Container Toolkit