# Render on the GPU

> Unity, Godot, Blender, Chrome, FFmpeg and any Vulkan, CUDA or video app on the workspace GPU: launch settings, video encode and decode, live status.

On the Startup and Scale plans every workspace renders on a GPU through Shardflux's own Vulkan driver, with the CUDA
Driver API (`libcuda.so.1`), NVENC and NVDEC (`libnvcuvid.so.1`) on the same GPU. Vulkan (Vulkan Video included), CUDA,
NVENC, NVDEC and VA-API apps run unchanged, headless or on the workspace desktop, and in containers started with
`--gpus all` or `runtime: nvidia`. OpenGL, EGL and OpenCL run on the
GPU through Mesa's zink in login shells. [GPU-hours](https://docs.shardflux.dev/limits.md#gpu-hours) measure what your apps use.

## Launch settings

| App | Settings |
| --- | --- |
| Unity | `-batchmode -force-vulkan` renders offscreen: `Camera.Render`, `RenderTexture` and `ReadPixels` work. A windowed player or the editor needs a display: `DISPLAY=:1` (the desktop) or Xvfb. Leave out `-nographics`, which turns rendering off. |
| Godot | `--rendering-driver vulkan` on a display (`DISPLAY=:1` or Xvfb). `--headless` turns rendering off. |
| Blender EEVEE and Workbench | `--gpu-backend vulkan`, with `SFGPU_READBACK=1` in the environment. |
| Blender Cycles | The CUDA compute device (below). |
| Chrome and Chromium | `--use-angle=vulkan --enable-features=Vulkan` puts WebGL, WebGL2 and WebGPU on the GPU, headed or headless. On the desktop, `--use-angle=vulkan --enable-features=Vulkan,VulkanFromANGLE,DefaultANGLEVulkan,VaapiIgnoreDriverChecks` also decodes H.264, VP9 and AV1 video on the GPU. |
| FFmpeg | See [Video](#video). |

## Video

FFmpeg 7, 8 and 9 encode and decode on the GPU through any of the three video APIs:

| API | Encode | Decode |
| --- | --- | --- |
| NVENC and NVDEC | `-c:v h264_nvenc`, `hevc_nvenc`, `av1_nvenc` | `-hwaccel cuda`, `-c:v h264_cuvid`, `hevc_cuvid` |
| VA-API | `-vaapi_device /dev/dri/renderD128 -vf format=nv12,hwupload -c:v h264_vaapi` (or `hevc_vaapi`) | `-hwaccel vaapi -hwaccel_device /dev/dri/renderD128`: H.264, HEVC (8 and 10 bit), VP9 and AV1 |
| Vulkan Video | `-init_hw_device vulkan=vk -filter_hw_device vk -vf format=nv12,hwupload -c:v h264_vulkan` (or `hevc_vulkan`) | `-hwaccel vulkan`: H.264, HEVC, VP9 and AV1 |

Other apps that use VA-API or Vulkan Video use the same drivers. Use `av1_nvenc` for AV1 encoding. A container
gets video with `--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all`. Video sessions are counted per workspace
([Limits](https://docs.shardflux.dev/limits.md)).

## Fast Blender renders

Cycles renders on the GPU with CUDA. Choose the CUDA device in the script you render with:

```python
import bpy

prefs = bpy.context.preferences.addons["cycles"].preferences
prefs.compute_device_type = "CUDA"
prefs.refresh_devices()
for device in prefs.devices:
    device.use = device.type == "CUDA"
bpy.context.scene.cycles.device = "GPU"
```

Run one long-lived Blender process per render worker and give it frame after frame: a whole frame range in one run,
or a script that renders the frames it takes from a queue.

```sh
blender -b shot.blend --python use_cuda.py -s 1 -e 240 -a
```

Each Blender process opens its own GPU session, creates its CUDA context, uploads the Cycles kernels and compiles the
EEVEE shaders before its first frame. A worker that keeps one process for the whole job pays that once; starting a
process for every few frames pays it every time.

## Frames on request

Offscreen work and windows off the desktop render the frames you ask for. Each GPU app has a control socket,
`/run/sfgpu/<pid>.sock`:

```sh
echo 'frame path=/tmp/frame.png' | nc -U /run/sfgpu/<pid>.sock
```

An app drawing to a window on the desktop, and any app started with `SFGPU_MODE=live`, renders every frame. Readbacks
work with no settings: `ReadPixels`, `ComputeBuffer.GetData` and copies into mapped memory return the GPU's results.
What a frame request can answer is in [GPU frame requests](https://docs.shardflux.dev/reference/errors.md#gpu-frame-requests).

Between requests the scene waits off the GPU. Its GPU memory leaves the GPU a fraction of a second after the last frame
and comes back with the next request, with every buffer, texture and pipeline the app created kept as it was: the next
frame of a 1 GiB scene comes back in about 0.12 s on an L4. [GPU-hours](https://docs.shardflux.dev/limits.md#gpu-hours) count a scene's GPU memory
only while it is on the GPU.

## See what each GPU app is doing

`status` on an app's control socket answers at once, also while its GPU work runs, with one JSON line:

```sh
echo status | nc -U /run/sfgpu/<pid>.sock
```

| Field | Meaning |
| --- | --- |
| `session` | `admitted`: on the GPU. `waiting`: in line for room on the GPU, at `queue_position` (1 is next), for `waiting_ms` so far. `parked`: its GPU state waits off the GPU (`parked_on`: `app` or `host`) and comes back at its next GPU call. `none`: no GPU session yet. Also `connecting`, `reconnecting`, `refused` (the workspace has no GPU access), `cpu` (OpenGL went on with the CPU renderer) and `lost` (restart the app). |
| `pinned` | `cuda` or `video`: CUDA, NVENC or video codec state keeps the session on the GPU, with its GPU memory, until the app exits. |
| `cuda` | The app uses CUDA, NVENC or NVDEC. |
| `vram_bytes` | GPU memory the app has allocated through Vulkan. |
| `last_work_ms` | Time since the app last submitted GPU work: a Vulkan submit or a CUDA, NVENC or NVDEC call. `-1` before its first. |
| `gpu_wait_ms` | How long the app's oldest call has been waiting for the GPU's answer, such as a fence wait or a CUDA synchronize. `0` when none waits. |
| `readback` | `all` with `SFGPU_READBACK=1`, else `app`. |
| `mode` | `live` (every frame renders) or `skip` (frames on request). |
| `frames` | Frames the app has presented. |
| `gpu` | The shared GPU from its latest report, `age_ms` old: `util` (percent busy), `vram_used` and `vram_total` in bytes, and `queued`, the GPU sessions waiting for room. |
| `preview` | What the app's window on the desktop shows. |