On current Strix Halo boxes (e.g., Ryzen AI MAX+), Ubuntu 25.04 “just works”: the stock kernel recognizes AMDXDNA and AMD’s Instinct 30.10 (amdgpu) + ROCm 7.0 packages install entirely via APT—no compiling, git pulls or tarballs. With Lemonade on Strix Halo, you can serve gpt‑oss‑120b (GGUF) on the iGPU through llama.cpp‑ROCm and expose an OpenAI‑compatible API. The setup is fully reproducible using uv, can be run headless, and takes very little time to get going.   Updated on 9/17/25 to reflect the ROCm 7.0 release.

All steps below use Ubuntu/Debian tooling (apt, bash, vi) and prioritize forward‑compatibility, and easy rollback.

Dual Boot Setup

• Cloned the original SSD to a 2nd M.2 NVMe drive using gparted from the Ubuntu USB live installer.
• Resized C: and Moved Recovery to create free space at the end of the disk, preserving Win11 recovery actions.
• In the system BIOS: enable SR-IOV/IOMMU, leave Secure Boot ON (allows us to enroll MOK for DKMS) – DEL to Ener BIOS, F7 for Boot Selection on the Bosgame M5

imac@ai2:~$ uname -a
Linux ai2 6.14.0-29-generic #29-Ubuntu SMP PREEMPT_DYNAMIC Thu Aug 7 18:32:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

imac@ai2:~$ journalctl -k | grep -i amdxdna
Sep 03 11:39:43 ai2 kernel: amdxdna 0000:c6:00.1: enabling device (0000 -> 0002)
Sep 03 11:39:44 ai2 kernel: [drm] Initialized amdxdna_accel_driver 0.0.0 for 0000:c6:00.1 on minor 0

The 2TB NVMe drive that came with this box is shown below. It was modified using gparted on the Ubuntu USB live boot prior to installation. New Linux users with a shipped device that includes Windows 11, may opt to to create a single new p5 partition for the Ubuntu 25.04 instance and skip the additional partitioning exercise. Creating a second new partition (p6) is not required for running Lemonade on Strix Halo, and has no impact on any steps described in this post. The Ubuntu installer will allocate all free space to a selected partition on the device during the installation process, and can “Install Ubuntu alongside Windows” handling all resizing on its own.

Disk /dev/nvme0n1: 1.86 TiB, 2048408248320 bytes, 4000797360 sectors
Disk model: KINGSTON OM8PGP42048N-A0
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 79708580-B666-424E-8D0D-C785190FA328

Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 206847 204800 100M EFI System
/dev/nvme0n1p2 206848 239615 32768 16M Microsoft reserved
/dev/nvme0n1p3 239616 616009727 615770112 293.6G Microsoft basic data
/dev/nvme0n1p4 616009728 618057727 2048000 1000M Windows recovery environm
/dev/nvme0n1p5 618057728 1071108095 453050368 216G Linux filesystem
/dev/nvme0n1p6 1071108096 4000794623 2929686528 1.4T Linux filesystem

The Strix Halo crypto performance is excellent. I wrap nvme0n1p6 with LUKS encryption, and the system hardly blinks, hitting over 13 GB/s in hardware-supported decode.  The second M.2 slot creates an opportunity for RAID 1+0 reconfiguration for added performance and redundancy, should those become considerations for a longer term deployment plan.

imac@ai2:~$ lsblk
...
nvme0n1 259:0 0 1.9T 0 disk 
├─nvme0n1p1 259:1 0 100M 0 part /boot/efi
├─nvme0n1p2 259:2 0 16M 0 part 
├─nvme0n1p3 259:3 0 293.6G 0 part 
├─nvme0n1p4 259:4 0 1000M 0 part 
├─nvme0n1p5 259:5 0 216G 0 part /
└─nvme0n1p6 259:6 0 1.4T 0 part 
└─lvm_crypt 252:0 0 1.4T 0 crypt 
└─nvme1-models 252:1 0 500G 0 lvm /mnt/models

imac@ai2:~$ cryptsetup benchmark
...
aes-xts 256b 13151.8 MiB/s 13010.5 MiB/s
...

Add AMDGPU & ROCm (preview) APT repos

• Add keys and repositories for both components and a preference to prefer them over the distribution packages.  You can browse for newer releases here and here.  NOTE: Updated to  ‘7.0.1‘ and ‘30.10.1‘ in the apt sources.  You may choose to use the older .list format, if you originally used AMD’s package to setup the repositories, as it will use the legacy .list apt format during updates. You can always use apt modernize-sources to convert .list repository files into .sources files after any upgrade if you end up with both.

# Key (/etc/apt/keyrings for user managed vs. /usr/share/keyrings where packages deploy)
curl -fsSL https://repo.radeon.com/rocm/rocm.gpg.key \
| sudo gpg --dearmor -o /etc/apt/keyrings/rocm.gpg

# AMDGPU 30.10
echo 'Types: deb
URIs: https://repo.radeon.com/amdgpu/30.10.1/ubuntu/
Suites: noble
Components: main
Signed-By: /etc/apt/keyrings/rocm.gpg' \
| sudo tee /etc/apt/sources.list.d/amdgpu.sources

# ROCm 7.0_rc1
echo 'Types: deb
URIs: https://repo.radeon.com/rocm/apt/7.0.1/
Suites: noble
Components: main
Signed-By: /etc/apt/keyrings/rocm.gpg' \
| sudo tee /etc/apt/sources.list.d/rocm.sources

# apt preferences
echo 'Package: *
Pin: release o=repo.radeon.com
Pin-Priority: 600' \
| sudo tee /etc/apt/preferences.d/rocm-pin-600

sudo apt update

Install the Graphics + Compute Stack (no compiling)

Secure Boot note: During DKMS install you’ll set a one‑time MOK password and enroll it on the next reboot so the kernel can load signed modules.

# AMDGPU kernel bits + userspace
sudo apt install amdgpu-dkms

# ROCm runtime
sudo apt install rocm rocminfo

Add your local user to the groups with access to the GPU hardware

# Group Permissions for local user to Access Hardware 
sudo usermod -a -G render,video $LOGNAME

DKMS will build kernel modules for you (okay, so technically there is some compiling and linking here, but none initiated by the user).  The rocm install pulls in a bunch of math libs and runtime packages. (rocblas rocsparse rocfft rocrand miopen-hip rocm-core hip-runtime-amd rocminfo rocm-hip-libraries)

It took me a minute to realize 30.10_rc1 is newer than 6.4.3/latest and is also aligned with 7.0_rc1. As shown below, the cli outputs the version  6.14.14 for both.  I have included this diagram from the AMD ROCm blog to show how the ROCm Toolkit and Instinct Driver (amdgpu) now evolve on separate paths.

imac@ai2:~$ dkms status
amdgpu/6.14.14-2193512.24.04, 6.14.0-29-generic, x86_64: installed

imac@ai2:~$ modinfo amdgpu | head -n 2
filename: /lib/modules/6.14.0-29-generic/updates/dkms/amdgpu.ko.zst
version: 6.14.14

imac@ai2:~$ rocminfo | head -n 1
ROCk module version 6.14.14 is loaded

imac@ai2:~$ apt show rocm-libs -a
Package: rocm-libs
Version: 7.0.0.70000-17~24.04
Priority: optional
Section: devel
Maintainer: ROCm Dev Support <rocm-dev.support@amd.com>
Installed-Size: 13.3 kB
Depends: hipblas (= 3.0.0.70000-17~24.04), hipblaslt (= 1.0.0.70000-17~24.04), hipfft (= 1.0.20.70000-17~24.04), hipsolver (= 3.0.0.70000-17~24.04), hipspars>
Homepage: https://github.com/RadeonOpenCompute/ROCm
Download-Size: 1,056 B
APT-Sources: https://repo.radeon.com/rocm/apt/7.0_rc1 noble/main amd64 Packages
Description: Radeon Open Compute (ROCm) Runtime software stack

Kernel & Memory Tunables

Strix Halo’s iGPU uses RDNA3.5 and can use GTT to dynamically allocate system memory to the GPU.  However, oversizing the GTT window could affect stability. It might also trigger OOM issues if the system lacks enough memory.  I have not tried to load anything large enough to cause this, but if you are curious about seeing oom-kill in your kernel logs, it can be triggered with memtest_vulkan on this platform. The AMD docs show a couple of environment variables that appear to control thresholds for GTT allocations to prevent this.  Limited testing has shown these, when enabled and using GTT, can prevent model loading when there is memory pressure.

One notable issue currently with Lemonade’s llama.cpp+rocm stack arises when VRAM is set to 96GB in the BIOS.  With this setting, when you try to load gpt-oss-120b,  or any large model,  you will just wait, forever.  This appears to be some kind of issue with the mmap enabled strategy in llama.cpp misbehaving, as noted in a lemonade issue here and here.  The llama-server and llama-bench under the hood of Lemonade work with –no-mmap passed directly to them, so for those trying to use Lemonade with 96GB VRAM pinned in the BIOS, there may be an option soon.

With Strix Halo, using GTT mode does not have the same restriction, and you can load models as long as you have free memory.  We now set GTT set to 125GB on a properly tuned headless system, but GTT set to 105GB was our starting point on vanilla Ubuntu desktop.

You have two strategies for gpt-oss-120b or other large models with Lemonade v8.1.10. You can either a) tune the VRAM BIOS settings down and allocate GTT as you like (27648000=105GB) or b) set your VRAM BIOS to 64GB and not use GTT.  It is unclear to this author what benefit there is leaving GTT allocated when VRAM is set in the BIOS.  For b) I set GTT to a low value of 512M, to simply avoid the default allocation of 16GB.

One note from my experience: using GTT results in slightly lower gpt-oss-120b TPS (38-41) compared to VRAM (40-45). However I have not tested this in a structured manner, or extensively as Leonard Lin. YMMV. It looks like ROCWMMA is right around the corner, which should show up in a uv –package-upgrade shortly along with using hipblaslt more effectively and enabling NPU capabilities in Linux.

a) Low VRAM, High GTT

The VRAM is set to 512MB in the BIOS and GTT is set to 105GB in the kernel parameters.  The 105GB value is based on an observation by Jeff Geerling about potential unstability with higher values. As of 9/14/25 we are running 128000M of GTT memory (amdttm.pages_limit=32768000 amdttm.page_pool_size=32768000).  GTT at 105GB is enough to load a top ten coding model (#6 on 9/14/25) like  GLM-4.5-Air-GGUF with 6bit quantization in Lemonade today.

sudo vi /etc/default/grub

# GTT to 105GB:
GRUB_CMDLINE_LINUX="transparent_hugepage=always numa_balancing=disable amdttm.pages_limit=27648000 amdttm.page_pool_size=27648000"

sudo update-grub

imac@ai2:~$ sudo dmesg | egrep "amdgpu: .*memory"
[ 3.375071] [drm] amdgpu: 512M of VRAM memory ready
[ 3.375074] [drm] amdgpu: 108000M of GTT memory ready.

With GTT set to 105GB, we can now load models beyond the 96GB BIOS limit.  On our optimized headless system (128000M of GTT memory), we load GLM-4.5-Air-UD-Q6_K_XL , Qwen3-235B-A22B-Instruct-2507-Q3_K_M.  Qwen3 235B yields about 12 TPS today. This platform’s ability to go beyond 96GB is exciting and unique.  I expect GTT performance to become equivalent to VRAM, despite being a few TPS slower today. No more BIOS tinkering. (You also must set GGML_CUDA_ENABLE_UNIFIED_MEMORY=1, described below)

Some common values for GTT are below for convienence:

131072=512MB
2097152=8G
27648000=~105GB
31457280=~120GB
32768000=~125GB

When the system is shuffling memory around (dumping disk cache), you can see the movement in the top cli tool memory statisitics.  GPT-OSS-120B takes about a minute to load, and GLM-4.5-AIR-UD-Q6-K-XL and Qwen3-235B-A22B-Instruct-2507-Q3_K_M take about five minutes to load.   Sometimes we see the following kernel message as memory is being shuffled around.

kernel: workqueue: svm_range_restore_work [amdgpu] hogged CPU for >10000us 19 times, consider switching to WQ_UNBOUND

b) High VRAM, Low GTT

The VRAM is set to 64GB in the BIOS and GTT is set to 512M in the kernel parameters. This is currently slightly more performant than Low VRAM, High GTT, but only by a few TPS for Lemonade on Strix Halo running gpt-oss-120b, so we opt to stay in GTT mode as our daily driver.

sudo vi /etc/default/grub

# GTT parameters removed:
GRUB_CMDLINE_LINUX="transparent_hugepage=always numa_balancing=disable amdttm.pages_limit=131072 amdttm.page_pool_size=131072"

sudo update-grub

If the VRAM is set higher that 64GB (96GB setting) in the BIOS, large models will not load currently.  Even with low GTT settings (512M), gpt-oss-120b will not load currently with any VRAM setting higher than 64GB.  This is due an issue with large VRAM set and the llama-server configured to use mmap.  The mmap setting is hard coded, and the result is an enless load, or an error about not being able to allocate ROCm buffers. A discussion tracked here indicates you can run llama-server manually with mmap disabled, and also notes that the GTT mode avoids this issue. A fix for this scenario is expected.

When operating with BIOS VRAM pinned to 96GB, leaving 32GB of system memory, and trying to load a large model in Lemonade (~>63GB), which ultimately fails, we see the following messages, often repeated many times.

Sep 02 17:49:11 ai2 kernel: amdgpu: SVM mapping failed, exceeds resident system memory limit

The dmesg output below shows this [current] problem configuration with VRAM set to 96GB in the BIOS and GTT to 512MB via kernel parameters. (amdttm.pages_limit=131072 amdttm.page_pool_size=131072 is shown as 512M below).

imac@ai2:~$ sudo dmesg | egrep "amdgpu: .*memory"
[ 3.333155] [drm] amdgpu: 65536M of VRAM memory ready
[ 3.333156] [drm] amdgpu: 512M of GTT memory ready.
[ 3.892481] amdgpu: HMM registered 65536MB device memory

Signal Unified Memory Allocation Logic

This setting is critical to allow loading of large models beyond 64GB in size.  A clue as to the usefulness of this flag is that it was renamed from HIP_UMA as noted here.  You can see it is defined in our sample systemd unit template below.

export GGML_CUDA_ENABLE_UNIFIED_MEMORY=1

When enabled, this setting will cause invalid values to be shown in rocm-smi currently.  Below is the output after a 98GB model has been loaded.  There is no inference going on,  as the GPU percentage reflects current load and typically goes to 99-100% when in use.

$ rocm-smi --showvbios --showmeminfo all --showuse

============================ ROCm System Management Interface ============================
========================================= VBIOS ==========================================
GPU[0] : VBIOS version: 113-STRXLGEN-001
==========================================================================================
=================================== % time GPU is busy ===================================
GPU[0] : GPU use (%): 0
==========================================================================================
================================== Memory Usage (Bytes) ==================================
GPU[0] : VRAM Total Memory (B): 536870912
GPU[0] : VRAM Total Used Memory (B): 169627648
GPU[0] : VIS_VRAM Total Memory (B): 536870912
GPU[0] : VIS_VRAM Total Used Memory (B): 169627648
GPU[0] : GTT Total Memory (B): 134217728000
GPU[0] : GTT Total Used Memory (B): 14753792
==========================================================================================
================================== End of ROCm SMI Log ===================================

Signal GEMM to use HipBlaslt

There is some design discussion on AMD’s site here.  Hipblaslt is available, and an evironment variable ensures it is used all the time, however there appear to be some quirks with it.  You can simply export the variable, to enable it on the command line.  In the systemd unit template further down in this article, you can see it defined in the [Service] description

export ROCBLAS_USE_HIPBLASLT=1

Swap File

I have disabled the swap file on my system. It seems to generate SVM messages from the kernel, usually during model load when swap is enabled.  With the current mmap issue, we see endless numbers of these with 32GB system memory while trying to load gpt-oss-120b if the swapfile is present.  No swapfile = no SVM mapping failed messages.

kernel: amdgpu: SVM mapping failed, exceeds resident system memory limit

This is achieved by simply commenting the swapfile load out of /etc/fstab, as shown in the last line below

imac@ai2:~$ cat /etc/fstab 
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
# / was on /dev/nvme0n1p5 during curtin installation
/dev/disk/by-uuid/cea76f55-f802-4ef3-a1cd-ebda84150293 / ext4 defaults 0 1
# /boot/efi was on /dev/nvme0n1p1 during curtin installation
/dev/disk/by-uuid/7E3F-BB4F /boot/efi vfat defaults 0 1
#/swap.img none swap sw 0 0

Environment Variables

There are a lot of environment variables that impact the ROCm at runtime.  ROCBLAS_USE_HIPBLASLT=1 was mentioned, and there are few others we know of that we are currently not using.

GPU_MAX_ALLOC_PERCENT=100
GPU_SINGLE_ALLOC_PERCENT=100
HSA_OVERRIDE_GFX_VERSION=11.0.0
HSA_OVERRIDE_CPU_AFFINITY_DEBUG=0
HIP_VISIBLE_DEVICES=0
ROCR_VISIBLE_DEVICES=0

llamacpp-rocm

The Lemonade team maintain their own build of llamacpp and ROCm libraries.   When Lemonade runs with debug enabled, you can see the LD_LIBRARY_PATH emitted indicating as much.  These are typically a bit newer, i.e. were 7.0_rc2  in Lemonade 8.1.8 with our local ROCm libraries at 7.0_rc1.  This close parity likely helps things along where interactions with kernel modules require ABI parity and feature alignment.

LD_LIBRARY_PATH=/path/to/ROCm/libraries # Lemonade sets to .venv/bin/rocm/llama_server

The first time you load Lemonade, you will see this custom build downloaded and added to your environment as shown here for v8.1.8.  It does not change if you go up and down Lemonade versions, so be careful to wipe it if you are rolling back Lemonade versions for test scenarios.

Sep 4 15:52:55 ai2 lemonade-server-dev[4168]: INFO: Downloading llama.cpp server from https://github.com/lemonade-sdk/llamacpp-rocm/releases/download/b1021/llama-b1021-ubuntu-rocm-gfx1151-x64.zip
Sep 4 15:53:03 ai2 lemonade-server-dev[4168]: INFO: Extracting llama-b1021-ubuntu-rocm-gfx1151-x64.zip to /home/imac/src/lemonade/.venv/bin/rocm/llama_server

Now on v8.1.10 we see the following after a uv upgrade

Sep 13 19:48:29 ai2 lemonade-server-dev[4430]: INFO: Downloading llama.cpp server from https://github.com/lemonade-sdk/llamacpp-rocm/releases/download/b1057/llama-b1057-ubuntu-rocm-gfx1151-x64.zip
Sep 13 19:48:38 ai2 lemonade-server-dev[4430]: INFO: Extracting llama-b1057-ubuntu-rocm-gfx1151-x64.zip to /home/imac/src/lemonade/.venv/bin/rocm/llama_server

Setup Python venv with uv

I prefer uv over pyenv/poetry and use a packaged version from debian.griffo.io.

# Key
curl -fsSL https://debian.griffo.io/EA0F721D231FDD3A0A17B9AC7808B4DD62C41256.asc \
| sudo gpg --dearmor -o /etc/apt/keyrings/debian.griffo.io.gpg

# Repo Source
echo 'Types: deb 
URIs: https://debian.griffo.io/apt 
Suites: trixie
Components: main
Signed-By: /etc/apt/keyrings/debian.griffo.io.gpg' \
| sudo tee /etc/apt/sources.list.d/debian.griffo.io.sources

apt update
apt install uv

Head to wherever you want to store your lemonade project.

cd ~/src/lemonade #replace with your own project location
uv init
uv venv
uv python pin 3.13 
uv add torch --index https://download.pytorch.org/whl/rocm6.4
uv sync
uv add lemonade-sdk[dev]

Lemonade on Strix Halo does not require Torch for GGUF+ROCm, but it is useful for other LLM related tools. Pinning the ROCm wheel extras index in your pyproject.toml helps resolve some dependency extras cleanly when you pull lemonade-sdk. This also avoids installing about 1GB of extra nvidia tools and libraries that will never be used with an AMD GPU.

Run Lemonade in the Background (screen)

Running in screen allows you to start Lemonade and leave it running in the background. You can then close your terminal window.   I picked a reasonable context size, which is configurable. I also set the host so that Lemonade listens on all interfaces, not just localhost. This system is on a private network.  Do not port forward or put this system on a public IP in this configuration, please.

cd ~/src/lemonade
screen -S lemony

# inside screen:
source .venv/bin/activate

lemonade-server-dev run gpt-oss-120b-GGUF \
--ctx-size 8192 \
--llamacpp rocm \
--host 0.0.0.0 \
--log-level debug |& tee -a ~/src/lemonade/lemonade-server.log

Detach from screen with CTRL-a d. Reattach with screen -r lemony. Access: http://STRIX_HALO_LAN_IP_ADDRESS:8000 from a browser on any device on the same network.  Debug log level will output TPS and other useful information, but can be removed when not needed.

Lock it down

Secure the interfaces once you have it working. In this case, only two ports are required. SSH port 22 is for administration. HTTP port 8000 is for web access to the model manager and API.

sudo ufw allow 22/tcp
sudo ufw allow 8000/tcp
sudo ufw enable

Screen is used here, but a systemd wrapper may be better for long-term use. This would be as a service to provide an API to something like Open WebUI.  When this Strix Halo is not tied up with other workloads, I have a separate Debian trixie instance that serves up Open WebUI to provide memory (look at your old chats) and advanced features for other local network users. It is a mature tool and great for engaging with private data. This is an alternative to enterprise AI chat tool subscriptions. It’s a clear candidate for enhancing household and small business productivity.

Run Lemonade at Startup (systemd)

If you are dedicating your Strix Halo to serving Lemonade, moving the service into systemd makes sense.  Using your project folder and an unprivledged user, this can be accomplished with the systemd configuration below. In my case, the project location path is /home/imac/src/lemonade.  Update /home/%i/src/lemonade in the unit file below to match your project location.  Some of the configurable environment options are explained here.

# /etc/systemd/system/lemonade@.service
[Unit]
# Running as an instance with the same name as the local user
Description=Lemonade Server (ROCm) for %i
Wants=network-online.target
After=network-online.target systemd-resolved.service

# If models in LVM, or a path that might not be ready
#RequiresMountsFor=/mnt/models

[Service]
Type=simple
User=%i
# Replace with your project location
WorkingDirectory=/home/%i/src/lemonade
# Tunable environment variables 
Environment=LEMONADE_LLAMACPP=rocm
Environment=LEMONADE_CTX_SIZE=65536
Environment=LEMONADE_HOST=0.0.0.0
Environment=LEMONADE_PORT=8000
#Environment=LEMONADE_LOG_LEVEL=debug
#Environment=ROCBLAS_USE_HIPBLASLT=1
Environment=GGML_CUDA_ENABLE_UNIFIED_MEMORY=1
#If you store your models in another locations, you can override the default huggingface cache location
#Environment=HF_HOME=/mnt/models/huggingface
# You can chose to load a model at startup, or wait to load a model using the web interface
#ExecStart=/home/%i/src/lemonade/.venv/bin/lemonade-server-dev run gpt-oss-120b-GGUF
ExecStart=/home/%i/src/lemonade/.venv/bin/lemonade-server-dev serve
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

Once the file is loaded, you can add it to your systemd configuration, and enable it to start automatically using the following commands, substituing imac with your own local username.

imac@ai2:~$ sudo systemctl daemon-reload
imac@ai2:~$ systemctl enable --now lemonade@imac.service

To see the console output, you can now use journalctl just like any other service.

imac@ai2:~$ journalctl -u lemonade@imac.service
Sep 09 16:03:21 ai2 systemd[1]: Started lemonade@imac.service - Lemonade Server (ROCm) for imac.
Sep 09 16:03:22 ai2 lemonade-server-dev[6125]: INFO: Started server process [6125]
Sep 09 16:03:22 ai2 lemonade-server-dev[6125]: INFO: Waiting for application startup.
Sep 09 16:03:22 ai2 lemonade-server-dev[6125]: INFO:
Sep 09 16:03:22 ai2 lemonade-server-dev[6125]: 🍋 Lemonade Server v8.1.8 Ready!
Sep 09 16:03:22 ai2 lemonade-server-dev[6125]: 🍋 Open http://0.0.0.0:8000 in your browser for:
Sep 09 16:03:22 ai2 lemonade-server-dev[6125]: 🍋 💬 chat
Sep 09 16:03:22 ai2 lemonade-server-dev[6125]: 🍋 💻 model management
Sep 09 16:03:22 ai2 lemonade-server-dev[6125]: 🍋 📄 docs
Sep 09 16:03:24 ai2 lemonade-server-dev[6125]: INFO: Application startup complete.
Sep 09 16:03:24 ai2 lemonade-server-dev[6125]: INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
Sep 09 16:03:25 ai2 lemonade-server-dev[6125]: Starting Lemonade Server...
Sep 09 16:03:25 ai2 lemonade-server-dev[6125]: Downloading gpt-oss-120b-GGUF (unsloth/gpt-oss-120b-GGUF:Q4_K_M)
Sep 09 16:03:25 ai2 lemonade-server-dev[6125]: [143B blob data]
Sep 09 16:03:25 ai2 lemonade-server-dev[6125]: INFO: 127.0.0.1:47284 - "POST /api/v1/pull HTTP/1.1" 200 OK
Sep 09 16:03:25 ai2 lemonade-server-dev[6125]: INFO: Loading llm: gpt-oss-120b-GGUF
Sep 09 16:03:25 ai2 lemonade-server-dev[6125]: INFO: Using backend: rocm
Sep 09 16:04:08 ai2 lemonade-server-dev[6125]: INFO: 127.0.0.1:47292 - "POST /api/v1/load HTTP/1.1" 200 OK

Running with 128000M GTT (~125GB) yielded about 1GB of free memory during inference using with GLM-4.5-Air-GGUF-Q8_0 on a service optimized Ubuntu 25.04 desktop system running headless with debug logging enabled.  It was under a few TPS at 128k context, but loaded.

root@ai2:~# free
total used free shared buff/cache available
Mem: 128649272 29146652 1064980 374200 99999544 99502620
Swap: 0 0 0

Basic Local Monitoring

Messages about VRAM and GTT allocations and ongoing SVM mapping failures

journalctl -b | egrep "amdgpu: .*memory"

Follow logs live to see errors in realtime

journalctl -f

Watch the GPU memory use

watch -n1 /opt/rocm/bin/rocm-smi --showuse

Inspect VRAM capacity

rocm-smi --showvbios --showmeminfo vram --showuse
rocm-smi --showvbios --showmeminfo gtt --showuse
rocm-smi --showvbios --showmeminfo all --showuse

Evergreening

apt update && apt upgrade

Check for bumps in the apt repositories here and here. Move to the stable or latest when the preview releases become final.  In the opinion of the author, there is a misleading note in the official AMD instructions that suggests ‘Upgrades and downgrades’ are not supported.  I can only guess this is a hangover from pre-packaging, pre-dkms, or from non-Debian-based distributions.  As support for this observation, the official instructions describe the ‘Uninstalling procedure’ as the apt package manager steps in a similar manner to what is expected to happen automatically during a regular apt upgrade.

uv lock --upgrade # Make a copy of uv.lock first for rollback, if not using a git repo with a tag
uv lock --upgrade-package lemonade-sdk 

Also keep an eye on your torch wheel if you are using torch, and either update the index in your pyproject.toml or remove it so its dependencies can not conflict with lemonade.

[[tool.uv.index]]
url = "https://download.pytorch.org/whl/rocm6.4"

Initial State 9/3/25 (At Publication)

apt managed

ii linux-image-6.14.0-29-generic 6.14.0-29.29 amd64 Signed kernel image generic
ii amdgpu-dkms 1:6.14.14.30100000-2193512.24.04 all amdgpu driver in DKMS format.
ii rocm 7.0.0.70000-17~24.04 amd64 Radeon Open Compute (ROCm) software stack meta package
ii uv 0.8.14-1+trixie amd64 An extremely fast Python package and project manager, written in Rust.

uv managed

lemonade-sdk 8.1.7 / llamacpp-rocm b1021
torch 2.8.0+rocm6.4

Current State 9/13/25 (Evergreening)

apt managed

ii linux-image-6.14.0-29-generic 6.14.0-29.29 amd64 Signed kernel image generic
ii amdgpu-dkms 1:6.14.14.30100000-2193512.24.04 all amdgpu driver in DKMS format.
ii rocm 7.0.0.70000-17~24.04 amd64 Radeon Open Compute (ROCm) software stack meta package
ii uv 0.8.17-1+trixie amd64 An extremely fast Python package and project manager, written in Rust.

uv managed

lemonade-sdk 8.1.10 / llamacpp-rocm b1057
torch 2.8.0+rocm6.4

Open WebUI

If you do want to spawn Open WebUI, similar steps below should work on Debian Trixie, and probably also on any Ubuntu Plucky instance.

sudo apt install pkg-config python3-dev build-essential libpq-dev
uv init 
uv venv
uv python pin 3.11
uv sync
source .venv/bin/activate
uv pip install setuptools wheel
uv add open-webui
open-webui serve

Performance

With the release of ROCm 7.0, a lot of people might be wondering what kind of performance they can expect.  This article was based on Ubuntu 25.04, as it provides a convienent way to enjoy a complete desktop experience right on top of a Strix Halo device, while taking advantage of 100GB+ models.  For more permanent workloads, we prefer pure Debian for a cleaner multi-user target that more closely resembles a commercial production environment without any overhead from desktop packages, and stricter policies and release cycles on the underlying operating system.  Debian 12 is fully supported by AMD (using the jammy repo), and we expect Ubuntu 25.04, and its Debian 13 base, to be added to the official mix any day now.  Below we include benchmarks on both Ubuntu and Debian, as well as a comparison to a Radeon RX 7900 XTX GPU in our benchmarks, using some popular models.

llama-bench is included with the lemonade package, and is easily marked executable to avoid having to pull down any additional code, or build any packages from source to execute local benchmarks of various models. It can be pointed directly at the models downloaded in the Lemonade model manager from Hugging Face to avoid any file replication, which quickly adds up with the larger models.

QWEN3-30B-Coder-A3B – 48 t/s – Strix Halo – Debian 12 – ROCm 6.4 – LLAMA-ROCM b1057 (Lemonade v8.1.10)

There is almost no change on Debian 12 with Linux 6.1+ROCm 6.4 and moving to ROCm 7.0.  Switching to Ubuntu Linux 6.14+ROCm 7.0  showing a 23% improvement in generation over Debian 12. We might expect similar for gains over Ubuntu 22.04 and will likely upgrade to Debian 13 at a later date. 

QWEN3-30B-Coder-A3B – 59 t/s – Strix Halo – Ubuntu 25.04 – ROCm 7.0 – LLAMA-ROCM b1057 (Lemonade v8.1.10)

GPT-OSS-120B – 46 t/s – Strix Halo – Debian 12 – ROCm 6.4 – LLAMA-ROCM b1057 (Lemonade v8.1.10)

The difference between Debian 12 (6.1) with ROCm 6.4 and Ubuntu 25.04 (6.14) with ROCm 7.0 for this model is much smaller than with Qwen 30B.

GPT-OSS-120B – 47 t/s – Strix Halo – Ubuntu 25.04 – ROCm 7.0 – LLAMA-ROCM b1057 (Lemonade v8.1.10)

QWEN3-30B-Coder-A3B – 98 t/s – RX 7900 XTX – Ubuntu 25.04 – ROCm 7.0 – LLAMA-ROCM b1057 (Lemonade v8.1.10)

The result below shows you a 24GB RDNA 3 card executing the same test.  Just over twice the performance on generation, but that is about as large a model as it can handle.  For 20GB models, these GPUs are available for approx $600 USD. (9/25)

 

QWEN3 Big and Small – 235B-Coder-A22B Instruct 2507 – 11 t/s – Strix Halo – Debian 12 – ROCk 7.0 – LLAMA-ROCM b1057 (Lemonade v8.1.10)

Here we see how the difference in performance running some of the smallest and largest Qwen3 models.   At 11 t/s if there is nothing else going on, letting Qwen 235B make optimizations and improvements to existing code, is just a fun background task.

Ready to Build on This?

Building for the future without creating technical debt is a powerful paradigm. But the real business advantage comes from mapping your unique business logic into multi-agent AI workflows that solve real problems and create real scalability.

At Netstatz Ltd., this is our focus. We leverage our enterprise experience to build intelligent agent systems on stable, secure, and cost-effective edge platforms like Strix Halo. If you are a small or medium sized business looking to prototype or deploy local AI solutions, contact us to see how we can help.  If you’re in the Toronto area, we can grab a coffee (or a beer) and talk shop.