How to Deploy GLM-4.5-Air with SGLang on a Dedicated GPU Server

Running a large language model on dedicated hardware gives you full control over the serving environment, model configuration and API access. In this guide, we deploy GLM-4.5-Air-FP8 from Z.ai on an NVIDIA GPU server using SGLang, exposing an OpenAI-compatible API.

This is a condensed version. The complete tutorial is on Leo Servers.

About the model

GLM-4.5-Air is a mixture-of-experts (MoE) model built for reasoning, coding and agent tasks. It has about 106B total parameters and about 12B active during inference. We use the FP8 checkpoint:

zai-org/GLM-4.5-Air-FP8 

Hardware

Z.ai documents these NVIDIA configurations for GLM-4.5-Air-FP8:

The full-featured conditions also call for more than 1TB of system RAM, a batch size of eight or less, and no CPU offloading. Treat these as a reference, not a universal minimum.

Prepare Ubuntu and verify the GPUs

sudo apt update && sudo apt upgrade -y sudo apt install -y curl git wget build-essential ubuntu-drivers devices sudo ubuntu-drivers install sudo reboot 

After the reboot:

nvidia-smi nvidia-smi -L nvidia-smi topo -m 

Install SGLang

curl -LsSf https://astral.sh/uv/install.sh | sh source "$HOME/.local/bin/env" mkdir -p ~/glm-deployment && cd ~/glm-deployment uv venv --python 3.12 source .venv/bin/activate uv pip install --prerelease=allow sglang 

Point the Hugging Face cache at a large volume:

sudo install -d -o "$USER" -g "$(id -gn)" /data/huggingface echo 'export HF_HOME=/data/huggingface' >> ~/.bashrc source ~/.bashrc 

Launch the model

python -m sglang.launch_server \ --model-path zai-org/GLM-4.5-Air-FP8 \ --served-model-name glm-4.5-air-fp8 \ --host 127.0.0.1 \ --port 30000 

Binding to 127.0.0.1 keeps the API off the public interface.

Test the API

curl http://127.0.0.1:30000/v1/models 

from openai import OpenAI client = OpenAI(api_key="local-development", base_url="http://127.0.0.1:30000/v1") response = client.chat.completions.create( model="glm-4.5-air-fp8", messages=[{"role": "user", "content": "Explain the role of GPU memory in LLM inference."}], max_tokens=300, ) print(response.choices[0].message.content) 

Run it with systemd and secure it

A systemd unit keeps SGLang running and restarts it on failure. For remote access, use an authenticated HTTPS reverse proxy, firewall rules and private networking. Never expose an unauthenticated inference endpoint to the internet.

Read the full guide

The full tutorial includes the complete systemd unit file, monitoring tips and a troubleshooting table:

👉 How to Deploy Zhipu AI GLM-4.5-Air with SGLang on a Dedicated GPU Server