DGX Station profile: the local AI box everyone wants benchmarked

NVIDIA says DGX Station can support models up to 1 trillion parameters.

The machine has 748GB of coherent memory, but only 252GB of it is HBM. A 1T-class model can fit only under specific assumptions about the model, quant, cache, and runtime. Reading the 748GB number as 748GB of GPU-speed memory would be a very expensive mistake.

We went looking for numbers that clarify the difference.

We have been trying to understand the actual machine underneath the pitch. We have talked to Cornell researchers who had temporary access. We talked to NVIDIA. We inquired about buying one and priced it at about $100,000 per machine. We searched the NVIDIA forums, Reddit, X, LocalMaxxing, model cards, and conference-floor posts. We asked people at the AI Engineer local table for GLM-5.2 numbers because public numbers were thin.

This profile comes from that reporting. We have useful numbers, but no full benchmark suite yet.

The local AI interest itself is part of the story. At AI Engineer, the Local AI room was full of people asking the same memory-residency and throughput questions we were asking.

Photo credit: Ahmad Osman. Original X post: x.com/TheAhmadOsman/status/2072789682254180776.

Does DGX Station let you buy a lot of local model capacity without giving up the memory behavior that makes GPU inference fast?

NVIDIA's DGX Station page says the machine is powered by GB300 Grace Blackwell Ultra, has 748GB of coherent memory, and supports models up to 1 trillion parameters.

It does not give you 748GB of HBM.

The split that matters is:

If your model and KV cache stay inside HBM, the machine is easy to understand. If the active workload crosses into LPDDR5X, the question becomes empirical: how much does prefill, decode, long context, and concurrency change?

Stas Bekman measured NVLink-C2C on DGX Station and reported that it was not behaving like 900GB/s full duplex end-to-end in his bidirectional test. Marketing bandwidth numbers do not automatically describe workload behavior.

Actual runs matter more than spec sheets.

The DGX Station price we got was about $100,000 per machine.

At that price, compare it with other ways to answer the same three questions: what can we run, how does it feel, and how much does it cost? The GLM-5.2-specific version of that comparison is in the GLM 5.2 local hardware requirements post.

For DGX Station, the competing buckets are:

The buying question is broader than "can I fit a big model?" A lot of machines can fit a heavily quantized big model somewhere in memory.

The buying question is: can we serve useful local frontier-ish workloads with enough speed, context, and concurrency to justify a six-figure workstation?

The objection is specific.

This r/LocalLLaMA thread about 4x-8x RTX PRO 6000 systems, especially the comment thread starting here, gets at the objection. Someone asks why not buy DGX Station instead of 4-8 RTX PRO 6000s. The reply is basically: DGX Station does not actually have 748GB of VRAM. It has a 252GB HBM tier plus a larger LPDDR5X tier.

The NVIDIA Developer Forums have the same question in more formal language: can a 1T-class model really be served well on this memory layout, and has anyone seen real tokens/sec benchmarks for the GB300 DGX Station when context or weights move past HBM3e?

The useful standard is what happens to prefill, decode, context, and concurrency when the workload crosses the fast memory tier.

DGX Station access is still scarce enough that the list of people with credible hands-on signal matters.

From our reporting, the names and groups we kept seeing were: Cornell's group, temporarily; Stas Bekman and Jeff Rasley around Snowflake work; Alex Cheema at Exo with Ahmad Osman and the local AI table around AI Engineer; Andrej Karpathy; and Matthew Berman.

Those names produced different kinds of signal. The public evidence is still coming from a small circle.

These are the public photos we can point at: lab machines, deliveries, show-floor hardware, Exo's local AI setup, and Snowflake's interior shot.

The most useful academic writeup we found came from Kilian Weinberger's group at Cornell, which had early remote access to a DGX Station in February. Three researchers used it on three different workloads: RL fine-tuning, diffusion language-model retrieval, and synthetic data generation.

The article is useful because it reports actual workloads. The RL work used the coherent memory to move from constrained Qwen3-4B runs toward cleaner 4B-7B experiments. The diffusion-LM retrieval work used the local GB300 box to iterate without cluster queues, doubling training batch size and cutting epoch time by about 20% versus their previous shared-cluster setup. The synthetic-data work ran Qwen3-30B-A3B-Instruct in BF16 with vLLM and reported 5.7x the throughput of a single A100 and 2.6x the throughput of a 4xA100 setup.

Stas Bekman and Snowflake published another useful example: post-training Qwen3-32B at 136K sequence length on one DGX Station. That run used ArcticTraining, TiledMLP, Liger-Kernel, BF16 optimizer offload in DeepSpeed, and CPU memory to make a long-sequence SFT recipe fit on a single B300. The writeup also notes the setup work that mattered: stable memory behavior, avoiding fragmentation, and switching the machine into the right coherent-memory mode so CPU memory was available for optimizer-state offload instead of fighting the GPU.

Cornell and Snowflake showed the machine helping real training and research workloads. Serving a 325GB or 500GB-class local frontier artifact on a machine with 252GB of HBM is a different question. They did not settle the local frontier-inference question. We talked to them about the big questions we had, and none of them knew the answer.

The buyer question:

NVIDIA says DGX Station supports models up to 1 trillion parameters. We believe the capacity claim, with all the caveats above. A buyer should not read that as "load the full FP8 weights of any 700B-1T frontier open model and expect it to behave like it all lives in HBM."

The best public GLM-5.2 clue we have is the AI Engineer booth run. We had been asking for any real GLM-5.2 number on DGX Station from people standing next to the machine.

On X, we asked the AI Engineer local table whether anyone could post metrics for the GLM-5.2 4-bit quant running on DGX Station. A couple hours later, 0xSero posted: GLM-5.2-REAP on DGX Station, drumroll...

Rick Blalock posted a short X video of that booth demo. You can get a visual sense of the generation speed.

For cloud context, OpenRouter's Claude Opus 4.8 page shows 64 tok/s as the best throughput across providers, and about 55 tok/s for Anthropic in its one-week provider breakdown.

Alec Fong later added that their first stab at running GLM with NVFP4 was about 25 tok/s, before Alex Cheema and 0xSero helped push the run to 60 tok/s. The evidence trail points toward Luke Alonso's GLM-5.2-NVFP4: Ahmad had posted that Luke uploaded a 467GB NVFP4 artifact, then replied to Sentdex that he was working on getting it running. Alec's post does not name the exact artifact, so we are treating that as strong context, not a confirmed model ID.

The model selector appears to show GLM-5.2 REAP 504B, which points to the 0xSero GLM-5.2 REAP 504B GGUF family rather than the full GLM-5.2 FP8/BF16 model. REAP is expert pruning for MoE models, and 0xSero's own Terminal-Bench 2.1 number is lower than the full-model number we have seen.

The model that ran locally was shaped to run locally.

Here are the big-model numbers we have been able to get:

These numbers deserve credit because they were hard to come by. They also have different measurement conditions, so the table should not be read as a leaderboard.

For the GLM-5.2 model-sizing side, see the GLM 5.2 local hardware requirements. This profile is about DGX Station as a machine.

NVIDIA lists DGX Station systems from ASUS, Dell, Exxact, Gigabyte, HP, MSI, and Supermicro on the official DGX Station page, and its marketplace links out to buying options for GB300 systems.

The lineup is already messy in the way workstation launches are messy: some pages are configure-and-quote, some are sales conversations, some are regional distributors, and some public photos are special hand-delivered early systems rather than normal orders.

A LocalLLaMA user put the GB300 OEM systems side by side. It is a community visual, not an official sizing chart. Source: r/LocalLLaMA.

Here is what we could pin down as of July 2, 2026:

Privately, we heard from an NVIDIA rep that the first non-special-delivery orders were starting to move this week, with Supermicro expected first. We separately heard 4-6 weeks lead time from the Supermicro side. The public early-delivery photos, though, have mostly been Dell Pro Max with GB300 units.

For DGX Station, a useful benchmark needs to disclose:

At Cyrus, we are going to keep collecting these numbers and publish the ones we can stand behind.

Right now, DGX Station looks like the most serious local frontier box you can plausibly buy as a single machine. It also still has the exact question local AI people care about: what happens when the workload needs more than 252GB of HBM?

That is the benchmark we want. That is the benchmark we're going to get you.