Skip to content
Compute Function

The inference workbench · GPU recommender

Find a home for your model.

Start with your traffic. Choose a model. Explore the memory footprint your inference workload could need.

01 · Your traffic

1 / day1M / day
Short promptsLong documents
Brief answersLong generations
Peak traffic assumptions

Peak concurrent requests = daily requests ÷ 86,400 × peak multiplier × response duration. Use measured values when available. These assumptions are not model performance benchmarks.

02 · Choose your model

An owner/model ID works too. Uses the main branch.

Or choose a popular model

18 publisher presets · Downloads checked October 17, 2026

Popular picks are ranked by monthly downloads among our supported publisher presets. Downloads indicate interest, not production usage. Hugging Face source ↗

View model configuration ↗

03 · Precision

Lower precision reduces memory, with potential quality and compatibility tradeoffs. Quantized weights and FP8 cache require support from your model, hardware and serving runtime.

Monthly tokens
4.6B

Input + output · 30 days

GPUs required
1

H100 SXM

Estimated memory fit

30-day month · 1,024 input + 512 output tokens/request. 105 peak concurrent requests, assuming 30s per request and 3× average traffic.

Memory estimate · 48.2 GB — view breakdown

04 · Your estimated memory footprint

48.2GB

Qwen3 8B · 1.5K context · 105 concurrent

Weights
16.4 GB
KV cache
23.8 GB
Allowance
8 GB

Memory estimate, including a 20% planning allowance. Throughput and latency require benchmarking against your serving stack.

Configurations to explore

Smallest memory fit per GPU type

These configurations fit estimated memory, not measured token throughput. GPU count for your traffic, utilization and cost require serving benchmarks and a quote. These are hardware possibilities; availability, runtime support, performance and pricing are confirmed during quoting.

What goes into the estimate?

Weights use parameter count × bytes per weight. KV cache uses 2 × layers × key/value heads × head dimension × context × concurrency × cache bytes. We add 20% to the combined total for planning. Figures use decimal GB. Presets and imported models use the Hub’s published total safetensors parameter count. All expert weights are included for MoE models. A full KV cache is assumed, so sliding-window runtimes may use less memory.

What should I validate before deploying?

This tool estimates text-request memory with a full-context attention envelope. Hybrid and multimodal models include architecture notes above. Activation peaks, CUDA graphs, quantization metadata and runtime buffers vary. It does not model training, expert routing or offloading, image/audio activation memory, throughput or service levels. Benchmark your workload and confirm per-device memory before reserving capacity.