Choosing a Cluster¶
This tutorial needs an OpenShift 4.20+ cluster where you can install operators and (ideally) attach a GPU. Several cluster options work — pick the one that matches your access and budget.
Why 4.20?
Red Hat OpenShift AI 3.x requires OpenShift 4.19 or later, and RHOAI 3.x
distributed inference (e.g. llm-d) requires 4.20. This tutorial is
validated on OpenShift 4.20 + RHOAI 3.3 (channel fast-3.x).
At a glance¶
| Option | GPU access | Time to ready | Cost | Path |
|---|---|---|---|---|
| Self-managed OpenShift | Yes (you provision) | Hours | Hardware/cloud | A |
| ROSA (Red Hat OpenShift on AWS) | Yes (GPU node pool) | ~1 hour | AWS billing | A |
| ARO (Azure Red Hat OpenShift) | Yes (GPU node pool) | ~1 hour | Azure billing | A |
| Local CRC (CodeReady Containers) | No (single-node) | ~30 min | Free | B |
| Red Hat Developer Sandbox | No | Instant | Free | B |
Path A = full tutorial experience with on-cluster vLLM.
Path B = deploy everything except the LLM; supply an external
OpenAI-compatible endpoint via MODEL_ENDPOINT.
Self-managed OpenShift¶
Install OpenShift on bare metal, VMware, or any supported platform. You'll need at least one worker node with a GPU (NVIDIA A10, L4, A100, H100, or similar — ~24 GB VRAM minimum for Granite 3.3 8B at fp16).
This is the most flexible option but also the most work.
→ Red Hat OpenShift install documentation
ROSA / ARO¶
Managed OpenShift on AWS or Azure. Add a GPU machine pool after the cluster comes up. Billing is hourly — remember to scale GPU nodes to zero when not in use.
→ ROSA documentation → ARO documentation
Local CRC¶
CodeReady Containers runs a single-node OpenShift on your laptop. There's no GPU, so vLLM serving is not feasible — use Path B.
CRC is fine for working through the agent / MCP server / gateway / UI parts of the tutorial against a remote LLM endpoint.
Red Hat Developer Sandbox¶
The Developer Sandbox gives you a shared OpenShift environment with no install required. There are no GPUs and limited resource quotas, but it's the fastest way to get the agent stack running. Use Path B.
FIPS mode¶
If you're standing up a fresh cluster and your environment requires (or might require) FIPS, enable it at install time — FIPS cannot be enabled on an existing cluster. Every component built in this tutorial works in FIPS mode.
→ Installing a cluster in FIPS mode
Next¶
Once you have a cluster: Install OpenShift AI.