AMDGPU Driver version updated (#6311 ) (#6312 )

Update preview release link in README
(cherry picked from commit 89321ae826)
2026-05-28 17:59:06 -04:00 · 2026-05-25 09:26:49 -04:00 · 2026-05-25 09:26:49 -04:00 · 2026-05-19 13:44:33 -04:00 · 2026-05-12 14:56:45 -04:00 · 2026-05-05 11:46:04 -04:00
27 changed files with 3919 additions and 439 deletions
@@ -3,11 +3,8 @@ ABI
 ACE
 ACEs
 ACS
-AccVGPR
-AccVGPRs
 AITER
 ALU
-AllReduce
 AMD
 AMDGPU
 AMDGPUs
@@ -22,31 +19,34 @@ APIC
 APIs
 APU
 APUs
+ASAN
 ASIC
 ASICs
 ASan
-ASAN
 ASm
-Async
 ATI
-atomicRMW
+AccVGPR
+AccVGPRs
 AddressSanitizer
 AlexNet
+AllReduce
 Andrej
 Arb
+Async
 Autocast
-autograd
-Backported
 BARs
-BatchNorm
 BKC
 BLAS
 BMC
 BabelStream
+Backported
+BatchNorm
+Bitwise
 Blit
 Blockwise
 Bluefield
 Bootloader
+BrainFloat
 Broadcom
 CAS
 CCD
@@ -54,19 +54,18 @@ CDNA
 CGUI
 CHTML
 CIFAR
-CNP
 CLI
 CLion
 CMake
 CMakeLists
 CMakePackage
+CNP
 CP
 CPC
 CPF
 CPP
 CPU
 CPUs
-Cron
 CSC
 CSDATA
 CSE
@@ -77,13 +76,12 @@ CTests
 CU
 CUDA
 CUs
-CXX
 CX
+CXX
 Cavium
 CentOS
 ChatGPT
 Cholesky
-cholesky
 CoRR
 Codespaces
 Commitizen
@@ -92,12 +90,8 @@ Concretized
 Conda
 ConnectX
 CountOnes
+Cron
 CuPy
-customizable
-da
-Dashboarding
-Dataloading
-dataflows
 DBRX
 DCQCN
 DDR
@@ -105,27 +99,26 @@ DF
 DGEMM
 DGL
 DGLGraph
-dGPU
-dGPUs
 DIMM
 DKMS
 DL
 DMA
-DOMContentLoaded
 DNN
 DNNL
 DOCA
+DOMContentLoaded
 DPM
 DRI
 DSCP
 DW
 DWORD
+Dashboarding
 Dask
 DataFrame
 DataLoader
 DataParallel
+Dataloading
 Debian
-decompositions
 DeepSeek
 DeepSpeed
 Dependabot
@@ -133,40 +126,32 @@ Deprecations
 DevCap
 DirectX
 Disaggregated
-disagg
-disaggregated
-disaggregation
 Dockerfile
 Dockerized
 Doxygen
-dropless
 ELMo
 ENDPGM
+EP
 EPYC
 ESXi
-EP
 EoS
-etcd
-equalto
-fas
 FBGEMM
-FiLM
-FIFOs
 FFT
 FFTs
 FFmpeg
 FHS
+FIFOs
 FIXME
 FMA
 FP
 FX
+FiLM
 Filesystem
 FindDb
 Flang
 FlashAttention
-FlashInfer’s
 FlashInfer
-flashinfer
+FlashInfer’s
 FluxBenchmark
 Fortran
 Fuyu
@@ -186,18 +171,13 @@ GDS
 GEMM
 GEMMs
 GFLOPS
-GFortran
 GFXIP
+GFortran
 GGUF
 GID
-Gbps
-Gemma
-GiB
 GIM
 GL
-Glibc
 GLXT
-Gloo
 GMI
 GNN
 GNNs
@@ -209,27 +189,29 @@ GPU's
 GPUDirect
 GPUVM
 GPUs
-GraphBolt
-GraphSage
 GRBM
 GRE
 GTT
+Gbps
+Gemma
 GenAI
 GenZ
+GiB
 GitHub
 Gitpod
-hardcoded
+Glibc
+Gloo
+GraphBolt
+GraphSage
 HBM
 HCA
 HGX
-HLO
 HIPCC
-hipDataType
 HIPExtension
 HIPIFY
 HIPification
-hipification
 HIPify
+HLO
 HPC
 HPCG
 HPE
@@ -240,12 +222,11 @@ HWE
 HWS
 Haswell
 Higgs
-href
-Hyperparameters
-HybridEngine
 Huggingface
 Hunyuan
 HunyuanVideo
+HybridEngine
+Hyperparameters
 IB
 ICD
 ICT
@@ -254,13 +235,10 @@ IDE
 IDEs
 IFWI
 IMDb
-IncDec
-instrSize
-interpolators
 IOMMU
 IOP
-IOPS
 IOPM
+IOPS
 IOV
 IPs
 IRQ
@@ -269,6 +247,7 @@ ISV
 ISVs
 ITL
 ImageNet
+IncDec
 InfiniBand
 Inlines
 IntelliSense
@@ -278,9 +257,8 @@ Intra
 Ioffe
 JAX's
 JAXLIB
-Jinja
-js
 JSON
+Jinja
 Jupyter
 KFD
 KFDTest
@@ -288,10 +266,10 @@ KMD
 KV
 KVM
 Karpathy's
-KiB
-Kineto
 Keras
 Khronos
+KiB
+Kineto
 LAPACK
 LCLK
 LDS
@@ -299,27 +277,26 @@ LLM
 LLMs
 LLVM
 LM
-logsumexp
 LPDDR
 LRU
 LSAN
-LSan
-lstsq
-LTS
 LSTMs
-LteAll
+LSan
+LTS
 LanguageCrossEntropy
 LoRA
+LteAll
+MBT
 MECO
 MEM
 MERCHANTABILITY
 MFMA
-MiB
 MIGraphX
 MIOpen
-MIOpenGEMM
 MIOpen's
+MIOpenGEMM
 MIVisionX
+MLA
 MLM
 MMA
 MMIO
@@ -330,35 +307,31 @@ MPI
 MPT
 MSVC
 MTU
-mul
 MVAPICH
 MVFFR
+MXFP
 Makefile
 Makefiles
 Matplotlib
 Matrox
 MaxText
-MBT
 Megablocks
 Megatrends
 Megatron
 Mellanox
 Mellanox's
 Meta's
+MiB
 Miniconda
 MirroredStrategy
 Mixtral
-MLA
-MosaicML
 MoEs
-Mooncake
 MoRI
+Mooncake
+MosaicML
 Mpops
 Multicore
-multihost
 Multithreaded
-mx
-MXFP
 MyEnvironment
 MyST
 NANOO
@@ -366,24 +339,28 @@ NBIO
 NBIOs
 NCCL
 NCF
+NCS
 NFS
 NIC
 NICs
 NLI
 NLP
 NN
+NOP
 NPKit
 NPS
 NSP
 NUMA
 NVCC
 NVIDIA
+NVLink
 NVPTX
 NaN
+NaNs
 Nano
 Navi
-Noncoherently
 NoReturn
+Noncoherently
 NousResearch's
 NumPy
 OAM
@@ -394,10 +371,10 @@ OFED
 OMM
 OMP
 OMPI
-OOM
 OMPT
 OMPX
 ONNX
+OOM
 OSS
 OSU
 OpenCL
@@ -411,13 +388,10 @@ OpenVX
 OpenXLA
 Optim
 Oversubscription
-PagedAttention
-Pallas
 PCC
 PCI
 PCIe
 PEFT
-perf
 PEQT
 PIL
 PILImage
@@ -430,6 +404,8 @@ PSID
 PTPC
 PaLM
 Pageable
+PagedAttention
+Pallas
 PeerDirect
 Pensando
 PerfDb
@@ -444,13 +420,12 @@ Pretraining
 Primus
 Profiler's
 PyPi
-Pytest
 PyTorch
+Pytest
 QPS
 Qcycles
 QoS
 Qwen
-Radix
 RAII
 RAS
 RCCL
@@ -459,16 +434,16 @@ RDC's
 RDMA
 RDNA
 README
-Recomputation
 RHEL
+RLHF
 RMW
 RNN
 RNNs
 ROC
 ROCProfiler
 ROCT
-ROCTx
 ROCTracer
+ROCTx
 ROCclr
 ROCdbgapi
 ROCgdb
@@ -483,9 +458,12 @@ RPP
 RST
 RW
 Radeon
+Radix
+Recomputation
 Redfish
 RelWithDebInfo
 Req
+ResNet
 Rickle
 RoCE
 Runfile
@@ -493,7 +471,6 @@ Ryzen
 SALU
 SBIOS
 SCA
-ScaledGEMM
 SDK
 SDMA
 SDPA
@@ -511,7 +488,6 @@ SKU
 SKUs
 SLES
 SLURM
-Slurm
 SMEM
 SMFMA
 SMI
@@ -522,18 +498,19 @@ SRAM
 SRAMECC
 SVD
 SWE
+ScaledGEMM
 SerDes
+Shardy
 ShareGPT
 Shlens
-simd
 Skylake
+Slurm
 Softmax
 Spack
 SplitK
 Strix
 Supermicro
 Szegedy
-TagRAM
 TCA
 TCC
 TCCs
@@ -541,35 +518,32 @@ TCI
 TCIU
 TCP
 TCR
-TTM
-TVM
-THREADGROUPS
-threadgroups
-TensorRT
-TensorFloat
 TF
 TFLOPS
+THREADGROUPS
 TP
 TPS
 TPU
 TPUs
 TSME
+TTM
+TVM
+TagRAM
 Tagram
 TensileLite
 TensorBoard
+TensorFloat
 TensorFlow
 TensorParallel
+TensorRT
 TheRock
 ToC
+TopK
 TorchAudio
-torchaudio
 TorchElastic
 TorchMIGraphX
-torchrec
 TorchScript
 TorchServe
-torchserve
-torchtext
 TorchVision
 TransferBench
 TrapStatus
@@ -577,30 +551,32 @@ UAC
 UC
 UCC
 UCX
-ud
 UE
+UI
 UIF
 UMC
 USM
+USM
+UTCL
 UTCL
 UTIL
+UTIL
+UX
 UltraChat
 Uncached
-Unittests
 Unhandled
-unwindowed
+Unittests
 VALU
 VBIOS
 VCN
-verl's
 VGPR
 VGPRs
 VM
+VMEM
 VMID
 VMIDs
-VMs
-VMEM
 VMWare
+VMs
 VRAM
 VSIX
 VSkipped
@@ -614,7 +590,6 @@ WikiText
 Wojna
 Workgroups
 Writebacks
-xcc
 XCD
 XCDs
 XGBoost
@@ -634,8 +609,8 @@ ZeRO
 ZenDNN
 accuracies
 activations
-addr
 addEventListener
+addr
 ade
 ai
 alloc
@@ -646,8 +621,10 @@ amdgpu
 api
 aten
 atmi
+atomicRMW
 atomics
 autogenerated
+autograd
 autotune
 avx
 awk
@@ -662,17 +639,16 @@ bilinear
 bitcode
 bitsandbytes
 bitwise
-Bitwise
 blit
 bootloader
 boson
 bosons
 br
-BrainFloat
 btn
 buildable
 bursty
 bzip
+cTDP
 cacheable
 carveout
 cd
@@ -681,6 +657,7 @@ centric
 changelog
 checkpointing
 chiplet
+cholesky
 classList
 cmake
 cmd
@@ -704,16 +681,20 @@ copyable
 cpp
 csn
 cuBLAS
-cuda
 cuDNN
-cudnn
 cuFFT
 cuLIB
 cuRAND
 cuSOLVER
 cuSPARSE
+cuda
+cudnn
+customizable
 customizations
-cTDP
+dGPU
+dGPUs
+da
+dataflows
 dataset
 datasets
 dataspace
@@ -723,8 +704,9 @@ datatypes
 dbgapi
 de
 deallocation
-debuggability
 debian
+debuggability
+decompositions
 deepseek
 denoise
 denoised
@@ -740,10 +722,14 @@ devicelibs
 devsel
 dgl
 dimensionality
+disagg
+disaggregated
+disaggregation
 disambiguates
 distro
 distros
 dkms
+dropless
 dtype
 eb
 el
@@ -756,13 +742,17 @@ endpgm
 enqueue
 env
 epilog
+equalto
+etcd
 etcetera
 eth
 ethernet
 exascale
 executables
+fas
 ffmpeg
 filesystem
+flashinfer
 forEach
 fortran
 fp
@@ -782,13 +772,15 @@ gnupg
 gpu
 grayscale
 gx
+gz
 gzip
+hardcoded
 heterogenous
 hipBLAS
 hipBLASLt
 hipBLASLt's
-hipblaslt
 hipCUB
+hipDataType
 hipFFT
 hipFORT
 hipLIB
@@ -799,10 +791,12 @@ hipSPARSELt
 hipTensor
 hipamd
 hipblas
+hipblaslt
 hipcc
 hipcub
 hipfft
 hipfort
+hipification
 hipify
 hipsolver
 hipsparse
@@ -811,6 +805,7 @@ hostname
 hotspotting
 hpc
 hpp
+href
 hsa
 hsakmt
 hyperparameter
@@ -826,7 +821,9 @@ init
 initializer
 inlining
 installable
+instrSize
 interop
+interpolators
 interprocedural
 intra
 intrinsics
@@ -834,6 +831,7 @@ invariants
 invocating
 ipo
 jax
+js
 json
 kdb
 kfd
@@ -853,8 +851,10 @@ llvm
 lm
 localscratch
 logits
+logsumexp
 loopback
 lossy
+lstsq
 macOS
 matchers
 maxtext
@@ -871,25 +871,25 @@ mjx
 mkdir
 mlirmiopen
 mtypes
+mul
+multihost
 mutex
 mvffr
+mx
 namespace
 namespaces
 nanoGPT
-NCS
-NOP
-NVLink
 netplan
 num
 numref
 ocl
+ol
 openai
 opencl
 opencv
 openmp
 openssl
 optimizers
-ol
 os
 oversubscription
 pageable
@@ -897,14 +897,16 @@ pallas
 parallelization
 parallelizing
 param
-params
 parameterization
+params
 passthrough
+pb
 pe
+perf
 perfcounter
 performant
-piecewise
 perl
+piecewise
 pragma
 pre
 prebuild
@@ -934,8 +936,6 @@ pseudorandom
 px
 py
 pytorch
-recommender
-recommenders
 quantile
 quantizer
 quasirandom
@@ -948,8 +948,10 @@ rc
 rccl
 rdc
 rdma
-reachability
 reStructuredText
+reachability
+recommender
+recommenders
 redirections
 refactorization
 reformats
@@ -963,7 +965,6 @@ rescaling
 reusability
 rhel
 rl
-RLHF
 roadmap
 roc
 rocAL
@@ -990,8 +991,8 @@ rocm
 rocminfo
 rocprim
 rocprof
-rocprofv
 rocprofiler
+rocprofv
 rocr
 rocrand
 rocsolver
@@ -1001,7 +1002,6 @@ roctracer
 rst
 runtime
 runtimes
-ResNet
 sL
 scalability
 scalable
@@ -1017,6 +1017,7 @@ sglang
 shader
 sharding
 sigmoid
+simd
 sles
 slurm
 sm
@@ -1047,6 +1048,7 @@ td
 tensorfloat
 tf
 th
+threadgroups
 tokenization
 tokenize
 tokenized
@@ -1054,23 +1056,27 @@ tokenizer
 tokenizes
 toolchain
 toolchains
-topk
 toolset
 toolsets
+topk
+torchaudio
+torchrec
+torchserve
+torchtext
 torchtitan
 torchvision
 tp
 tqdm
 tracebacks
 txt
-TopK
 uarch
-uncached
+ud
 uncacheable
+uncached
 uncorrectable
 underoptimized
-unhandled
 unfused
+unhandled
 uninstallation
 unmapped
 unsqueeze
@@ -1078,12 +1084,9 @@ unstacking
 unswitching
 untrusted
 untuned
+unwindowed
 upvote
-USM
-UTCL
-UTIL
 utils
-UX
 vL
 variational
 vdi
@@ -1094,6 +1097,7 @@ vectorized
 vectorizer
 vectorizes
 verl
+verl's
 virtualize
 virtualized
 vjxb
@@ -1101,6 +1105,7 @@ vllm
 voxel
 walkthrough
 walkthroughs
+warmup
 watchpoints
 wavefront
 wavefronts
@@ -1112,11 +1117,13 @@ writeback
 writebacks
 wrreq
 wzo
-xargs
-xdit
 xDiT
 xGMI
 xPacked
+xargs
+xcc
+xdit
+xplane
 xz
 yaml
 ysvmadyb
@@ -136,7 +136,7 @@ For a complete list of ROCm components and version information, see the
 ## Release notes

 - [Latest version of ROCm](https://rocm.docs.amd.com/en/latest/about/release-notes.html) - production
- [ROCm 7.12.0](https://rocm.docs.amd.com/en/7.12.0-preview/about/release-notes.html) – preview stream
+- [ROCm 7.13.0](https://rocm.docs.amd.com/en/7.13.0-preview/about/release-notes.html) – preview stream

 ---

@@ -108,7 +108,7 @@ GPU and baseboard firmware versioning might differ across GPU families.
              01.25.16.03
          </td>
          <td>
-              30.30.x where x (0-2)<br>
+              30.30.x where x (0-3)<br>
              30.20.x where x (0-1)<br>
              30.10.x where x (0-2)
            </td>
@@ -119,7 +119,7 @@ GPU and baseboard firmware versioning might differ across GPU families.
              01.25.06.08<br>
              01.25.04.02
          </td>
-          <td>30.30.x where x (0-2)<br>
+          <td>30.30.x where x (0-3)<br>
              30.20.x where x (0-1)<a href="#footnote1"><sup>[1]</sup></a><br>
              30.10.x where x (0-2)<br>
              6.4.z where z (0-3)<br>
@@ -132,7 +132,7 @@ GPU and baseboard firmware versioning might differ across GPU families.
              01.25.03.12<br>
              01.25.02.04</td>
          <td rowspan="6" style="vertical-align: middle;">
-              30.30.x where x (0-2)<br>
+              30.30.x where x (0-3)<br>
              30.20.x where x (0-1)<br>
              30.10.x where x (0-2)<br>
              6.4.z where z (0–3)<br>
@@ -1,5 +1,5 @@
 ROCm Version,7.2.3,7.2.2/7.2.1,7.2.0,7.1.1,7.1.0,7.0.2,7.0.1/7.0.0,6.4.3,6.4.2,6.4.1,6.4.0,6.3.3,6.3.2,6.3.1,6.3.0,6.2.4,6.2.2,6.2.1,6.2.0, 6.1.5, 6.1.2, 6.1.1, 6.1.0, 6.0.2, 6.0.0
-      :ref:`Operating systems & kernels <OS-kernel-versions>` [#os-compatibility-past-60]_,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,"Ubuntu 24.04.1, 24.04","Ubuntu 24.04.1, 24.04","Ubuntu 24.04.1, 24.04",Ubuntu 24.04,,,,,,
+      :ref:`Operating systems & kernels <OS-kernel-versions>` [#os-compatibility-past-60]_,Ubuntu 24.04.4,Ubuntu 24.04.4,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,"Ubuntu 24.04.1, 24.04","Ubuntu 24.04.1, 24.04","Ubuntu 24.04.1, 24.04",Ubuntu 24.04,,,,,,
      ,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,"Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3, 22.04.2","Ubuntu 22.04.4, 22.04.3, 22.04.2"
      ,,,,,,,,,,,,,,,,,,,,"Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5"
      ,"RHEL 10.1, 10.0, 9.7, 9.6, 9.4","RHEL 10.1, 10.0, 9.7, 9.6, 9.4","RHEL 10.1, 10.0, 9.7, 9.6, 9.4","RHEL 10.1, 10.0, 9.7, 9.6, 9.4","RHEL 10.0, 9.6, 9.4","RHEL 10.0, 9.6, 9.4","RHEL 9.6, 9.4","RHEL 9.6, 9.4","RHEL 9.6, 9.4","RHEL 9.6, 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4, 9.3, 9.2","RHEL 9.4, 9.3, 9.2","RHEL 9.4, 9.3, 9.2","RHEL 9.4, 9.3, 9.2","RHEL 9.3, 9.2","RHEL 9.3, 9.2"
@@ -25,7 +25,7 @@ compatibility and system requirements.
      :header: "ROCm Version", "7.2.3", "7.2.2/7.2.1", "7.1.0"
      :stub-columns: 1

-      :ref:`Operating systems & kernels <OS-kernel-versions>` [#os-compatibility]_,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3
+      :ref:`Operating systems & kernels <OS-kernel-versions>` [#os-compatibility]_,Ubuntu 24.04.4,Ubuntu 24.04.4,Ubuntu 24.04.3
      ,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5
      ,"RHEL 10.1, 10.0, |br| 9.7, 9.6, 9.4","RHEL 10.1, 10.0, |br| 9.7, 9.6, 9.4","RHEL 10.0, 9.6, 9.4"
      ,RHEL 8.10,RHEL 8.10,RHEL 8.10
@@ -152,8 +152,8 @@ compatibility and system requirements.

 .. rubric:: Footnotes

-.. [#os-compatibility] Some operating systems are supported on specific GPUs. For detailed information about operating systems supported on ROCm 7.2.3, see the latest :ref:`supported_distributions`. For version specific information, see `ROCm 7.2.2 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.2/reference/system-requirements.html#supported-operating-systems>`__, and `ROCm 7.1.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.1.0/reference/system-requirements.html#supported-operating-systems>`__.
-.. [#gpu-compatibility] Some GPUs have limited operating system support. For detailed information about GPUs supporting ROCm 7.2.3, see the latest :ref:`supported_GPUs`. For version specific information, see `ROCm 7.2.2 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.2/reference/system-requirements.html#supported-gpus>`__, and `ROCm 7.1.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.1.0/reference/system-requirements.html#supported-gpus>`__.
+.. [#os-compatibility] Some operating systems are supported on specific GPUs. For detailed information about operating systems supported on ROCm 7.2.3, see the latest :ref:`supported_distributions`. For version specific information, see `ROCm 7.2.2/7.2.1 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.2/reference/system-requirements.html#supported-operating-systems>`__, and `ROCm 7.1.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.1.0/reference/system-requirements.html#supported-operating-systems>`__.
+.. [#gpu-compatibility] Some GPUs have limited operating system support. For detailed information about GPUs supporting ROCm 7.2.3, see the latest :ref:`supported_GPUs`. For version specific information, see `ROCm 7.2.2/7.2.1 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.2/reference/system-requirements.html#supported-gpus>`__, and `ROCm 7.1.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.1.0/reference/system-requirements.html#supported-gpus>`__.
 .. [#mi325x_KVM] For AMD Instinct MI325X KVM SR-IOV users, do not use AMD GPU Driver (amdgpu) 30.20.0.
 .. [#driver_patch] AMD GPU Driver (amdgpu) 30.10.1 is a quality release that resolves an issue identified in the 30.10 release. There are no other significant changes or feature additions in ROCm 7.0.1 from ROCm 7.0.0. AMD GPU Driver (amdgpu) 30.10.1 is compatible with ROCm 7.0.1 and ROCm 7.0.0.
 .. [#kfd_support] As of ROCm 6.4.0, forward and backward compatibility between the AMD GPU Driver (amdgpu) and its user space software is provided up to a year apart. For earlier ROCm releases, the compatibility is provided for +/- 2 releases. The supported user space versions on this page were accurate as of the time of initial ROCm release. For the most up-to-date information, see the latest version of this information at `User and AMD GPU Driver support matrix <https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/user-kernel-space-compat-matrix.html>`_.
@@ -164,7 +164,7 @@ compatibility and system requirements.
 Operating systems, kernel and Glibc versions
 *********************************************

-For detailed information on operating system supported on ROCm 7.2.2/7.2.1 and associated Kernel and Glibc version, see the latest :ref:`supported_distributions`. For version specific information, see `ROCm 7.2.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.0/reference/system-requirements.html#supported-operating-systems>`__, and `ROCm 6.4.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-6.4.0/reference/system-requirements.html#supported-operating-systems>`__.
+For detailed information on operating system supported on ROCm 7.2.3 and associated Kernel and Glibc version, see the latest :ref:`supported_distributions`. For version specific information, see `ROCm 7.2.2/7.2.1 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.2/reference/system-requirements.html#supported-operating-systems>`__, and `ROCm 7.1.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.1.0/reference/system-requirements.html#supported-operating-systems>`__.

 .. note::

@@ -0,0 +1,361 @@
+docker:
+  pull_tag: rocm/pytorch-xdit:v26.4
+  docker_hub_url: https://hub.docker.com/layers/rocm/pytorch-xdit/v26.4/images/sha256-b4296a638eb8dc7ebcafc808e180b78a3c44177580c21986082ec9539496067c
+  ROCm: 7.12.0
+  whats_new:
+    - "Qwen-Image-2512 support"
+    - "Z-Image support"
+    - "Parallel VAE decode support for Wan models"
+    - "Batch inference and data parallel support"
+  components:
+    TheRock: 
+      version: 9b611c6
+      url: https://github.com/ROCm/TheRock
+    rocm-libraries:
+      version: 7567d83
+      url: https://github.com/ROCm/rocm-libraries
+    rocm-systems:
+      version: 93bc019
+      url: https://github.com/ROCm/rocm-systems
+    torch:
+      version: ff65f5b
+      url: https://github.com/ROCm/pytorch
+    torchaudio:
+      version: e3c6ee2
+      url: https://github.com/pytorch/audio
+    torchvision:
+      version: b919bd0
+      url: https://github.com/pytorch/vision
+    triton:
+      version: a272dfa
+      url: https://github.com/ROCm/triton
+    accelerate:
+      version: 46ba481
+      url: https://github.com/huggingface/accelerate
+    aiter:
+      version: a169e14
+      url: https://github.com/ROCm/aiter
+    diffusers:
+      version: a80b192
+      url: https://github.com/huggingface/diffusers
+    distvae:
+      version: bf7531e
+      url: https://github.com/xdit-project/DistVAE
+    xfuser:
+      version: 45c44e7
+      url: https://github.com/xdit-project/xDiT
+    yunchang:
+      version: 631bdfd
+      url: https://github.com/feifeibear/long-context-attention
+  supported_models:
+    - group: Hunyuan Video
+      js_tag: hunyuan
+      models:
+        - model: Hunyuan Video
+          model_repo: tencent/HunyuanVideo
+          revision: refs/pr/18
+          url: https://huggingface.co/tencent/HunyuanVideo
+          github: https://github.com/Tencent-Hunyuan/HunyuanVideo
+          mad_tag: pyt_xdit_hunyuanvideo
+          js_tag: hunyuan_tag
+          benchmark_command:
+            - mkdir results
+            - 'xdit \'
+            - '--model {model_repo} \'
+            - '--prompt "In the large cage, two puppies were wagging their tails at each other." \'
+            - '--batch_size 1 \'
+            - '--height 720 --width 1280 \'
+            - '--seed 1168860793 \'
+            - '--num_frames 129 \'
+            - '--num_inference_steps 50 \'
+            - '--warmup_calls 1 \'
+            - '--num_iterations 1 \'
+            - '--ulysses_degree 8 \'
+            - '--enable_tiling --enable_slicing \'
+            - '--guidance_scale 6.0 \'
+            - '--use_torch_compile \'
+            - '--attention_backend aiter \'
+            - '--output_directory results'
+        - model: Hunyuan Video 1.5
+          model_repo: hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-720p_t2v
+          url: https://huggingface.co/hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-720p_t2v
+          github: https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5
+          mad_tag: pyt_xdit_hunyuanvideo_1_5
+          js_tag: hunyuan_1_5_tag
+          benchmark_command:
+            - mkdir results
+            - 'xdit \'
+            - '--model {model_repo} \'
+            - '--prompt "In the large cage, two puppies were wagging their tails at each other." \'
+            - '--task t2v \'
+            - '--height 720 --width 1280 \'
+            - '--seed 1168860793 \'
+            - '--num_frames 129 \'
+            - '--num_inference_steps 50 \'
+            - '--num_iterations 1 \'
+            - '--ulysses_degree 8 \'
+            - '--enable_tiling --enable_slicing \'
+            - '--use_torch_compile \'
+            - '--attention_backend aiter \'
+            - '--output_directory results'
+    - group: Wan-AI
+      js_tag: wan
+      models:
+        - model: Wan2.1
+          model_repo: Wan-AI/Wan2.1-I2V-14B-720P-Diffusers
+          url: https://huggingface.co/Wan-AI/Wan2.1-I2V-14B-720P-Diffusers
+          github: https://github.com/Wan-Video/Wan2.1
+          mad_tag: pyt_xdit_wan_2_1
+          js_tag: wan_21_tag
+          benchmark_command:
+            - mkdir results
+            - 'xdit \'
+            - '--model {model_repo} \'
+            - '--prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline''s intricate details and the refreshing atmosphere of the seaside." \'
+            - '--height 720 \'
+            - '--width 1280 \'
+            - '--input_images /app/data/wan_input.jpg \'
+            - '--num_frames 81 \'
+            - '--ulysses_degree 8 \'
+            - '--use_parallel_vae \'
+            - '--seed 42 \'
+            - '--num_iterations 1 \'
+            - '--num_inference_steps 40 \'
+            - '--use_torch_compile \'
+            - '--attention_backend aiter \'
+            - '--output_directory results'
+        - model: Wan2.2
+          model_repo: Wan-AI/Wan2.2-I2V-A14B-Diffusers
+          url: https://huggingface.co/Wan-AI/Wan2.2-I2V-A14B-Diffusers
+          github: https://github.com/Wan-Video/Wan2.2
+          mad_tag: pyt_xdit_wan_2_2
+          js_tag: wan_22_tag
+          benchmark_command:
+            - mkdir results
+            - 'xdit \'
+            - '--model {model_repo} \'
+            - '--prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline''s intricate details and the refreshing atmosphere of the seaside." \'
+            - '--height 720 \'
+            - '--width 1280 \'
+            - '--input_images /app/data/wan_input.jpg \'
+            - '--num_frames 81 \'
+            - '--ulysses_degree 8 \'
+            - '--use_parallel_vae \'
+            - '--seed 42 \'
+            - '--num_iterations 1 \'
+            - '--num_inference_steps 40 \'
+            - '--use_torch_compile \'
+            - '--attention_backend aiter \'
+            - '--output_directory results'
+    - group: FLUX
+      js_tag: flux
+      models:
+        - model: FLUX.1
+          model_repo: black-forest-labs/FLUX.1-dev
+          url: https://huggingface.co/black-forest-labs/FLUX.1-dev
+          github: https://github.com/black-forest-labs/flux
+          mad_tag: pyt_xdit_flux
+          js_tag: flux_1_tag
+          benchmark_command:
+            - mkdir results
+            - 'xdit \'
+            - '--model {model_repo} \'
+            - '--seed 42 \'
+            - '--prompt "A small cat" \'
+            - '--height 1024 \'
+            - '--width 1024 \'
+            - '--num_inference_steps 25 \'
+            - '--max_sequence_length 256 \'
+            - '--warmup_calls 5 \'
+            - '--ulysses_degree 8 \'
+            - '--use_torch_compile \'
+            - '--guidance_scale 0.0 \'
+            - '--num_iterations 50 \'
+            - '--attention_backend aiter \'
+            - '--output_directory results'
+        - model: FLUX.1 Kontext
+          model_repo: black-forest-labs/FLUX.1-Kontext-dev
+          url: https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev
+          github: https://github.com/black-forest-labs/flux
+          mad_tag: pyt_xdit_flux_kontext
+          js_tag: flux_1_kontext_tag
+          benchmark_command:
+            - mkdir results
+            - 'xdit \'
+            - '--model {model_repo} \'
+            - '--seed 42 \'
+            - '--prompt "Add a cool hat to the cat" \'
+            - '--height 1024 \'
+            - '--width 1024 \'
+            - '--num_inference_steps 30 \'
+            - '--max_sequence_length 512 \'
+            - '--warmup_calls 5 \'
+            - '--ulysses_degree 8 \'
+            - '--use_torch_compile \'
+            - '--input_images /app/data/flux_cat.png \'
+            - '--guidance_scale 2.5 \'
+            - '--num_iterations 25 \'
+            - '--attention_backend aiter \'
+            - '--output_directory results'
+        - model: FLUX.2
+          model_repo: black-forest-labs/FLUX.2-dev
+          url: https://huggingface.co/black-forest-labs/FLUX.2-dev
+          github: https://github.com/black-forest-labs/flux2
+          mad_tag: pyt_xdit_flux_2
+          js_tag: flux_2_tag
+          benchmark_command:
+            - mkdir results
+            - 'xdit \'
+            - '--model {model_repo} \'
+            - '--seed 42 \'
+            - '--prompt "Add a cool hat to the cat" \'
+            - '--height 1024 \'
+            - '--width 1024 \'
+            - '--num_inference_steps 50 \'
+            - '--max_sequence_length 512 \'
+            - '--warmup_calls 5 \'
+            - '--ulysses_degree 8 \'
+            - '--use_torch_compile \'
+            - '--input_images /app/data/flux_cat.png \'
+            - '--guidance_scale 4.0 \'
+            - '--num_iterations 25 \'
+            - '--attention_backend aiter \'
+            - '--output_directory results'
+        - model: FLUX.2 Klein
+          model_repo: black-forest-labs/FLUX.2-klein-9B
+          url: https://huggingface.co/black-forest-labs/FLUX.2-klein-9B
+          github: https://github.com/black-forest-labs/flux2
+          mad_tag: pyt_xdit_flux_2_klein
+          js_tag: flux_2_klein_tag
+          benchmark_command:
+            - mkdir results
+            - 'xdit \'
+            - '--model {model_repo} \'
+            - '--seed 42 \'
+            - '--prompt "A spectacular sunset over the ocean" \'
+            - '--height 2048 \'
+            - '--width 2048 \'
+            - '--num_inference_steps 4 \'
+            - '--warmup_calls 5 \'
+            - '--ulysses_degree 8 \'
+            - '--use_torch_compile \'
+            - '--guidance_scale 1.0 \'
+            - '--num_iterations 25 \'
+            - '--attention_backend aiter \'
+            - '--output_directory results'
+    - group: StableDiffusion
+      js_tag: stablediffusion
+      models:
+        - model: stable-diffusion-3.5-large
+          model_repo: stabilityai/stable-diffusion-3.5-large
+          url: https://huggingface.co/stabilityai/stable-diffusion-3.5-large
+          github: https://github.com/Stability-AI/sd3.5
+          mad_tag: pyt_xdit_sd_3_5
+          js_tag: stable_diffusion_3_5_large_tag
+          benchmark_command:
+            - mkdir results
+            - 'xdit \'
+            - '--model {model_repo} \'
+            - '--prompt "A capybara holding a sign that reads Hello World" \'
+            - '--num_iterations 50 \'
+            - '--num_inference_steps 28 \'
+            - '--pipefusion_parallel_degree 4 \'
+            - '--use_cfg_parallel \'
+            - '--use_torch_compile \'
+            - '--attention_backend aiter \'
+            - '--output_directory results'
+    - group: Z-Image
+      js_tag: z_image
+      models:
+        - model: Z-Image
+          model_repo: Tongyi-MAI/Z-Image
+          url: https://huggingface.co/Tongyi-MAI/Z-Image
+          github: https://github.com/Tongyi-MAI/Z-Image
+          mad_tag: pyt_xdit_z_image
+          js_tag: z_image_tag
+          benchmark_command:
+            - mkdir results
+            - 'xdit \'
+            - '--model {model_repo} \'
+            - '--seed 42 \'
+            - '--prompt "A crowded beach" \'
+            - '--height 1088 \'
+            - '--width 1920 \'
+            - '--num_inference_steps 50 \'
+            - '--ulysses_degree 2 \'
+            - '--ring_degree 2 \'
+            - '--use_cfg_parallel \'
+            - '--use_torch_compile \'
+            - '--guidance_scale 4.0 \'
+            - '--num_iterations 25 \'
+            - '--attention_backend aiter \'
+            - '--output_directory results'
+    - group: LTX
+      js_tag: ltx
+      models:
+        - model: LTX-2
+          model_repo: Lightricks/LTX-2
+          url: https://huggingface.co/Lightricks/LTX-2
+          github: https://github.com/Lightricks/LTX-2
+          mad_tag: pyt_xdit_ltx2
+          js_tag: ltx2_tag
+          benchmark_command:
+            - mkdir results
+            - 'xdit \'
+            - '--model {model_repo} \'
+            - '--seed 42 \'
+            - '--prompt "Cinematic action packed shot. The man says silently: \"We need to run.\". The camera zooms in on his mouth then immediately screams: \"NOW!\". The camera zooms back out, he turns around and bolts it." \'
+            - '--height 1088 \'
+            - '--width 1920 \'
+            - '--num_inference_steps 40 \'
+            - '--ulysses_degree 8 \'
+            - '--use_torch_compile \'
+            - '--guidance_scale 4.0 \'
+            - '--num_iterations 1 \'
+            - '--attention_backend aiter \'
+            - '--output_directory results'
+    - group: Qwen-Image
+      js_tag: qwen_image
+      models:
+        - model: Qwen-Image
+          model_repo: Qwen/Qwen-Image-2512
+          url: https://huggingface.co/Qwen/Qwen-Image-2512
+          github: https://github.com/QwenLM/Qwen-Image
+          mad_tag: pyt_xdit_qwen_image
+          js_tag: qwen_image_tag
+          benchmark_command:
+            - mkdir results
+            - 'xdit \'
+            - '--model {model_repo} \'
+            - '--seed 42 \'
+            - '--prompt "A cat holding a sign that says hello world" \'
+            - '--height 2048 \'
+            - '--width 2048 \'
+            - '--num_inference_steps 50 \'
+            - '--ulysses_degree 8 \'
+            - '--use_torch_compile \'
+            - '--num_iterations 1 \'
+            - '--attention_backend aiter \'
+            - '--output_directory results'
+        - model: Qwen-Image-Edit
+          model_repo: Qwen/Qwen-Image-Edit
+          url: https://huggingface.co/Qwen/Qwen-Image-Edit
+          github: https://github.com/QwenLM/Qwen-Image
+          mad_tag: pyt_xdit_qwen_image_edit
+          js_tag: qwen_image_edit_tag
+          benchmark_command:
+            - mkdir results
+            - 'xdit \'
+            - '--model {model_repo} \'
+            - '--seed 42 \'
+            - '--prompt "Add a cool hat to the cat." \'
+            - '--negative_prompt "" \'
+            - '--input_images /app/data/flux_cat.png \'
+            - '--height 2048 \'
+            - '--width 2048 \'
+            - '--num_inference_steps 50 \'
+            - '--ulysses_degree 8 \'
+            - '--use_torch_compile \'
+            - '--num_iterations 1 \'
+            - '--attention_backend aiter \'
+            - '--output_directory results'
@@ -1,21 +1,20 @@
 docker:
-  pull_tag: rocm/pytorch-xdit:v26.4
-  docker_hub_url: https://hub.docker.com/layers/rocm/pytorch-xdit/v26.4/images/sha256-b4296a638eb8dc7ebcafc808e180b78a3c44177580c21986082ec9539496067c
-  ROCm: 7.12.0
+  pull_tag: rocm/pytorch-xdit:v26.5
+  docker_hub_url: https://hub.docker.com/layers/rocm/pytorch-xdit/v26.5/images/sha256-b8ad9fd4b41bc116ac2aff07c1066bf369cf7fc110b1a323f6302191985a51fd
+  ROCm: 7.13.0
  whats_new:
-    - "Qwen-Image-2512 support"
-    - "Z-Image support"
-    - "Parallel VAE decode support for Wan models"
-    - "Batch inference and data parallel support"
+    - "Hunyuan Video 1.5 sparse attention (SSTA) support"
+    - "Support fp8 MLA for MI355"
+    - "Block wise sparsity support for AMD triton FAv3 Sage attention"
  components:
    TheRock: 
-      version: 9b611c6
+      version: cbff3d1
      url: https://github.com/ROCm/TheRock
    rocm-libraries:
-      version: 7567d83
+      version: a668483b
      url: https://github.com/ROCm/rocm-libraries
    rocm-systems:
-      version: 93bc019
+      version: c76140fa
      url: https://github.com/ROCm/rocm-systems
    torch:
      version: ff65f5b
@@ -33,16 +32,16 @@ docker:
      version: 46ba481
      url: https://github.com/huggingface/accelerate
    aiter:
-      version: a169e14
+      version: bc5ea32c
      url: https://github.com/ROCm/aiter
    diffusers:
-      version: a80b192
+      version: 447e571a
      url: https://github.com/huggingface/diffusers
    distvae:
-      version: bf7531e
+      version: 5a0fcbb
      url: https://github.com/xdit-project/DistVAE
    xfuser:
-      version: 45c44e7
+      version: 051db68f
      url: https://github.com/xdit-project/xDiT
    yunchang:
      version: 631bdfd
@@ -119,6 +118,7 @@ docker:
            - '--ulysses_degree 8 \'
            - '--use_parallel_vae \'
            - '--seed 42 \'
+            - '--guidance_scale 3.0 \'
            - '--num_iterations 1 \'
            - '--num_inference_steps 40 \'
            - '--use_torch_compile \'
@@ -334,7 +334,8 @@ docker:
            - '--num_inference_steps 50 \'
            - '--ulysses_degree 8 \'
            - '--use_torch_compile \'
-            - '--num_iterations 1 \'
+            - '--guidance_scale 0.0 \'
+            - '--num_iterations 25 \'
            - '--attention_backend aiter \'
            - '--output_directory results'
        - model: Qwen-Image-Edit
@@ -356,6 +357,7 @@ docker:
            - '--num_inference_steps 50 \'
            - '--ulysses_degree 8 \'
            - '--use_torch_compile \'
-            - '--num_iterations 1 \'
+            - '--guidance_scale 4.0 \'
+            - '--num_iterations 25 \'
            - '--attention_backend aiter \'
            - '--output_directory results'
@@ -1,12 +1,12 @@
 dockers:
-  - pull_tag: rocm/jax-training:maxtext-v26.2
-    docker_hub_url: https://hub.docker.com/layers/rocm/jax-training/maxtext-v26.2/images/sha256-a89643388487b1e2fc6b6ef7bd3c44378c05d217309c977a1c18c72d05ebcaeb
+  - pull_tag: rocm/jax-training:maxtext-v26.3
+    docker_hub_url: https://hub.docker.com/layers/rocm/jax-training/maxtext-v26.3/images/sha256-548381e496fa1367772964341130cc014ee1a9a5ca715ff4ca3aa92960d162c3
    components:
-      ROCm: 7.1.1
+      ROCm: 7.2.1
      JAX: 0.8.2
      Python: 3.12
-      Transformer Engine: 2.8.0.dev0+aec00a7f
-      hipBLASLt: 1.2.x
+      Transformer Engine: 2.8.0.dev0+9b312832
+      hipBLASLt: 1.3.0+bfcf25fa18
 model_groups:
  - group: Meta Llama
    tag: llama
@@ -93,3 +93,24 @@ model_groups:
          gfx950: env_scripts/gfx950_mixtral_8x7b.yml
          gfx942: env_scripts/llama3_8x7b.yml
        doc_options: ["single-node", "multi-node"]
+  - group: Qwen
+    tag: qwen
+    models:
+      - model: Qwen 14B
+        mad_tag: jax_maxtext_train_qwen3-14b
+        model_repo: Qwen3-14B
+        precision: bf16
+        primus_config_name: qwen3_14B-pretrain.yaml
+        multinode_config:
+          gfx950: env_scripts/gfx950_qwen3_14b.yml
+          gfx942: env_scripts/qwen3_14b.yml
+        doc_options: ["single-node", "multi-node"]
+      - model: Qwen 30B A3B
+        mad_tag: jax_maxtext_train_qwen3-30b-a3b
+        model_repo: Qwen3-30B-A3B
+        precision: bf16
+        primus_config_name: qwen3_30B_a3b-pretrain.yaml
+        multinode_config:
+          gfx950: env_scripts/gfx950_qwen3_30b_a3b.yml
+          gfx942: env_scripts/qwen3_30b_a3b.yml
+        doc_options: ["single-node", "multi-node"]
@@ -0,0 +1,95 @@
+dockers:
+  - pull_tag: rocm/jax-training:maxtext-v26.2
+    docker_hub_url: https://hub.docker.com/layers/rocm/jax-training/maxtext-v26.2/images/sha256-a89643388487b1e2fc6b6ef7bd3c44378c05d217309c977a1c18c72d05ebcaeb
+    components:
+      ROCm: 7.1.1
+      JAX: 0.8.2
+      Python: 3.12
+      Transformer Engine: 2.8.0.dev0+aec00a7f
+      hipBLASLt: 1.2.x
+model_groups:
+  - group: Meta Llama
+    tag: llama
+    models:
+      - model: Llama 2 7B
+        mad_tag: jax_maxtext_train_llama-2-7b
+        model_repo: Llama-2-7B
+        precision: bf16
+        primus_config_name: llama2_7B-pretrain.yaml
+        multinode_config:
+          gfx950: env_scripts/gfx950_llama2_7b.yml
+          gfx942: env_scripts/llama2_7b.yml
+        doc_options: ["single-node", "multi-node"]
+      - model: Llama 2 70B
+        mad_tag: jax_maxtext_train_llama-2-70b
+        model_repo: Llama-2-70B
+        precision: bf16
+        primus_config_name: llama2_70B-pretrain.yaml
+        multinode_config:
+          gfx950: env_scripts/gfx950_llama2_70b.yml
+          gfx942: env_scripts/llama2_70b.yml
+        doc_options: ["single-node", "multi-node"]
+      - model: Llama 3 8B
+        mad_tag: jax_maxtext_train_llama-3-8b
+        primus_config_name: llama3_8B-pretrain.yaml
+        multinode_config:
+          gfx950: env_scripts/gfx950_llama3_8b.yml
+          gfx942: env_scripts/llama3_8b.yml
+        doc_options: ["multi-node"]
+      - model: Llama 3 70B
+        mad_tag: jax_maxtext_train_llama-3-70b
+        primus_config_name: llama3_70B-pretrain.yaml
+        multinode_config:
+          gfx950: env_scripts/gfx950_llama3_70b.yml
+          gfx942: env_scripts/llama3_70b.yml
+        doc_options: ["multi-node"]
+      - model: Llama 3.1 8B
+        mad_tag: jax_maxtext_train_llama-3.1-8b
+        model_repo: Llama-3.1-8B
+        precision: bf16
+        doc_options: ["single-node"]
+      - model: Llama 3.1 70B
+        mad_tag: jax_maxtext_train_llama-3.1-70b
+        model_repo: Llama-3.1-70B
+        precision: bf16
+        doc_options: ["single-node"]
+      - model: Llama 3.1 405B (multi-node)
+        mad_tag: jax_maxtext_train_llama-3.1-405b
+        model_repo: Llama-3.1-405B
+        precision: bf16
+        multinode_config:
+          gfx950: env_scripts/gfx950_llama3_405b.yml
+        doc_options: ["multi-node"]
+      - model: Llama 3.3 70B
+        mad_tag: jax_maxtext_train_llama-3.3-70b
+        model_repo: Llama-3.3-70B
+        precision: bf16
+        primus_config_name: llama3.3_70B-pretrain.yaml
+        multinode_config:
+          gfx950: env_scripts/gfx950_llama3.3_70b.yml
+          gfx942: env_scripts/llama3.3_70b.yml
+        doc_options: ["single-node", "multi-node"]
+  - group: DeepSeek
+    tag: deepseek
+    models:
+      - model: DeepSeek-V2-Lite (16B)
+        mad_tag: jax_maxtext_train_deepseek-v2-lite-16b
+        model_repo: DeepSeek-V2-lite
+        precision: bf16
+        primus_config_name: deepseek_v2_16B-pretrain.yaml
+        multinode_config:
+          gfx950: env_scripts/gfx950_deepseek2_16b.yml
+          gfx942: env_scripts/deepseek2_16b.yml
+        doc_options: ["single-node", "multi-node"]
+  - group: Mistral AI
+    tag: mistral
+    models:
+      - model: Mixtral 8x7B
+        mad_tag: jax_maxtext_train_mixtral-8x7b
+        model_repo: Mixtral-8x7B
+        precision: bf16
+        primus_config_name: mixtral_8x7B-pretrain.yaml
+        multinode_config:
+          gfx950: env_scripts/gfx950_mixtral_8x7b.yml
+          gfx942: env_scripts/llama3_8x7b.yml
+        doc_options: ["single-node", "multi-node"]
@@ -0,0 +1,75 @@
+docker:
+  pull_tag: rocm/primus:v26.2
+  docker_hub_url: https://hub.docker.com/layers/rocm/primus/v26.2/images/sha256-9148d1bfcd579bf92f44bd89090e0d8c958f149c134b4b34b9674ab559244585
+  components:
+    ROCm: 7.2.0
+    PyTorch: 2.10.0a0+git449b176
+    Python: "3.12.3"
+    Transformer Engine: 2.8.0.dev0+51f74fa7
+    Flash Attention: 2.8.3
+    hipBLASLt: 1.2.0-de5c1aebb6
+    Triton: 3.6.0
+    RCCL: 2.27.7
+model_groups:
+  - group: Meta Llama
+    tag: llama
+    models:
+      - model: Llama 3.3 70B
+        mad_tag: primus_pyt_megatron_lm_train_llama-3.3-70b
+        config_name: llama3.3_70B-pretrain.yaml
+      - model: Llama 3.1 8B
+        mad_tag: primus_pyt_megatron_lm_train_llama-3.1-8b
+        config_name: llama3.1_8B-pretrain.yaml
+      - model: Llama 3.1 70B
+        mad_tag: primus_pyt_megatron_lm_train_llama-3.1-70b
+        config_name: llama3.1_70B-pretrain.yaml
+      - model: Llama 2 7B
+        mad_tag: primus_pyt_megatron_lm_train_llama-2-7b
+        config_name: llama2_7B-pretrain.yaml
+      - model: Llama 2 70B
+        mad_tag: primus_pyt_megatron_lm_train_llama-2-70b
+        config_name: llama2_70B-pretrain.yaml
+  - group: AMD Zebra-Llama
+    tag: zebra-llama
+    models:
+      - model: Zebra-Llama 1B
+        mad_tag: primus_pyt_megatron_lm_train_zebra-llama-1b
+        config_name: zebra_llama_1b-pretrain.yaml
+      - model: Zebra-Llama 3B
+        mad_tag: primus_pyt_megatron_lm_train_zebra-llama-3b
+        config_name: zebra_llama_3b-pretrain.yaml
+      - model: Zebra-Llama 8B
+        mad_tag: primus_pyt_megatron_lm_train_zebra-llama-8b
+        config_name: zebra_llama_8b-pretrain.yaml
+  - group: DeepSeek
+    tag: deepseek
+    models:
+      - model: DeepSeek-V3 (proxy)
+        mad_tag: primus_pyt_megatron_lm_train_deepseek-v3-proxy
+        config_name: deepseek_v3-pretrain.yaml
+      - model: DeepSeek-V2-Lite
+        mad_tag: primus_pyt_megatron_lm_train_deepseek-v2-lite-16b
+        config_name: deepseek_v2_lite-pretrain.yaml
+  - group: Mistral AI
+    tag: mistral
+    models:
+      - model: Mixtral 8x7B
+        mad_tag: primus_pyt_megatron_lm_train_mixtral-8x7b
+        config_name: mixtral_8x7B_v0.1-pretrain.yaml
+      - model: Mixtral 8x22B (proxy)
+        mad_tag: primus_pyt_megatron_lm_train_mixtral-8x22b-proxy
+        config_name: mixtral_8x22B_v0.1-pretrain.yaml
+  - group: Qwen
+    tag: qwen
+    models:
+      - model: Qwen 3 32B SFT
+        mad_tag: primus_pyt_megatron_lm_train_qwen3-32b-sft
+      - model: Qwen 3 32B LoRA
+        mad_tag: primus_pyt_megatron_lm_train_qwen3-32b-lora
+        config_name: primus_qwen2.5_7B-pretrain.yaml
+      - model: Qwen 2.5 7B
+        mad_tag: primus_pyt_megatron_lm_train_qwen2.5-7b
+        config_name: primus_qwen2.5_7B-pretrain.yaml
+      - model: Qwen 2.5 72B
+        mad_tag: primus_pyt_megatron_lm_train_qwen2.5-72b
+        config_name: qwen2.5_72B-pretrain.yaml
@@ -0,0 +1,34 @@
+docker:
+  pull_tag: rocm/primus:v26.2
+  docker_hub_url: https://hub.docker.com/layers/rocm/primus/v26.2/images/sha256-9148d1bfcd579bf92f44bd89090e0d8c958f149c134b4b34b9674ab559244585
+  components:
+    ROCm: 7.2.0
+    PyTorch: 2.10.0a0+git449b176
+    Python: "3.12.3"
+    Transformer Engine: 2.8.0.dev0+51f74fa7
+    Flash Attention: 2.8.3
+    hipBLASLt: 1.2.0-de5c1aebb6
+    Triton: 3.6.0
+    RCCL: 2.27.7
+model_groups:
+  - group: Meta Llama
+    tag: llama
+    models:
+      - model: Llama 3.1 8B
+        mad_tag: primus_pyt_train_llama-3.1-8b
+        model_repo: Llama-3.1-8B
+        url: https://huggingface.co/meta-llama/Llama-3.1-8B
+        precision: BF16
+      - model: Llama 3.1 70B
+        mad_tag: primus_pyt_train_llama-3.1-70b
+        model_repo: Llama-3.1-70B
+        url: https://huggingface.co/meta-llama/Llama-3.1-70B
+        precision: BF16
+  - group: DeepSeek
+    tag: deepseek
+    models:
+      - model: DeepSeek V3 16B
+        mad_tag: primus_pyt_train_deepseek-v3-16b
+        model_repo: DeepSeek-V3
+        url: https://huggingface.co/deepseek-ai/DeepSeek-V3
+        precision: BF16
@@ -1,13 +1,13 @@
 docker:
-  pull_tag: rocm/primus:v26.2
-  docker_hub_url: https://hub.docker.com/layers/rocm/primus/v26.2/images/sha256-9148d1bfcd579bf92f44bd89090e0d8c958f149c134b4b34b9674ab559244585
+  pull_tag: rocm/primus:v26.3
+  docker_hub_url: https://hub.docker.com/layers/rocm/primus/v26.3/images/sha256-da50bfe9dc4bb70ad683d3ee4a176dccb66f596f48d30e8e52323b41892759b1
  components:
-    ROCm: 7.2.0
-    PyTorch: 2.10.0a0+git449b176
+    ROCm: 7.2.1
+    PyTorch: 2.10.0+git94c6e04
    Python: "3.12.3"
-    Transformer Engine: 2.8.0.dev0+51f74fa7
+    Transformer Engine: 2.12.0.dev0+40434cf6
    Flash Attention: 2.8.3
-    hipBLASLt: 1.2.0-de5c1aebb6
+    hipBLASLt: 1.3.0-c4b2dc9869
    Triton: 3.6.0
    RCCL: 2.27.7
 model_groups:
@@ -41,12 +41,15 @@ model_groups:
      - model: Zebra-Llama 8B
        mad_tag: primus_pyt_megatron_lm_train_zebra-llama-8b
        config_name: zebra_llama_8b-pretrain.yaml
+  - group: OpenAI GPT-OSS
+    tag: gpt-oss
+    models:
+      - model: GPT-OSS-20B
+        mad_tag: primus_pyt_megatron_lm_train_gpt-oss-20b
+        config_name: gpt_oss_20B-BF16-pretrain.yaml
  - group: DeepSeek
    tag: deepseek
    models:
-      - model: DeepSeek-V3 (proxy)
-        mad_tag: primus_pyt_megatron_lm_train_deepseek-v3-proxy
-        config_name: deepseek_v3-pretrain.yaml
      - model: DeepSeek-V2-Lite
        mad_tag: primus_pyt_megatron_lm_train_deepseek-v2-lite-16b
        config_name: deepseek_v2_lite-pretrain.yaml
@@ -56,9 +59,6 @@ model_groups:
      - model: Mixtral 8x7B
        mad_tag: primus_pyt_megatron_lm_train_mixtral-8x7b
        config_name: mixtral_8x7B_v0.1-pretrain.yaml
-      - model: Mixtral 8x22B (proxy)
-        mad_tag: primus_pyt_megatron_lm_train_mixtral-8x22b-proxy
-        config_name: mixtral_8x22B_v0.1-pretrain.yaml
  - group: Qwen
    tag: qwen
    models:
@@ -67,6 +67,9 @@ model_groups:
      - model: Qwen 3 32B LoRA
        mad_tag: primus_pyt_megatron_lm_train_qwen3-32b-lora
        config_name: primus_qwen2.5_7B-pretrain.yaml
+      - model: Qwen 3 30B A3B
+        mad_tag: primus_pyt_megatron_lm_train_qwen3-30b-a3b
+        config_name: qwen3_30B_A3B-pretrain.yaml
      - model: Qwen 2.5 7B
        mad_tag: primus_pyt_megatron_lm_train_qwen2.5-7b
        config_name: primus_qwen2.5_7B-pretrain.yaml
@@ -1,13 +1,13 @@
 docker:
-  pull_tag: rocm/primus:v26.2
-  docker_hub_url: https://hub.docker.com/layers/rocm/primus/v26.2/images/sha256-9148d1bfcd579bf92f44bd89090e0d8c958f149c134b4b34b9674ab559244585
+  pull_tag: rocm/primus:v26.3
+  docker_hub_url: https://hub.docker.com/layers/rocm/primus/v26.3/images/sha256-da50bfe9dc4bb70ad683d3ee4a176dccb66f596f48d30e8e52323b41892759b1
  components:
-    ROCm: 7.2.0
-    PyTorch: 2.10.0a0+git449b176
+    ROCm: 7.2.1
+    PyTorch: 2.10.0+git94c6e04
    Python: "3.12.3"
-    Transformer Engine: 2.8.0.dev0+51f74fa7
+    Transformer Engine: 2.12.0.dev0+40434cf6
    Flash Attention: 2.8.3
-    hipBLASLt: 1.2.0-de5c1aebb6
+    hipBLASLt: 1.3.0-c4b2dc9869
    Triton: 3.6.0
    RCCL: 2.27.7
 model_groups:
@@ -0,0 +1,321 @@
+:orphan:
+:no-search:
+
+.. meta::
+   :description: Learn to validate diffusion model video generation on MI300X, MI350X and MI355X accelerators using
+                 prebuilt and optimized docker images.
+   :keywords: xDiT, diffusion, video, video generation, image, image generation, validate, benchmark
+
+************************
+xDiT diffusion inference
+************************
+
+.. caution::
+
+   This documentation does not reflect the latest version of the xDiT diffusion
+   inference performance documentation. See
+   :doc:`/how-to/rocm-for-ai/inference/xdit-diffusion-inference` for the latest
+   version.
+
+.. _xdit-video-diffusion-264:
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
+
+   {% set docker = data.docker %}
+
+   The `rocm/pytorch-xdit <{{ docker.docker_hub_url }}>`_ Docker image offers a prebuilt, optimized environment based on `xDiT <https://github.com/xdit-project/xDiT>`_ for
+   benchmarking diffusion model video and image generation on gfx942 and gfx950 series (AMD Instinct™ MI300X, MI325X, MI350X, and MI355X) GPUs.
+   The image runs `ROCm {{docker.ROCm}} (preview) <https://rocm.docs.amd.com/en/7.12.0-preview/about/release-notes.html>`__ based on `TheRock <https://github.com/ROCm/TheRock>`_
+   and includes the following components:
+
+   .. dropdown:: Software components - {{ docker.pull_tag.split('-')|last }}
+
+      .. list-table::
+         :header-rows: 1
+
+         * - Software component
+           - Version
+
+         {% for component_name, component_data in docker.components.items() %}
+         * - `{{ component_name }} <{{ component_data.url }}>`_
+           - {{ component_data.version }}
+         {% endfor %}
+
+Follow this guide to pull the required image, spin up a container, download the model, and run a benchmark.
+For preview and development releases, see `amdsiloai/pytorch-xdit <https://hub.docker.com/r/amdsiloai/pytorch-xdit>`_.
+
+What's new
+==========
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
+
+   {% set docker = data.docker %}
+
+   {% for item in docker.whats_new %}
+   * {{ item }}
+   {% endfor %}
+
+.. _xdit-video-diffusion-supported-models-264:
+
+Supported models
+================
+
+The following models are supported for inference performance benchmarking.
+Some instructions, commands, and recommendations in this documentation might
+vary by model -- select one to get started.
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
+
+   {% set docker = data.docker %}
+
+   .. raw:: html
+
+      <div id="vllm-benchmark-ud-params-picker" class="container-fluid">
+          <div class="row gx-0">
+              <div class="col-2 me-1 px-2 model-param-head">Model</div>
+              <div class="row col-10 pe-0">
+        {% for model_group in docker.supported_models %}
+               <div class="col-6 px-2 model-param" data-param-k="model-group" data-param-v="{{ model_group.js_tag }}" tabindex="0">{{ model_group.group }}</div>
+        {% endfor %}
+              </div>
+          </div>
+
+          <div class="row gx-0 pt-1">
+              <div class="col-2 me-1 px-2 model-param-head">Variant</div>
+              <div class="row col-10 pe-0">
+        {% for model_group in docker.supported_models %}
+            {% set models = model_group.models %}
+            {% for model in models %}
+                {% if models|length % 3 == 0 %}
+                <div class="col-4 px-2 model-param" data-param-k="model" data-param-v="{{ model.js_tag }}" data-param-group="{{ model_group.js_tag }}" tabindex="0">{{ model.model }}</div>
+                {% else %}
+                <div class="col-6 px-2 model-param" data-param-k="model" data-param-v="{{ model.js_tag }}" data-param-group="{{ model_group.js_tag }}" tabindex="0">{{ model.model }}</div>
+                {% endif %}
+            {% endfor %}
+        {% endfor %}
+              </div>
+          </div>
+      </div>
+
+   {% for model_group in docker.supported_models %}
+       {% for model in model_group.models %}
+
+   .. container:: model-doc {{ model.js_tag }}
+
+      .. note::
+
+         To learn more about your specific model see the `{{ model.model }} model card on Hugging Face <{{ model.url }}>`_
+         or visit the `GitHub page <{{ model.github }}>`__. Note that some models require access authorization before use via an
+         external license agreement through a third party.
+
+       {% endfor %}
+   {% endfor %}
+
+System validation
+=================
+
+Before running AI workloads, it's important to validate that your AMD hardware is configured
+correctly and performing optimally.
+
+If you have already validated your system settings, including aspects like NUMA auto-balancing, you
+can skip this step. Otherwise, complete the procedures in the :ref:`System validation and
+optimization <rocm-for-ai-system-optimization>` guide to properly configure your system settings
+before starting.
+
+To test for optimal performance, consult the recommended :ref:`System health benchmarks
+<rocm-for-ai-system-health-bench>`. This suite of tests will help you verify and fine-tune your
+system's configuration.
+
+Pull the Docker image
+=====================
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
+
+   {% set docker = data.docker %}
+
+   For this tutorial, it's recommended to use the latest ``{{ docker.pull_tag }}`` Docker image.
+   Pull the image using the following command:
+
+   .. code-block:: shell
+
+      docker pull {{ docker.pull_tag }}
+
+Validate and benchmark
+======================
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
+
+   {% set docker = data.docker %}
+
+   Once the image has been downloaded you can follow these steps to
+   run benchmarks and generate outputs.
+
+   {% for model_group in docker.supported_models %}
+     {% for model in model_group.models %}
+
+   .. container:: model-doc {{model.js_tag}}
+
+      The following commands are written for {{ model.model }}.
+      See :ref:`xdit-video-diffusion-supported-models-264` to switch to another available model.
+
+     {% endfor %}
+   {% endfor %}
+
+Choose your setup method
+------------------------
+
+You can either use an existing Hugging Face cache or download the model fresh inside the container.
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
+
+   {% set docker = data.docker %}
+
+   {% for model_group in docker.supported_models %}
+     {% for model in model_group.models %}
+   .. container:: model-doc {{model.js_tag}}
+
+      .. tab-set::
+
+         .. tab-item:: Option 1: Use existing Hugging Face cache
+
+            If you already have models downloaded on your host system, you can mount your existing cache.
+
+            1. Set your Hugging Face cache location.
+
+               .. code-block:: shell
+
+                  export HF_HOME=/your/hf_cache/location
+
+            2. Download the model (if not already cached).
+
+               .. code-block:: shell
+
+                  hf download {{ model.model_repo }} {% if model.revision %} --revision {{ model.revision }} {% endif %}
+
+            3. Launch the container with mounted cache.
+
+               .. code-block:: shell
+
+                  docker run \
+                      -it --rm \
+                      --cap-add=SYS_PTRACE \
+                      --security-opt seccomp=unconfined \
+                      --user root \
+                      --device=/dev/kfd \
+                      --device=/dev/dri \
+                      --group-add video \
+                      --ipc=host \
+                      --network host \
+                      --privileged \
+                      --shm-size 128G \
+                      --name pytorch-xdit \
+                      -e HSA_NO_SCRATCH_RECLAIM=1 \
+                      -e OMP_NUM_THREADS=16 \
+                      -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
+                      -e HF_HOME=/app/huggingface_models \
+                      -v $HF_HOME:/app/huggingface_models \
+                      {{ docker.pull_tag }}
+
+         .. tab-item:: Option 2: Download inside container
+
+            If you prefer to keep the container self-contained or don't have an existing cache.
+
+            1. Launch the container
+
+               .. code-block:: shell
+
+                  docker run \
+                      -it --rm \
+                      --cap-add=SYS_PTRACE \
+                      --security-opt seccomp=unconfined \
+                      --user root \
+                      --device=/dev/kfd \
+                      --device=/dev/dri \
+                      --group-add video \
+                      --ipc=host \
+                      --network host \
+                      --privileged \
+                      --shm-size 128G \
+                      --name pytorch-xdit \
+                      -e HSA_NO_SCRATCH_RECLAIM=1 \
+                      -e OMP_NUM_THREADS=16 \
+                      -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
+                      {{ docker.pull_tag }}
+
+            2. Inside the container, set the Hugging Face cache location and download the model.
+
+               .. code-block:: shell
+
+                  export HF_HOME=/app/huggingface_models
+                  hf download {{ model.model_repo }} {% if model.revision %} --revision {{ model.revision }} {% endif %}
+
+               .. warning::
+
+                  Models will be downloaded to the container's filesystem and will be lost when the container is removed unless you persist the data with a volume.
+     {% endfor %}
+   {% endfor %}
+
+Run inference
+=============
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
+
+   {% set docker = data.docker %}
+
+   {% for model_group in docker.supported_models %}
+     {% for model in model_group.models %}
+
+   .. container:: model-doc {{ model.js_tag }}
+
+      .. tab-set::
+
+         .. tab-item:: MAD-integrated benchmarking
+
+            1. Clone the ROCm Model Automation and Dashboarding (`<https://github.com/ROCm/MAD>`__) repository to a local
+               directory and install the required packages on the host machine.
+
+               .. code-block:: shell
+
+                  git clone https://github.com/ROCm/MAD
+                  cd MAD
+                  pip install -r requirements.txt
+
+            2. On the host machine, use this command to run the performance benchmark test on
+               the `{{model.model}} <{{ model.url }}>`_ model using one node.
+
+               .. code-block:: shell
+
+                  export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
+                  madengine run \
+                      --tags {{model.mad_tag}} \
+                      --keep-model-dir \
+                      --live-output
+
+            MAD launches a Docker container with the name
+            ``container_ci-{{model.mad_tag}}``. The throughput and serving reports of the
+            model are collected in the following paths: ``{{ model.mad_tag }}_throughput.csv``
+            and ``{{ model.mad_tag }}_serving.csv``.
+
+         .. tab-item:: Standalone benchmarking
+
+            To run the benchmarks for {{ model.model }}, use the following command:
+
+            .. code-block:: shell
+
+               {{ model.benchmark_command
+                  | map('replace', '{model_repo}', model.model_repo)
+                  | map('trim')
+                  | join('\n               ') }}
+
+            The generated content and timing information will be stored under the results directory.
+
+      {% endfor %}
+    {% endfor %}
+
+Previous versions
+=================
+
+See
+:doc:`/how-to/rocm-for-ai/inference/benchmark-docker/previous-versions/xdit-history`
+to find documentation for previous releases of xDiT diffusion inference
+performance testing.
@@ -15,12 +15,20 @@ benchmarking, see the version-specific documentation.
     - Components
     - Resources

-   * - ``rocm/pytorch-xdit:v26.4`` (latest)
+   * - ``rocm/pytorch-xdit:v26.5`` (latest)
+     -
+       * ROCm 7.13.0
+       * TheRock cbff3d1
+     -
+       * :doc:`Documentation </how-to/rocm-for-ai/inference/xdit-diffusion-inference>`
+       * `Docker Hub <https://hub.docker.com/layers/rocm/pytorch-xdit/v26.5/images/sha256-b8ad9fd4b41bc116ac2aff07c1066bf369cf7fc110b1a323f6302191985a51fd>`__
+
+   * - ``rocm/pytorch-xdit:v26.4``
     -
       * `ROCm 7.12.0 preview <https://rocm.docs.amd.com/en/7.12.0-preview/about/release-notes.html>`__
       * TheRock 9b611c6
     -
-       * :doc:`Documentation </how-to/rocm-for-ai/inference/xdit-diffusion-inference>`
+       * :doc:`Documentation <xdit-26.4>`
       * `Docker Hub <https://hub.docker.com/layers/rocm/pytorch-xdit/v26.4/images/sha256-b4296a638eb8dc7ebcafc808e180b78a3c44177580c21986082ec9539496067c>`__

   * - ``rocm/pytorch-xdit:v26.3``
@@ -15,7 +15,7 @@ xDiT diffusion inference

   The `rocm/pytorch-xdit <{{ docker.docker_hub_url }}>`_ Docker image offers a prebuilt, optimized environment based on `xDiT <https://github.com/xdit-project/xDiT>`_ for
   benchmarking diffusion model video and image generation on gfx942 and gfx950 series (AMD Instinct™ MI300X, MI325X, MI350X, and MI355X) GPUs.
-   The image runs `ROCm {{docker.ROCm}} (preview) <https://rocm.docs.amd.com/en/7.12.0-preview/about/release-notes.html>`__ based on `TheRock <https://github.com/ROCm/TheRock>`_
+   The image runs `ROCm {{docker.ROCm}}` based on `TheRock <https://github.com/ROCm/TheRock>`_
   and includes the following components:

   .. dropdown:: Software components - {{ docker.pull_tag.split('-')|last }}
@@ -52,7 +52,7 @@ MaxText with on ROCm provides the following key features to train large language

 - NANOO FP8 (for MI300X series GPUs) and FP8 (for MI355X and MI350X) quantization support

-.. _amd-maxtext-model-support-v26.2:
+.. _amd-maxtext-model-support-v26.3:

 Supported models
 ================
@@ -72,7 +72,7 @@ started.
            <div class="col-2 me-1 px-2 model-param-head">Model</div>
            <div class="row col-10 pe-0">
      {% for model_group in model_groups %}
-               <div class="col-4 px-2 model-param" data-param-k="model-group" data-param-v="{{ model_group.tag }}" tabindex="0">{{ model_group.group }}</div>
+               <div class="col-3 px-2 model-param" data-param-k="model-group" data-param-v="{{ model_group.tag }}" tabindex="0">{{ model_group.group }}</div>
      {% endfor %}
            </div>
         </div>
@@ -134,7 +134,7 @@ Use the following command to pull the Docker image from Docker Hub.

      docker pull {{ docker.pull_tag }}

-.. _amd-maxtext-multi-node-setup-v26.2:
+.. _amd-maxtext-multi-node-setup-v26.3:

 Multi-node configuration
 ------------------------
@@ -142,7 +142,7 @@ Multi-node configuration
 See :doc:`/how-to/rocm-for-ai/system-setup/multi-node-setup` to configure your
 environment for multi-node training.

-.. _amd-maxtext-get-started-v26.2:
+.. _amd-maxtext-get-started-v26.3:

 Benchmarking
 ============
@@ -169,7 +169,7 @@ benchmark results:
            .. container:: model-doc {{ model.mad_tag }}

               The following run commands are tailored to {{ model.model }}.
-               See :ref:`amd-maxtext-model-support-v26.2` to switch to another available model.
+               See :ref:`amd-maxtext-model-support-v26.3` to switch to another available model.

               .. rubric:: Download the Docker image and required packages

@@ -214,7 +214,7 @@ benchmark results:

                     git clone https://github.com/AMD-AIG-AIMA/Primus.git
                     cd Primus
-                     git checkout dev/fuyuajin/maxtext-backend-test
+                     git checkout main
                     git submodule update --init third_party/maxtext/

               .. rubric:: Run the training job with primus-cli
@@ -265,7 +265,7 @@ benchmark results:

                       .. code-block:: shell

-                          ./primus-cli container --image rocm/jax-training:maxtext-v26.2 \
+                          ./primus-cli container --image rocm/jax-training:maxtext-v26.3 \
                            -- train pretrain \
                            --config examples/maxtext/configs/MI300X/{{ model.primus_config_name }}

@@ -301,7 +301,7 @@ benchmark results:
         .. tab-item:: MAD-integrated benchmarking

            The following run command is tailored to {{ model.model }}.
-            See :ref:`amd-maxtext-model-support-v26.2` to switch to another available model.
+            See :ref:`amd-maxtext-model-support-v26.3` to switch to another available model.

            1. Clone the ROCm Model Automation and Dashboarding (`<https://github.com/ROCm/MAD>`__) repository to a local
               directory and install the required packages on the host machine.
@@ -332,7 +332,7 @@ benchmark results:
         .. tab-item:: Standalone benchmarking

            The following commands are optimized for {{ model.model }}. See
-            :ref:`amd-maxtext-model-support-v26.2` to switch to another
+            :ref:`amd-maxtext-model-support-v26.3` to switch to another
            available model. Some instructions and resources might not be
            available for all models and configurations.

@@ -452,7 +452,7 @@ benchmark results:

            [docker_image] (optional)
               The Docker image to use. If not specified, it defaults to
-               ``rocm/jax-training:maxtext-v26.2``.
+               ``rocm/jax-training:maxtext-v26.3``.

            For example, to run a multi-node training benchmark on {{ model.model }}:

@@ -477,18 +477,192 @@ benchmark results:
         {% else %}
            .. rubric:: Multi-node training

-            For multi-node training examples, choose a model from :ref:`amd-maxtext-model-support-v26.2`
+            For multi-node training examples, choose a model from :ref:`amd-maxtext-model-support-v26.3`
            with an available `multi-node training script <https://github.com/ROCm/MAD/tree/develop/scripts/jax-maxtext/env_scripts>`__.
         {% endif %}
      {% endfor %}
   {% endfor %}

+Profiling with JAX XPlane Profiler
+===================================
+
+MaxText has built-in XPlane profiling support via JAX's profiler. Traces
+capture GPU kernel timelines, RCCL collectives, HLO graphs, and more. The
+output can be viewed in TensorBoard's Trace Viewer or analyzed with
+TraceLens.
+
+Key MaxText profiler flags
+--------------------------
+
+The following MaxText config keys control profiling:
+
+.. code-block:: text
+
+   profiler=xplane                    # Use xplane format (produces .xplane.pb files)
+   skip_first_n_steps_for_profiler=2  # Skip compilation/warmup steps
+   profiler_steps=5                   # Number of steps to profile
+   upload_all_profiler_results=True   # Save all GPU profiles (not just GPU0)
+
+``steps`` should be greater than ``skip_first_n_steps_for_profiler`` +
+``profiler_steps`` (for example, ``steps=12`` with ``skip=2`` and
+``profile=5`` gives 5 warmup + 5 profiled + 2 cooldown).
+``skip_first_n_steps_for_profiler=2`` skips step 0 (compilation) and step
+1 (warmup). ``profiler_steps=5`` is typically sufficient; more steps
+produce larger ``.xplane.pb`` files.
+
+Profiling with MAD or madengine
+-------------------------------
+
+The model YAML configs under ``scripts/jax-maxtext/env_scripts/`` include
+a ``profiler`` key (set to ``""`` by default). To enable profiling when
+running through MAD or madengine, edit the YAML config for your model and
+set the profiler fields:
+
+.. code-block:: yaml
+
+   profiler: "xplane"
+   skip_first_n_steps_for_profiler: 2
+   profiler_steps: 5
+   upload_all_profiler_results: True
+   steps: 12
+
+Then run the benchmark as usual:
+
+.. code-block:: shell
+
+   export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
+   madengine run \
+       --tags jax_maxtext_train_llama-3.1-8b \
+       --keep-model-dir \
+       --live-output \
+       --timeout 28800
+
+Use ``--keep-model-dir`` so the container's output directory is preserved
+after the run. Profile output is written under the ``base_output_directory``
+specified in the YAML.
+
+Example: Profile a model standalone in Docker
+----------------------------------------------
+
+.. code-block:: shell
+
+   #!/bin/bash
+   set -e
+
+   IMAGE="$1"       # Docker image, e.g. rocm/jax-training:maxtext-v26.3
+   TAG="$2"         # Short tag for output folder, e.g. v26.3_llama2_7b
+   PROFILE_DIR="/path/to/profiles/${TAG}"
+
+   mkdir -p "${PROFILE_DIR}"
+
+   docker run --rm --privileged --network=host \
+     --device=/dev/dri --device=/dev/kfd --ipc=host \
+     -v "${PROFILE_DIR}:/mnt/profile" \
+     "${IMAGE}" bash -c '
+   export XLA_PYTHON_CLIENT_MEM_FRACTION=.97
+   export LD_LIBRARY_PATH=/usr/local/lib/:/opt/rocm/lib:$LD_LIBRARY_PATH
+   export XLA_FLAGS="--xla_gpu_enable_latency_hiding_scheduler=True --xla_gpu_enable_command_buffer= <your other XLA flags>"
+   export GPU_MAX_HW_QUEUES=2
+
+   cd /workspace/maxtext
+
+   python3 -m MaxText.train src/MaxText/configs/base.yml \
+     run_name=profile \
+     base_output_directory=/mnt/profile \
+     hardware=gpu \
+     steps=12 \
+     model_name=<your-model> \
+     dataset_type=synthetic \
+     enable_checkpointing=False \
+     enable_goodput_recording=False \
+     monitor_goodput=False \
+     <your model-specific flags> \
+     profiler=xplane \
+     skip_first_n_steps_for_profiler=2 \
+     profiler_steps=5 \
+     upload_all_profiler_results=True
+   ' 2>&1 | tee "${PROFILE_DIR}/run.log"
+
+   echo "Profile files:"
+   find "${PROFILE_DIR}" -name "*.xplane.pb" -o -name "*.trace.json.gz" 2>/dev/null
+
+Output structure
+----------------
+
+MaxText writes profiles in TensorBoard format:
+
+.. code-block:: text
+
+   <base_output_directory>/
+   └── profile/
+       └── tensorboard/
+           └── plugins/
+               └── profile/
+                   └── <YYYY_MM_DD_HH_MM_SS>/
+                       ├── <hostname>.xplane.pb          # Raw XPlane proto (GPU timelines)
+                       ├── <hostname>.trace.json.gz       # Trace viewer data
+                       └── *.hlo_proto.pb                 # HLO graphs for each compiled module
+
+Viewing traces in TensorBoard
+-----------------------------
+
+.. code-block:: shell
+
+   pip install tensorboard tensorboard-plugin-profile
+
+   # Point --logdir at the directory containing the tensorboard/ folder
+   tensorboard --logdir /path/to/profiles/<TAG>/profile --port 6006
+
+Navigate to **Profile > Trace Viewer** in the TensorBoard UI. Zoom into a
+single training step (skip the first profiled step as it may have residual
+warmup) and look at individual GPU streams to see compute/RCCL overlap.
+
+To keep profile files small, use ``profiler_steps=5`` to keep
+``.xplane.pb`` files under approximately 100 MB. Too many steps can produce
+files over 500 MB that TensorBoard struggles to load. Use
+``enable_checkpointing=False`` to avoid checkpoint I/O noise in the trace,
+and ``dataset_type=synthetic`` to eliminate data loading variability.
+
+Profiling with rocprofv3
+========================
+
+If you need to collect a trace without the JAX profiler, use ``rocprofv3``:
+
+.. code-block:: shell
+
+   rocprofv3 --hip-trace --kernel-trace --memory-copy-trace --rccl-trace \
+       --output-format pftrace -d ./v3_traces -- <command>
+
+Replace ``<command>`` with the command you want to profile, such as
+``./jax-maxtext_benchmark_report.sh -m Llama-2-7B``. Use ``-d
+<TRACE_DIRECTORY>`` to specify where the ``.json`` traces are saved. The
+resulting traces can be opened in `Perfetto <https://ui.perfetto.dev/>`__.
+
 Known issues
 ============

- You might see NaNs in the losses when setting ``packing=True``. As
-  a workaround, turn off input sequence packing (``packing=False``).
-  This will be fixed in a future release.
+- You might see NaNs in the losses while using real data (not synthetic
+  data) when setting ``packing=True`` and ``NVTE_CK_IS_V3_ATOMIC_FP32=0``.
+  Set ``NVTE_CK_IS_V3_ATOMIC_FP32=1`` for production training when using
+  real data and input sequence packing (``packing=True``).
+
+- There is a known slight performance regression for DeepSeek-V2-lite
+  (16B) in v26.3. This is being tracked and will be addressed in a future
+  release.
+
+- **JAX 0.9.1 Early Access known issues:**
+
+  - There is a known performance regression for MoE models
+    (DeepSeek-V2-lite and Mixtral-8x7B).
+
+  - The trace viewer in profiling may be missing some information in the
+    flame graph.
+
+- Shardy is a new config in JAX 0.6.0. You might get related errors if
+  it's not configured correctly. To disable it, set ``shardy=False``
+  during the training run. See the `Shardy migration guide
+  <https://docs.jax.dev/en/latest/shardy_jax_migration.html>`__ to
+  enable it.

 Further reading
 ===============
@@ -1,4 +1,5 @@
 :orphan:
+:no-search:

 .. meta::
   :description: How to train a model using Megatron-LM for ROCm.
@@ -0,0 +1,515 @@
+:orphan:
+:no-search:
+
+.. meta::
+   :description: How to train a model using JAX MaxText for ROCm.
+   :keywords: ROCm, AI, LLM, train, jax, torch, Llama, flux, tutorial, docker
+
+********************************************
+Training a model with Primus and JAX MaxText
+********************************************
+
+.. caution::
+
+   This documentation does not reflect the latest version of ROCm JAX MaxText
+   training performance documentation. See :doc:`../jax-maxtext` for the latest version.
+
+The JAX MaxText for ROCm training Docker image provides a prebuilt environment
+for training on AMD Instinct MI355X, MI350X, MI325X, and MI300X GPUs, with
+essential components such as JAX, XLA, ROCm libraries, and MaxText utilities.
+
+The image also integrates with `Primus <https://github.com/AMD-AGI/Primus>`__,
+a high-level training framework that supports multiple backends. You can use
+the unified ``primus-cli`` to run training jobs using the JAX MaxText backend.
+
+It includes the following software components:
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/jax-maxtext-v26.2-benchmark-models.yaml
+
+   {% set dockers = data.dockers %}
+   .. tab-set::
+
+      {% for docker in dockers %}
+      {% set jax_version = docker.components["JAX"] %}
+
+      .. tab-item:: ``{{ docker.pull_tag }}``
+         :sync: {{ docker.pull_tag }}
+
+         .. list-table::
+            :header-rows: 1
+
+            * - Software component
+              - Version
+
+            {% for component_name, component_version in docker.components.items() %}
+            * - {{ component_name }}
+              - {{ component_version }}
+
+            {% endfor %}
+      {% endfor %}
+
+MaxText with on ROCm provides the following key features to train large language models efficiently:
+
+- Transformer Engine (TE)
+
+- Flash Attention (FA) 3 -- with or without sequence input packing
+
+- GEMM tuning
+
+- Multi-node support
+
+- NANOO FP8 (for MI300X series GPUs) and FP8 (for MI355X and MI350X) quantization support
+
+.. _amd-maxtext-model-support-v26.2:
+
+Supported models
+================
+
+The following models are pre-optimized for performance on AMD Instinct
+GPUs. Some instructions, commands, and available training
+configurations in this documentation might vary by model -- select one to get
+started.
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/jax-maxtext-v26.2-benchmark-models.yaml
+
+   {% set model_groups = data.model_groups %}
+   .. raw:: html
+
+      <div id="vllm-benchmark-ud-params-picker" class="container-fluid">
+         <div class="row gx-0">
+            <div class="col-2 me-1 px-2 model-param-head">Model</div>
+            <div class="row col-10 pe-0">
+      {% for model_group in model_groups %}
+               <div class="col-4 px-2 model-param" data-param-k="model-group" data-param-v="{{ model_group.tag }}" tabindex="0">{{ model_group.group }}</div>
+      {% endfor %}
+            </div>
+         </div>
+
+         <div class="row gx-0 pt-1">
+            <div class="col-2 me-1 px-2 model-param-head">Variant</div>
+            <div class="row col-10 pe-0">
+      {% for model_group in model_groups %}
+         {% set models = model_group.models %}
+         {% for model in models %}
+            {% if models|length % 3 == 0 %}
+               <div class="col-4 px-2 model-param" data-param-k="model" data-param-v="{{ model.mad_tag }}" data-param-group="{{ model_group.tag }}" tabindex="0">{{ model.model }}</div>
+            {% else %}
+               <div class="col-6 px-2 model-param" data-param-k="model" data-param-v="{{ model.mad_tag }}" data-param-group="{{ model_group.tag }}" tabindex="0">{{ model.model }}</div>
+            {% endif %}
+         {% endfor %}
+      {% endfor %}
+            </div>
+         </div>
+      </div>
+
+.. note::
+
+   Some models, such as Llama 3, require an external license agreement through
+   a third party (for example, Meta).
+
+System validation
+=================
+
+Before running AI workloads, it's important to validate that your AMD hardware is configured
+correctly and performing optimally.
+
+If you have already validated your system settings, including aspects like NUMA auto-balancing, you
+can skip this step. Otherwise, complete the procedures in the :ref:`System validation and
+optimization <rocm-for-ai-system-optimization>` guide to properly configure your system settings
+before starting training.
+
+To test for optimal performance, consult the recommended :ref:`System health benchmarks
+<rocm-for-ai-system-health-bench>`. This suite of tests will help you verify and fine-tune your
+system's configuration.
+
+Environment setup
+=================
+
+This Docker image is optimized for specific model configurations outlined
+as follows. Performance can vary for other training workloads, as AMD
+doesn’t validate configurations and run conditions outside those described.
+
+Pull the Docker image
+---------------------
+
+Use the following command to pull the Docker image from Docker Hub.
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/jax-maxtext-v26.2-benchmark-models.yaml
+
+   {% set docker = data.dockers[0] %}
+
+   .. code-block:: shell
+
+      docker pull {{ docker.pull_tag }}
+
+.. _amd-maxtext-multi-node-setup-v26.2:
+
+Multi-node configuration
+------------------------
+
+See :doc:`/how-to/rocm-for-ai/system-setup/multi-node-setup` to configure your
+environment for multi-node training.
+
+.. _amd-maxtext-get-started-v26.2:
+
+Benchmarking
+============
+
+Once the setup is complete, choose between two options to reproduce the
+benchmark results:
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/jax-maxtext-v26.2-benchmark-models.yaml
+
+   .. _vllm-benchmark-mad:
+
+   {% set docker = data.dockers[0] %}
+   {% set model_groups = data.model_groups %}
+   {% for model_group in model_groups %}
+      {% for model in model_group.models %}
+
+   .. container:: model-doc {{model.mad_tag}}
+
+      .. tab-set::
+
+         {% if model.primus_config_name %}
+         .. tab-item:: Primus benchmarking
+
+            .. container:: model-doc {{ model.mad_tag }}
+
+               The following run commands are tailored to {{ model.model }}.
+               See :ref:`amd-maxtext-model-support-v26.2` to switch to another available model.
+
+               .. rubric:: Download the Docker image and required packages
+
+               1. Pull the ``{{ docker.pull_tag }}`` Docker image from Docker Hub.
+
+                  .. code-block:: shell
+
+                     docker pull {{ docker.pull_tag }}
+
+               2. Run the Docker container.
+
+                  .. code-block:: shell
+
+                     docker run -it \
+                         --device /dev/dri \
+                         --device /dev/kfd \
+                         --network host \
+                         --ipc host \
+                         --group-add video \
+                         --cap-add SYS_PTRACE \
+                         --security-opt seccomp=unconfined \
+                         --privileged \
+                         -v $HOME:$HOME \
+                         -v $HOME/.ssh:/root/.ssh \
+                         -v $HF_HOME:/hf_cache \
+                         -e HF_HOME=/hf_cache \
+                         -e MAD_SECRETS_HFTOKEN=$MAD_SECRETS_HFTOKEN
+                         --shm-size 64G \
+                         --name training_env \
+                         {{ docker.pull_tag }}
+
+                  Use these commands if you exit the ``training_env`` container and need to return to it.
+
+                  .. code-block:: shell
+
+                     docker start training_env
+                     docker exec -it training_env bash
+
+               3. Clone the Primus repository.
+
+                  .. code-block:: shell
+
+                     git clone https://github.com/AMD-AIG-AIMA/Primus.git
+                     cd Primus
+                     git checkout dev/fuyuajin/maxtext-backend-test
+                     git submodule update --init third_party/maxtext/
+
+               .. rubric:: Run the training job with primus-cli
+
+               For detailed usage instructions for ``primus-cli``, see the
+               `Primus CLI User Guide
+               <https://github.com/AMD-AGI/Primus/blob/main/docs/cli/PRIMUS-CLI-GUIDE.md>`__.
+
+               Use the following examples to run training with ``primus-cli``:
+
+               - Direct mode: run directly on the current host or within an existing Docker container
+
+                 .. tab-set::
+
+                    .. tab-item:: MI355X
+                       :sync: mi355x
+
+                       .. code-block:: shell
+
+                          ./primus-cli direct \
+                            -- train pretrain \
+                            --config examples/maxtext/configs/MI355X/{{ model.primus_config_name }}
+
+                    .. tab-item:: MI300X
+                       :sync: mi300x
+
+                       .. code-block:: shell
+
+                          ./primus-cli direct \
+                            -- train pretrain \
+                            --config examples/maxtext/configs/MI300X/{{ model.primus_config_name }}
+
+               - Container mode: run in Docker containers
+
+                 .. tab-set::
+
+                    .. tab-item:: MI355X
+                       :sync: mi355x
+
+                       .. code-block:: shell
+
+                          ./primus-cli container --image {{ docker.pull_tag }} \
+                            -- train pretrain \
+                            --config examples/maxtext/configs/MI355X/{{ model.primus_config_name }}
+
+                    .. tab-item:: MI300X
+                       :sync: mi300x
+
+                       .. code-block:: shell
+
+                          ./primus-cli container --image rocm/jax-training:maxtext-v26.2 \
+                            -- train pretrain \
+                            --config examples/maxtext/configs/MI300X/{{ model.primus_config_name }}
+
+
+               - Slurm mode: run distributed training on a Slurm cluster
+
+                 .. tab-set::
+
+                    .. tab-item:: MI355X
+                       :sync: mi355x
+
+                       .. code-block:: shell
+
+                          # Use a custom config file, where you can specify
+                          # the Docker image and set environment variables.
+                          ./primus-cli --config my_maxtext_config.yaml slurm srun -N 8 \
+                            -- train pretrain \
+                            --config examples/maxtext/configs/MI355X/{{ model.primus_config_name }}
+
+                    .. tab-item:: MI300X
+                       :sync: mi300x
+
+                       .. code-block:: shell
+
+                          # Use a custom config file, where you can specify
+                          # the Docker image and set environment variables.
+                          ./primus-cli --config my_maxtext_config.yaml slurm srun -N 8 \
+                            -- train pretrain \
+                            --config examples/maxtext/configs/MI300X/{{ model.primus_config_name }}
+         {% endif %}
+
+         {% if model.mad_tag and "single-node" in model.doc_options %}
+         .. tab-item:: MAD-integrated benchmarking
+
+            The following run command is tailored to {{ model.model }}.
+            See :ref:`amd-maxtext-model-support-v26.2` to switch to another available model.
+
+            1. Clone the ROCm Model Automation and Dashboarding (`<https://github.com/ROCm/MAD>`__) repository to a local
+               directory and install the required packages on the host machine.
+
+               .. code-block:: shell
+
+                  git clone https://github.com/ROCm/MAD
+                  cd MAD
+                  pip install -r requirements.txt
+
+            2. Use this command to run the performance benchmark test on the {{ model.model }} model
+               using one GPU with the :literal:`{{model.precision}}` data type on the host machine.
+
+               .. code-block:: shell
+
+                  export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
+                  madengine run \
+                      --tags {{model.mad_tag}} \
+                      --keep-model-dir \
+                      --live-output \
+                      --timeout 28800
+
+            MAD launches a Docker container with the name
+            ``container_ci-{{model.mad_tag}}``. The latency and throughput reports of the
+            model are collected in the following path: ``~/MAD/perf.csv/``.
+         {% endif %}
+
+         .. tab-item:: Standalone benchmarking
+
+            The following commands are optimized for {{ model.model }}. See
+            :ref:`amd-maxtext-model-support-v26.2` to switch to another
+            available model. Some instructions and resources might not be
+            available for all models and configurations.
+
+            .. rubric:: Download the Docker image and required scripts
+
+            Run the JAX MaxText benchmark tool independently by starting the
+            Docker container as shown in the following snippet.
+
+            .. code-block:: shell
+
+               docker pull {{ docker.pull_tag }}
+
+            {% if model.model_repo and "single-node" in model.doc_options %}
+            .. rubric:: Single node training
+
+            1. Set up environment variables.
+
+               .. code-block:: shell
+
+                  export MAD_SECRETS_HFTOKEN=<Your Hugging Face token>
+                  export HF_HOME=<Location of saved/cached Hugging Face models>
+
+               ``MAD_SECRETS_HFTOKEN`` is your Hugging Face access token to access models, tokenizers, and data.
+               See `User access tokens <https://huggingface.co/docs/hub/en/security-tokens>`__.
+
+               ``HF_HOME`` is where ``huggingface_hub`` will store local data. See `huggingface_hub CLI <https://huggingface.co/docs/huggingface_hub/main/en/guides/cli#huggingface-cli-download>`__.
+               If you already have downloaded or cached Hugging Face artifacts, set this variable to that path.
+               Downloaded files typically get cached to ``~/.cache/huggingface``.
+
+            2. Launch the Docker container.
+
+               .. code-block:: shell
+
+                  docker run -it \
+                      --device=/dev/dri \
+                      --device=/dev/kfd \
+                      --network host \
+                      --ipc host \
+                      --group-add video \
+                      --cap-add=SYS_PTRACE \
+                      --security-opt seccomp=unconfined \
+                      --privileged \
+                      -v $HOME:$HOME \
+                      -v $HOME/.ssh:/root/.ssh \
+                      -v $HF_HOME:/hf_cache \
+                      -e HF_HOME=/hf_cache \
+                      -e MAD_SECRETS_HFTOKEN=$MAD_SECRETS_HFTOKEN
+                      --shm-size 64G \
+                      --name training_env \
+                      {{ docker.pull_tag }}
+
+            3. In the Docker container, clone the ROCm MAD repository and navigate to the
+               benchmark scripts directory at ``MAD/scripts/jax-maxtext``.
+
+               .. code-block:: shell
+
+                  git clone https://github.com/ROCm/MAD
+                  cd MAD/scripts/jax-maxtext
+
+            4. Run the setup scripts to install libraries and datasets needed
+               for benchmarking.
+
+               .. code-block:: shell
+
+                  ./jax-maxtext_benchmark_setup.sh -m {{ model.model_repo }}
+
+            5. To run the training benchmark without quantization, use the following command:
+
+               .. code-block:: shell
+
+                  ./jax-maxtext_benchmark_report.sh -m {{ model.model_repo }}
+
+               For quantized training, run the script with the appropriate option for your Instinct GPU.
+
+               .. tab-set::
+
+                  .. tab-item:: MI355X and MI350X
+
+                     For ``fp8`` quantized training on MI355X and MI350X GPUs, use the following command:
+
+                     .. code-block:: shell
+
+                        ./jax-maxtext_benchmark_report.sh -m {{ model.model_repo }} -q fp8
+
+                  {% if model.model_repo not in ["Llama-3.1-70B", "Llama-3.3-70B"] %}
+                  .. tab-item:: MI325X and MI300X
+
+                     For ``nanoo_fp8`` quantized training on MI300X series GPUs, use the following command:
+
+                     .. code-block:: shell
+
+                        ./jax-maxtext_benchmark_report.sh -m {{ model.model_repo }} -q nanoo_fp8
+                  {% endif %}
+
+            {% endif %}
+            {% if model.multinode_config and "multi-node" in model.doc_options %}
+            .. rubric:: Multi-node training
+
+            The following SLURM scripts will launch the Docker container and
+            run the benchmark. Run them outside of any Docker container. The
+            unified multi-node benchmark script accepts a configuration file
+            that specifies the model and training parameters.
+
+            .. code-block:: shell
+
+               sbatch -N <NUM_NODES> jax_maxtext_multinode_benchmark.sh <config_file.yml> [docker_image]
+
+            <NUM_NODES>
+               The number of nodes to use for training (for example, 2, 4,
+               8).
+
+            <config_file.yml>
+               Path to the YAML configuration file containing model and
+               training parameters. Configuration files are available in the
+               ``scripts/jax-maxtext/env_scripts/`` directory for different
+               models and GPU architectures.
+
+            [docker_image] (optional)
+               The Docker image to use. If not specified, it defaults to
+               ``rocm/jax-training:maxtext-v26.2``.
+
+            For example, to run a multi-node training benchmark on {{ model.model }}:
+
+            .. tab-set::
+
+               {% if model.multinode_config.gfx950 %}
+               .. tab-item:: MI355X and MI350X (gfx950)
+
+                  .. code-block:: bash
+
+                     sbatch -N 4 jax_maxtext_multinode_benchmark.sh {{ model.multinode_config.gfx950 }}
+               {% endif %}
+
+               {% if model.multinode_config.gfx942 %}
+               .. tab-item:: MI325X and MI300X (gfx942)
+
+                  .. code-block:: bash
+
+                     sbatch -N 4 jax_maxtext_multinode_benchmark.sh {{ model.multinode_config.gfx942 }}
+               {% endif %}
+
+         {% else %}
+            .. rubric:: Multi-node training
+
+            For multi-node training examples, choose a model from :ref:`amd-maxtext-model-support-v26.2`
+            with an available `multi-node training script <https://github.com/ROCm/MAD/tree/develop/scripts/jax-maxtext/env_scripts>`__.
+         {% endif %}
+      {% endfor %}
+   {% endfor %}
+
+Known issues
+============
+
+- You might see NaNs in the losses when setting ``packing=True``. As
+  a workaround, turn off input sequence packing (``packing=False``).
+
+Further reading
+===============
+
+- To learn more about MAD and the ``madengine`` CLI, see the `MAD usage guide <https://github.com/ROCm/MAD?tab=readme-ov-file#usage-guide>`__.
+
+- To learn more about system settings and management practices to configure your system for
+  AMD Instinct MI300X Series GPUs, see `AMD Instinct MI300X system optimization <https://instinct.docs.amd.com/projects/amdgpu-docs/en/latest/system-optimization/mi300x.html>`_.
+
+- For a list of other ready-made Docker images for AI with ROCm, see
+  `AMD Infinity Hub <https://www.amd.com/en/developer/resources/infinity-hub.html#f-amd_hub_category=AI%20%26%20ML%20Models>`_.
+
+Previous versions
+=================
+
+See :doc:`jax-maxtext-history` to find documentation for previous releases
+of the ``ROCm/jax-training`` Docker image.
@@ -16,12 +16,20 @@ previous releases of the ``ROCm/primus`` Docker image on `Docker Hub <https://hu
     - Components
     - Resources

-   * - v26.2 (latest)
+   * - v26.3 (latest)
     -
       * ROCm 7.2.0
       * PyTorch 2.10.0+git94c6e04
     -
       * :doc:`Primus Megatron documentation <../primus-megatron>`
+       * `Docker Hub <https://hub.docker.com/layers/rocm/primus/v26.3/images/sha256-da50bfe9dc4bb70ad683d3ee4a176dccb66f596f48d30e8e52323b41892759b1>`__
+
+   * - v26.2
+     -
+       * ROCm 7.2.0
+       * PyTorch 2.10.0+git94c6e04
+     -
+       * :doc:`Primus Megatron documentation <primus-megatron-v26.2>`
       * `Docker Hub <https://hub.docker.com/layers/rocm/primus/v26.2/images/sha256-9148d1bfcd579bf92f44bd89090e0d8c958f149c134b4b34b9674ab559244585>`__

   * - v26.1
@@ -0,0 +1,403 @@
+:orphan:
+:no-search:
+
+.. meta::
+   :description: How to train a model using PyTorch for ROCm.
+   :keywords: ROCm, AI, LLM, train, PyTorch, torch, Llama, flux, tutorial, docker
+
+****************************************
+Training a model with Primus and PyTorch
+****************************************
+
+.. caution::
+
+   This documentation does not reflect the latest version of ROCm Primus PyTorch training
+   performance benchmark documentation. See :doc:`../primus-pytorch` for the latest version.
+
+`Primus <https://github.com/AMD-AGI/Primus>`__ is a unified and flexible
+LLM training framework designed to streamline training. It streamlines LLM
+training on AMD Instinct GPUs using a modular, reproducible configuration paradigm.
+Primus now supports the PyTorch torchtitan backend.
+
+.. note::
+
+   For a unified training solution on AMD GPUs with ROCm, the `rocm/pytorch-training
+   <https://hub.docker.com/r/rocm/pytorch-training/>`__ Docker Hub registry will be
+   deprecated soon in favor of `rocm/primus <https://hub.docker.com/r/rocm/primus>`__.
+   The ``rocm/primus`` Docker containers will cover PyTorch training ecosystem frameworks,
+   including torchtitan and :doc:`Megatron-LM <../primus-megatron>`.
+
+   Primus with the PyTorch torchtitan backend is designed to replace the
+   :doc:`ROCm PyTorch training <../pytorch-training>` workflow. See
+   :doc:`../pytorch-training` to see steps to run workloads without Primus.
+
+AMD provides a ready-to-use Docker image for MI355X, MI350X, MI325X, and
+MI300X GPUs containing essential components for Primus and PyTorch training
+with Primus Turbo optimizations.
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/primus-pytorch-v26.2-benchmark-models.yaml
+
+   .. tab-set::
+
+      .. tab-item:: {{ data.docker.pull_tag }}
+         :sync: {{ data.docker.pull_tag }}
+
+         .. list-table::
+            :header-rows: 1
+
+            * - Software component
+              - Version
+
+            {% for component_name, component_version in data.docker.components.items() %}
+            * - {{ component_name }}
+              - {{ component_version }}
+            {% endfor %}
+
+.. _amd-primus-pytorch-model-support-v26.2:
+
+Supported models
+================
+
+The following models are pre-optimized for performance on the AMD Instinct MI325X and MI300X GPUs.
+Some instructions, commands, and training recommendations in this documentation might
+vary by model -- select one to get started.
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/primus-pytorch-v26.2-benchmark-models.yaml
+
+   {% set model_groups = data.model_groups %}
+   .. raw:: html
+
+      <div id="vllm-benchmark-ud-params-picker" class="container-fluid">
+         <div class="row gx-0">
+            <div class="col-2 me-1 px-2 model-param-head">Model</div>
+            <div class="row col-10 pe-0">
+      {% for model_group in model_groups %}
+               <div class="col-6 px-2 model-param" data-param-k="model-group" data-param-v="{{ model_group.tag }}" tabindex="0">{{ model_group.group }}</div>
+      {% endfor %}
+            </div>
+         </div>
+
+         <div class="row gx-0 pt-1">
+            <div class="col-2 me-1 px-2 model-param-head">Variant</div>
+            <div class="row col-10 pe-0">
+      {% for model_group in model_groups %}
+         {% set models = model_group.models %}
+         {% for model in models %}
+            {% if models|length % 3 == 0 %}
+               <div class="col-4 px-2 model-param" data-param-k="model" data-param-v="{{ model.mad_tag }}" data-param-group="{{ model_group.tag }}" tabindex="0">{{ model.model }}</div>
+            {% else %}
+               <div class="col-6 px-2 model-param" data-param-k="model" data-param-v="{{ model.mad_tag }}" data-param-group="{{ model_group.tag }}" tabindex="0">{{ model.model }}</div>
+            {% endif %}
+         {% endfor %}
+      {% endfor %}
+            </div>
+         </div>
+      </div>
+
+.. seealso::
+
+   For additional workloads, including Llama 3.3, Llama 3.2, Llama 2, GPT OSS, Qwen, and Flux models,
+   see the documentation :doc:`../pytorch-training` (without Primus)
+
+.. _amd-primus-pytorch-performance-measurements-v26.2:
+
+System validation
+=================
+
+Before running AI workloads, it's important to validate that your AMD hardware is configured
+correctly and performing optimally.
+
+If you have already validated your system settings, including aspects like NUMA auto-balancing, you
+can skip this step. Otherwise, complete the procedures in the :ref:`System validation and
+optimization <rocm-for-ai-system-optimization>` guide to properly configure your system settings
+before starting training.
+
+To test for optimal performance, consult the recommended :ref:`System health benchmarks
+<rocm-for-ai-system-health-bench>`. This suite of tests will help you verify and fine-tune your
+system's configuration.
+
+This Docker image is optimized for specific model configurations outlined
+below. Performance can vary for other training workloads, as AMD
+doesn’t test configurations and run conditions outside those described.
+
+Pull the Docker image
+=====================
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/primus-pytorch-v26.2-benchmark-models.yaml
+
+   Use the following command to pull the Docker image from Docker Hub.
+
+   .. code-block:: shell
+
+      docker pull {{ data.docker.pull_tag }}
+
+Run training
+============
+
+Once the setup is complete, choose between the following two workflows to start benchmarking training.
+For fine-tuning workloads and multi-node training examples, see :doc:`../pytorch-training` (without Primus).
+For best performance on MI325X, MI350X, and MI355X GPUs, you might need to
+tweak some configurations (such as batch sizes).
+
+.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/primus-pytorch-v26.2-benchmark-models.yaml
+
+   {% set docker = data.docker %}
+   {% set model_groups = data.model_groups %}
+
+   .. tab-set::
+
+      .. tab-item:: Primus benchmarking
+
+   {% for model_group in model_groups %}
+      {% for model in model_group.models %}
+
+         .. container:: model-doc {{ model.mad_tag }}
+
+            The following run commands are tailored to {{ model.model }}.
+            See :ref:`amd-primus-pytorch-model-support-v26.2` to switch to another available model.
+
+            .. rubric:: Download the Docker image and required packages
+
+            1. Pull the ``{{ docker.pull_tag }}`` Docker image from Docker Hub.
+
+               .. code-block:: shell
+
+                  docker pull {{ docker.pull_tag }}
+
+            2. Run the Docker container.
+
+               .. code-block:: shell
+
+                  docker run -it \
+                      --device /dev/dri \
+                      --device /dev/kfd \
+                      --network host \
+                      --ipc host \
+                      --group-add video \
+                      --cap-add SYS_PTRACE \
+                      --security-opt seccomp=unconfined \
+                      --privileged \
+                      -v $HOME:$HOME \
+                      -v $HOME/.ssh:/root/.ssh \
+                      --shm-size 64G \
+                      --name training_env \
+                      {{ docker.pull_tag }}
+
+               Use these commands if you exit the ``training_env`` container and need to return to it.
+
+               .. code-block:: shell
+
+                  docker start training_env
+                  docker exec -it training_env bash
+
+               The Docker container hosts verified commit ``9c529cd4`` of the `Primus
+               <https://github.com/AMD-AGI/Primus/tree/9c529cd4a934a68a880ede036c3e97b792e38167/>`__ repository.
+
+            .. rubric:: Prepare training datasets and dependencies
+
+            The following benchmarking examples require downloading models and datasets
+            from Hugging Face. To ensure successful access to gated repos, set your
+            ``HF_TOKEN``.
+
+            .. code-block:: shell
+
+               export HF_TOKEN=$your_personal_hugging_face_access_token
+
+            .. rubric:: Pretraining
+
+            To get started, navigate to the ``Primus`` directory in your container.
+
+            .. code-block::
+
+               cd /workspace/Primus
+
+            Now, to start the pretraining benchmark, use the ``run_pretrain.sh`` script
+            included with Primus with the appropriate options.
+
+            .. rubric:: Benchmarking examples
+
+            .. container:: model-doc primus_pyt_train_llama-3.1-8b
+
+               Use the following command to run train Llama 3.1 8B with BF16 precision using Primus torchtitan.
+
+               .. tab-set::
+
+                  .. tab-item:: MI355X and MI350X
+                     :sync: MI355X
+
+                     .. code-block:: shell
+
+                        bash runner/primus-cli direct \
+                          --log_file /tmp/primus_llama3.1_8B.log \
+                          -- train pretrain \
+                          --config examples/torchtitan/configs/MI355X/llama3.1_8B-BF16-pretrain.yaml
+
+                  .. tab-item:: MI300X
+                     :sync: MI300X
+
+                     .. code-block:: shell
+
+                        bash runner/primus-cli direct \
+                          --log_file /tmp/primus_llama3.1_8B.log \
+                          -- train pretrain \
+                          --config examples/torchtitan/configs/MI300X/llama3.1_8B-BF16-pretrain.yaml
+
+               To train Llama 3.1 8B with FP8 precision, use the following command.
+
+               .. tab-set::
+
+                  .. tab-item:: MI355X and MI350X
+                     :sync: MI355X
+
+                     .. code-block:: shell
+
+                        bash runner/primus-cli direct \
+                          --log_file /tmp/primus_llama3.1_8B_fp8.log \
+                          -- train pretrain \
+                          --config examples/torchtitan/configs/MI355X/llama3.1_8B-FP8-pretrain.yaml
+
+                  .. tab-item:: MI300X
+                     :sync: MI300X
+
+                     .. code-block:: shell
+
+                        bash runner/primus-cli direct \
+                          --log_file /tmp/primus_llama3.1_8B_fp8.log \
+                          -- train pretrain \
+                          --config examples/torchtitan/configs/MI300X/llama3.1_8B-FP8-pretrain.yaml
+
+            .. container:: model-doc primus_pyt_train_llama-3.1-70b
+
+               Use the following command to run train Llama 3.1 70B with BF16 precision using Primus torchtitan.
+
+               .. tab-set::
+
+                  .. tab-item:: MI355X and MI350X
+                     :sync: MI355X and MI300X
+
+                     .. code-block:: shell
+
+                        bash runner/primus-cli direct \
+                          --log_file /tmp/primus_llama3.1_70B.log \
+                          -- train pretrain \
+                          --config examples/torchtitan/configs/MI355X/llama3.1_70B-BF16-pretrain.yaml
+
+                  .. tab-item:: MI300X
+                     :sync: MI300X
+
+                     .. code-block:: shell
+
+                        bash runner/primus-cli direct \
+                          --log_file /tmp/primus_llama3.1_70B.log \
+                          -- train pretrain \
+                          --config examples/torchtitan/configs/MI300X/llama3.1_70B-BF16-pretrain.yaml
+
+               To train Llama 3.1 70B with FP8 precision, use the following command.
+
+               .. tab-set::
+
+                  .. tab-item:: MI355X and MI350X
+                     :sync: MI355X
+
+                     .. code-block:: shell
+
+                        bash runner/primus-cli direct \
+                          --log_file /tmp/primus_llama3.1_70B_fp8.log \
+                          -- train pretrain \
+                          --config examples/torchtitan/configs/MI355X/llama3.1_70B-FP8-pretrain.yaml
+
+                  .. tab-item:: MI300X
+                     :sync: MI300X
+
+                     .. code-block:: shell
+
+                        bash runner/primus-cli direct \
+                          --log_file /tmp/primus_llama3.1_70B_fp8.log \
+                          -- train pretrain \
+                          --config examples/torchtitan/configs/MI300X/llama3.1_70B-FP8-pretrain.yaml
+
+            .. container:: model-doc primus_pyt_train_deepseek-v3-16b
+
+               Use the following command to run train DeepSeek V3 16B with BF16 precision using Primus torchtitan.
+
+               .. tab-set::
+
+                  .. tab-item:: MI355X and MI350X
+                     :sync: MI355X and MI300X
+
+                     .. code-block:: shell
+
+                        bash runner/primus-cli direct \
+                          --log_file /tmp/primus_deepseek_v3_16b.log \
+                          -- train pretrain \
+                          --config examples/torchtitan/configs/MI355X/deepseek_v3_16b-pretrain.yaml
+
+                  .. tab-item:: MI300X
+                     :sync: MI300X
+
+                     .. code-block:: shell
+
+                        bash runner/primus-cli direct \
+                          --log_file /tmp/primus_deepseek_v3_16b.log \
+                          -- train pretrain \
+                          --config examples/torchtitan/configs/MI300X/deepseek_v3_16b-pretrain.yaml
+      {% endfor %}
+   {% endfor %}
+
+      .. tab-item:: MAD-integrated benchmarking
+
+   {% for model_group in model_groups %}
+      {% for model in model_group.models %}
+
+         .. container:: model-doc {{ model.mad_tag }}
+
+            The following run command is tailored to {{ model.model }}.
+            See :ref:`amd-primus-pytorch-model-support-v26.2` to switch to another available model.
+
+            1. Clone the ROCm Model Automation and Dashboarding (`<https://github.com/ROCm/MAD>`__) repository to a local
+               directory and install the required packages on the host machine.
+
+               .. code-block:: shell
+
+                  git clone https://github.com/ROCm/MAD
+                  cd MAD
+                  pip install -r requirements.txt
+
+            2. For example, use this command to run the performance benchmark test on the {{ model.model }} model
+               using one node with the {{ model.precision }} data type on the host machine.
+
+               .. code-block:: shell
+
+                  export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
+                  madengine run \
+                      --tags {{ model.mad_tag }} \
+                      --keep-model-dir \
+                      --live-output \
+                      --timeout 28800
+
+               MAD launches a Docker container with the name
+               ``container_ci-{{ model.mad_tag }}``. The latency and throughput reports of the
+               model are collected in ``~/MAD/perf.csv``.
+
+      {% endfor %}
+   {% endfor %}
+
+Further reading
+===============
+
+- For an introduction to Primus, see `Primus: A Lightweight, Unified Training
+  Framework for Large Models on AMD GPUs <https://rocm.blogs.amd.com/software-tools-optimization/primus/README.html>`__.
+
+- To learn more about MAD and the ``madengine`` CLI, see the `MAD usage guide <https://github.com/ROCm/MAD?tab=readme-ov-file#usage-guide>`__.
+
+- To learn more about system settings and management practices to configure your system for
+  AMD Instinct MI300X Series GPUs, see `AMD Instinct MI300X system optimization <https://instinct.docs.amd.com/projects/amdgpu-docs/en/latest/system-optimization/mi300x.html>`_.
+
+- For a list of other ready-made Docker images for AI with ROCm, see
+  `AMD Infinity Hub <https://www.amd.com/en/developer/resources/infinity-hub.html#f-amd_hub_category=AI%20%26%20ML%20Models>`_.
+
+Previous versions
+=================
+
+See :doc:`pytorch-training-history` to find documentation for previous releases
+of the ``ROCm/pytorch-training`` Docker image.
@@ -16,12 +16,20 @@ previous releases of the ``ROCm/primus`` Docker image on `Docker Hub <https://hu
     - Components
     - Resources

-   * - v26.2 (latest)
+   * - v26.3 (latest)
+     -
+       * ROCm 7.2.1
+       * PyTorch 2.10.0+git94c6e04
+     -
+       * :doc:`Primus PyTorch training documentation <../primus-pytorch>`
+       * `Docker Hub <https://hub.docker.com/layers/rocm/primus/v26.3/images/sha256-da50bfe9dc4bb70ad683d3ee4a176dccb66f596f48d30e8e52323b41892759b1>`__
+
+   * - v26.2
     -
       * ROCm 7.2.0
       * PyTorch 2.10.0+git94c6e04
     -
-       * :doc:`Primus PyTorch training documentation <../primus-pytorch>`
+       * :doc:`Primus PyTorch training documentation <primus-pytorch-v26.2>`
       * `Docker Hub <https://hub.docker.com/layers/rocm/primus/v26.2/images/sha256-9148d1bfcd579bf92f44bd89090e0d8c958f149c134b4b34b9674ab559244585>`__

   * - v26.1
@@ -14,9 +14,8 @@ model training. Performance acceleration is powered by `Primus Turbo

 .. note::

-   For a unified training solution on AMD GPUs with ROCm, the `rocm/megatron-lm
-   <https://hub.docker.com/r/rocm/megatron-lm/>`__ Docker Hub registry will be
-   deprecated soon in favor of `rocm/primus <https://hub.docker.com/r/rocm/primus>`__.
+   The ``rocm/pytorch-training`` Docker Hub registry will be deprecated soon in
+   favor of `rocm/primus <https://hub.docker.com/r/rocm/primus>`__.
   The ``rocm/primus`` Docker containers will cover PyTorch training ecosystem frameworks,
   including Megatron-LM and :doc:`torchtitan <primus-pytorch>`.

@@ -47,7 +46,7 @@ Megatron-LM.
              - {{ component_version }}
            {% endfor %}

-.. _amd-primus-megatron-lm-model-support-v26.2:
+.. _amd-primus-megatron-lm-model-support-v26.3:

 Supported models
 ================
@@ -65,21 +64,9 @@ might vary by model -- select one to get started.
         <div class="row gx-0">
            <div class="col-2 me-1 px-2 model-param-head">Model</div>
            <div class="row col-10 pe-0">
-               {% set tag = "llama" %}
-               {% set group = "Meta Llama" %}
-               <div class="col-6 px-2 model-param" data-param-k="model-group" data-param-v="{{ tag }}" tabindex="0">{{ group }}</div>
-               {% set tag = "zebra-llama" %}
-               {% set group = "AMD Zebra-Llama" %}
-               <div class="col-6 px-2 model-param" data-param-k="model-group" data-param-v="{{ tag }}" tabindex="0">{{ group }}</div>
-               {% set tag = "deepseek" %}
-               {% set group = "DeepSeek" %}
-               <div class="col-4 px-2 model-param" data-param-k="model-group" data-param-v="{{ tag }}" tabindex="0">{{ group }}</div>
-               {% set tag = "mistral" %}
-               {% set group = "Mistral AI" %}
-               <div class="col-4 px-2 model-param" data-param-k="model-group" data-param-v="{{ tag }}" tabindex="0">{{ group }}</div>
-               {% set tag = "qwen" %}
-               {% set group = "Qwen" %}
-               <div class="col-4 px-2 model-param" data-param-k="model-group" data-param-v="{{ tag }}" tabindex="0">{{ group }}</div>
+      {% for model_group in model_groups %}
+               <div class="col-6 px-2 model-param" data-param-k="model-group" data-param-v="{{ model_group.tag }}" tabindex="0">{{ model_group.group }}</div>
+      {% endfor %}
            </div>
         </div>

@@ -120,7 +107,7 @@ To test for optimal performance, consult the recommended :ref:`System health ben
 <rocm-for-ai-system-health-bench>`. This suite of tests will help you verify and fine-tune your
 system's configuration.

-.. _mi300x-amd-primus-megatron-lm-training-v26.2:
+.. _mi300x-amd-primus-megatron-lm-training-v26.3:

 Environment setup
 =================
@@ -130,7 +117,7 @@ Environment setup
   Use the following instructions to set up the environment, configure the script to train models, and
   reproduce the benchmark results on AMD Instinct GPUs.

-.. _amd-primus-megatron-lm-requirements-v26.2:
+.. _amd-primus-megatron-lm-requirements-v26.3:

 Pull the Docker image

@@ -169,16 +156,16 @@ Pull the Docker image
         docker start primus_training_env
         docker exec -it primus_training_env bash

-The Docker container hosts verified commit ``9c529cd4`` of the `Primus
-<https://github.com/AMD-AGI/Primus/tree/9c529cd4a934a68a880ede036c3e97b792e38167>`__ repository.
+The Docker container hosts verified commit ``43a6e00`` of the `Primus
+<https://github.com/AMD-AGI/Primus/tree/43a6e006c419697208295c5523b99070e8198ad9>`__ repository.

-.. _amd-primus-megatron-lm-environment-setup-v26.2:
+.. _amd-primus-megatron-lm-environment-setup-v26.3:

 Configuration
 =============

 Primus defines a training configuration in YAML for each model in
-`examples/megatron/configs <https://github.com/AMD-AGI/Primus/tree/9c529cd4a934a68a880ede036c3e97b792e38167/examples/megatron/configs>`__.
+`examples/megatron/configs <https://github.com/AMD-AGI/Primus/tree/43a6e006c419697208295c5523b99070e8198ad9/examples/megatron/configs>`__.

 .. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/primus-megatron-benchmark-models.yaml

@@ -219,7 +206,7 @@ You can use either mock data or real data for training.

  Ensure that the files are accessible inside the Docker container.

-.. _amd-primus-megatron-lm-tokenizer-v26.2:
+.. _amd-primus-megatron-lm-tokenizer-v26.3:

 Tokenizer
 ---------
@@ -232,7 +219,7 @@ right permissions to access the tokenizer for each model.
   # Export your HF_TOKEN in the workspace
   export HF_TOKEN=<your_hftoken>

-.. _amd-primus-megatron-lm-run-training-v26.2:
+.. _amd-primus-megatron-lm-run-training-v26.3:

 Run training
 ============
@@ -254,7 +241,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Llama 3.3 70B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run pre-training for Llama 3.3 70B BF16, run:

@@ -277,6 +264,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -289,7 +277,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Llama 3.1 8B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run pre-training for Llama 3.1 8B FP8, run:

@@ -312,6 +300,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -341,6 +330,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -353,7 +343,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Llama 3.1 70B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run pre-training for Llama 3.1 70B BF16, run:

@@ -378,6 +368,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -413,6 +404,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -429,7 +421,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Llama 2 7B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run pre-training for Llama 2 7B FP8, run:

@@ -452,6 +444,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -481,6 +474,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -493,7 +487,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Llama 2 70B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run pre-training for Llama 2 70B BF16, run:

@@ -516,6 +510,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -524,59 +519,11 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the
              -- train pretrain \
              --config examples/megatron/configs/MI300X/llama2_70B-BF16-pretrain.yaml

-.. container:: model-doc primus_pyt_megatron_lm_train_deepseek-v3-proxy
-
-   Once setup is complete, run the appropriate training command.
-   The following run commands are tailored to DeepSeek-V3.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
-
-   To run training on a single node for DeepSeek-V3 (MoE with expert parallel) BF16 with 3-layer proxy,
-   use the following command:
-
-   .. tab-set::
-
-      .. tab-item:: MI355X and MI350X
-         :sync: MI355X and MI350X
-
-         .. code-block:: shell
-
-            bash runner/primus-cli direct \
-              --log_file /tmp/primus_deepseek_v3_proxy.log \
-              -- train pretrain \
-              --config examples/megatron/configs/MI355X/deepseek_v3-BF16-pretrain.yaml \
-              --num_layers 3 \
-              --moe_layer_freq 1 \
-              --train_iters 50 \
-              --micro_batch_size 8 \
-              --global_batch_size 64 \
-              --moe_use_fused_router_with_aux_score True \
-              --moe_permute_fusion True
-
-      .. tab-item:: MI300X
-         :sync: MI325X and MI300X
-
-         .. code-block:: shell
-
-            # Set the variables for better performance
-            # only on MI325X and MI300X
-            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
-            export NVTE_CK_IS_V3_ATOMIC_FP32=1
-
-            bash runner/primus-cli direct \
-              --log_file /tmp/primus_deepseek_v3_proxy.log \
-              -- train pretrain \
-              --config examples/megatron/configs/MI300X/deepseek_v3-BF16-pretrain.yaml \
-              --num_layers 3 \
-              --moe_layer_freq 1 \
-              --micro_batch_size 3 \
-              --global_batch_size 192 \
-              --train_iters 50
-
 .. container:: model-doc primus_pyt_megatron_lm_train_deepseek-v2-lite-16b

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to DeepSeek-V2-Lite.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run training on a single node for DeepSeek-V2-Lite (MoE with expert parallel) BF16,
   use the following command:
@@ -604,6 +551,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -616,7 +564,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Mixtral 8x7B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run training on a single node for Mixtral 8x7B (MoE with expert parallel),
   use the following command:
@@ -640,6 +588,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -648,52 +597,11 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the
              -- train pretrain \
              --config examples/megatron/configs/MI300X/mixtral_8x7B_v0.1-BF16-pretrain.yaml

-.. container:: model-doc primus_pyt_megatron_lm_train_mixtral-8x22b-proxy
-
-   Once setup is complete, run the appropriate training command.
-   The following run commands are tailored to Mixtral 8x22B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
-
-   To run training on a single node for Mixtral 8x22B BF16 (MoE with expert parallel) 4-layer proxy,
-   use the following command:
-
-   .. tab-set::
-
-      .. tab-item:: MI355X and MI350X
-         :sync: MI355X and MI350X
-
-         .. code-block:: shell
-
-            bash runner/primus-cli direct \
-              --log_file /tmp/primus_mixtral_8x22B_proxy.log \
-              -- train pretrain \
-              --config examples/megatron/configs/MI355X/mixtral_8x22B_v0.1-BF16-pretrain.yaml
-
-      .. tab-item:: MI300X
-         :sync: MI325X and MI300X
-
-         .. code-block:: shell
-
-            # Set the variables for better performance
-            # only on MI325X and MI300X
-            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
-            export NVTE_CK_IS_V3_ATOMIC_FP32=1
-
-            bash runner/primus-cli direct \
-              --log_file /tmp/primus_mixtral_8x22B_proxy.log \
-              -- train pretrain \
-              --config examples/megatron/configs/MI300X/mixtral_8x22B_v0.1-BF16-pretrain.yaml \
-              --num_layers 4 \
-              --pipeline_model_parallel_size 1 \
-              --micro_batch_size 1 \
-              --global_batch_size 16 \
-              --train_iters 50
-
 .. container:: model-doc primus_pyt_megatron_lm_train_qwen3-32b-lora

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to post-training Qwen 3 32B (LoRA).
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run training on a single node for Qwen 3 32B BF16 (SFT), use the following
   command:
@@ -717,6 +625,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -729,7 +638,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to post-training Qwen 3 32B (SFT).
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run training on a single node for Qwen 3 32B BF16 (SFT), use the following
   command:
@@ -753,6 +662,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -765,7 +675,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Qwen 2.5 7B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run training on a single node for Qwen 2.5 7B BF16, use the following
   command:
@@ -789,6 +699,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -818,6 +729,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -830,7 +742,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Qwen 2.5 72B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run the training on a single node for Qwen 2.5 72B BF16, use the following command.

@@ -853,6 +765,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -865,7 +778,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Zebra-Llama 1B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run the training on a single node for AMD Zebra-Llama 1B BF16, use the following command.

@@ -888,6 +801,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -900,7 +814,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Zebra-Llama 3B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run the training on a single node for AMD Zebra-Llama 3B BF16, use the following command.

@@ -923,6 +837,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -935,7 +850,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Zebra Llama 8B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To run the training on a single node for AMD Zebra-Llama 8B BF16, use the following command.

@@ -958,6 +873,7 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the

            # Set the variables for better performance
            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
            export NVTE_CK_IS_V3_ATOMIC_FP32=1

@@ -966,7 +882,139 @@ To run training on a single node, navigate to ``/workspace/Primus`` and use the
              -- train pretrain \
              --config examples/megatron/configs/MI300X/zebra_llama_8B-pretrain.yaml

-.. _amd-primus-megatron-multi-node-examples-v26.2:
+.. container:: model-doc primus_pyt_megatron_lm_train_qwen3-30b-a3b
+
+   Once setup is complete, run the appropriate training command.
+   The following run commands are tailored to Qwen 3 30B (A3B).
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.
+
+   To run training on a single node for Qwen 3 30B (A3B) BF16, use the following command:
+
+   .. tab-set::
+
+      .. tab-item:: MI355X and MI350X
+         :sync: MI355X and MI350X
+
+         .. code-block:: shell
+
+            bash runner/primus-cli direct \
+              --log_file /tmp/primus_qwen3_30B_A3B.log \
+              -- train pretrain \
+              --config examples/megatron/configs/MI355X/qwen3_30B_A3B-BF16-pretrain.yaml
+
+      .. tab-item:: MI300X
+         :sync: MI325X and MI300X
+
+         .. code-block:: shell
+
+            # Set the variables for better performance
+            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
+            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
+            export NVTE_CK_IS_V3_ATOMIC_FP32=1
+
+            bash runner/primus-cli direct \
+              --log_file /tmp/primus_qwen3_30B_A3B.log \
+              -- train pretrain \
+              --config examples/megatron/configs/MI300X/qwen3_30B_A3B-BF16-pretrain.yaml
+
+   For FP8, use the following command.
+
+   .. tab-set::
+
+      .. tab-item:: MI355X and MI350X
+         :sync: MI355X and MI350X
+
+         .. code-block:: shell
+
+            bash runner/primus-cli direct \
+              --log_file /tmp/primus_qwen3_30B_A3B_fp8.log \
+              -- train pretrain \
+              --config examples/megatron/configs/MI355X/qwen3_30B_A3B-FP8-pretrain.yaml
+
+      .. tab-item:: MI300X
+         :sync: MI325X and MI300X
+
+         .. code-block:: shell
+
+            # Set the variables for better performance
+            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
+            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
+            export NVTE_CK_IS_V3_ATOMIC_FP32=1
+
+            bash runner/primus-cli direct \
+              --log_file /tmp/primus_qwen3_30B_A3B_fp8.log \
+              -- train pretrain \
+              --config examples/megatron/configs/MI300X/qwen3_30B_A3B-FP8-pretrain.yaml
+
+.. container:: model-doc primus_pyt_megatron_lm_train_gpt-oss-20b
+
+   Once setup is complete, run the appropriate training command.
+   The following run commands are tailored to GPT-OSS-20B.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.
+
+   To run training on a single node for GPT-OSS-20B BF16, use the following command:
+
+   .. tab-set::
+
+      .. tab-item:: MI355X and MI350X
+         :sync: MI355X and MI350X
+
+         .. code-block:: shell
+
+            bash runner/primus-cli direct \
+              --log_file /tmp/primus_gpt_oss_20B.log \
+              -- train pretrain \
+              --config examples/megatron/configs/MI355X/gpt_oss_20B-BF16-pretrain.yaml
+
+      .. tab-item:: MI300X
+         :sync: MI325X and MI300X
+
+         .. code-block:: shell
+
+            # Set the variables for better performance
+            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
+            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
+            export NVTE_CK_IS_V3_ATOMIC_FP32=1
+
+            bash runner/primus-cli direct \
+              --log_file /tmp/primus_gpt_oss_20B.log \
+              -- train pretrain \
+              --config examples/megatron/configs/MI300X/gpt_oss_20B-BF16-pretrain.yaml
+
+   For FP8, use the following command.
+
+   .. tab-set::
+
+      .. tab-item:: MI355X and MI350X
+         :sync: MI355X and MI350X
+
+         .. code-block:: shell
+
+            bash runner/primus-cli direct \
+              --log_file /tmp/primus_gpt_oss_20B_fp8.log \
+              -- train pretrain \
+              --config examples/megatron/configs/MI355X/gpt_oss_20B-FP8-pretrain.yaml
+
+      .. tab-item:: MI300X
+         :sync: MI325X and MI300X
+
+         .. code-block:: shell
+
+            # Set the variables for better performance
+            # only on MI325X and MI300X
+            export HSA_NO_SCRATCH_RECLAIM=1
+            export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
+            export NVTE_CK_IS_V3_ATOMIC_FP32=1
+
+            bash runner/primus-cli direct \
+              --log_file /tmp/primus_gpt_oss_20B_fp8.log \
+              -- train pretrain \
+              --config examples/megatron/configs/MI300X/gpt_oss_20B-FP8-pretrain.yaml
+
+.. _amd-primus-megatron-multi-node-examples-v26.3:

 Multi-node training examples
 ----------------------------
@@ -974,44 +1022,57 @@ Multi-node training examples
 Refer to :doc:`/how-to/rocm-for-ai/system-setup/multi-node-setup` to configure your environment for multi-node
 training.

-To run training on multiple nodes, you can use the
-`run_slurm_pretrain.sh <https://github.com/AMD-AGI/Primus/blob/9c529cd4a934a68a880ede036c3e97b792e38167/examples/run_slurm_pretrain.sh>`__
-to launch the multi-node workload. Use the following steps to setup your environment:
+To run training on multiple nodes, you can use ``primus-cli`` (recommended) or the
+`run_slurm_pretrain.sh <https://github.com/AMD-AGI/Primus/blob/main/examples/run_slurm_pretrain.sh>`__
+script to launch multi-node workloads. Use the following steps to set up your environment:

-.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/primus-megatron-benchmark-models.yaml
+.. important::

-   {% set docker = data.docker %}
-   .. code-block:: shell
+   **Verify NCCL / network environment first.** The ``primus-cli`` launcher sets sensible
+   ``NCCL_*`` defaults via ``base_env.sh``, but auto-detection can pick the wrong device
+   on multi-NIC nodes. Always confirm ``NCCL_IB_HCA``, ``NCCL_IB_GID_INDEX``,
+   ``NCCL_SOCKET_IFNAME``, and ``GLOO_SOCKET_IFNAME`` (set to the same value as
+   ``NCCL_SOCKET_IFNAME``) are correct for your fabric. If necessary, export these
+   environment variables before running.

-      git clone --recurse-submodules https://github.com/AMD-AGI/Primus.git
-      cd Primus/
-      git checkout 44f780d
-      git submodule update --init --recursive
-      export DOCKER_IMAGE={{ docker.pull_tag }}
-      export HF_TOKEN=<your_HF_token>
-      export NCCL_IB_HCA=<your_NCCL_IB_HCA> # specify which RDMA interfaces to use for communication
-      export NCCL_SOCKET_IFNAME=<your_NCCL_SOCKET_IFNAME> # your Network Interface
-      export GLOO_SOCKET_IFNAME=<your_GLOO_SOCKET_IFNAME> # your Network Interface
-      export NCCL_IB_GID_INDEX=3 # Set InfiniBand GID index for NCCL communication. Default is 3 for ROCE
+.. code-block:: shell

-      # Set the variables for better performance
-      # only on MI325X and MI300X
-      export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
-      export NVTE_CK_IS_V3_ATOMIC_FP32=1
+   git clone --recurse-submodules https://github.com/AMD-AGI/Primus.git
+   cd Primus/
+   git checkout release/v26.3
+   git submodule update --init --recursive
+   export DOCKER_IMAGE=rocm/primus:v26.3
+   export HF_TOKEN=<your_HF_token>
+   export NCCL_IB_HCA=<your_NCCL_IB_HCA> # specify which RDMA interfaces to use for communication
+   export NCCL_SOCKET_IFNAME=<your_NCCL_SOCKET_IFNAME> # your Network Interface
+   export GLOO_SOCKET_IFNAME=<your_GLOO_SOCKET_IFNAME> # your Network Interface
+   export NCCL_IB_GID_INDEX=3 # Set InfiniBand GID index for NCCL communication. Default is 3 for ROCE
+
+   # MI300/MI325X only -- for better performance
+   export HSA_NO_SCRATCH_RECLAIM=1
+   export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
+   export NVTE_CK_IS_V3_ATOMIC_FP32=1
+
+For clusters using AMD AINIC, also set the following:
+
+.. code-block:: shell
+
+   export USING_AINIC=1
+   export NCCL_PXN_DISABLE=0
+   export NCCL_IB_GID_INDEX=1

 .. note::

-   * Make sure correct network drivers are installed on the nodes. If inside a Docker, either install the drivers inside the Docker container or pass the network drivers from the host while creating Docker container.
-   * If ``NCCL_IB_HCA`` and ``NCCL_SOCKET_IFNAME`` are not set, Primus will try to auto-detect. However, since NICs can vary accross different cluster, it is encouraged to explicitly export your NCCL parameters for the cluster.
+   * Make sure correct network drivers are installed on the nodes. If inside a Docker, either install the drivers inside the Docker container or pass the network drivers from the host while creating the Docker container.
+   * If ``NCCL_IB_HCA`` and ``NCCL_SOCKET_IFNAME`` are not set, Primus will try to auto-detect. However, since NICs can vary across different clusters, it is encouraged to explicitly export your NCCL parameters for the cluster.
   * To find your network interface, you can use ``ip a``.
-   * To find RDMA interfaces, you can use ``ibv_devices`` to get the list of all the RDMA/IB  devices.
-   * Remember to set ``DOCKER_IMAGE`` and ``HF_TOKEN`` (see :ref:`amd-primus-megatron-lm-tokenizer-v26.2`) as appropriate.
+   * To find RDMA interfaces, you can use ``ibv_devices`` to get the list of all the RDMA/IB devices.

 .. container:: model-doc primus_pyt_megatron_lm_train_llama-3.1-8b

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Llama 3.1 8B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To train Llama 3.1 8B FP8 on 8 nodes, run:

@@ -1019,16 +1080,14 @@ to launch the multi-node workload. Use the following steps to setup your environ

      # Adjust the training parameters.
      # For example, `global_batch_size: 8 * #single_node_bs` for 8 nodes in this case.
-      NNODES=8 \
-      EXP=examples/megatron/configs/MI300X/llama3.1_8B-FP8-pretrain.yaml \
-      bash ./examples/run_slurm_pretrain.sh \
-          --global_batch_size 1024 \
+      NNODES=8 EXP=examples/megatron/configs/MI300X/llama3.1_8B-FP8-pretrain.yaml \
+      bash ./examples/run_slurm_pretrain.sh --global_batch_size 1024

 .. container:: model-doc primus_pyt_megatron_lm_train_llama-2-7b

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Llama 2 7B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To train Llama 2 7B FP8 on 8 nodes, run:

@@ -1036,139 +1095,161 @@ to launch the multi-node workload. Use the following steps to setup your environ

      # Adjust the training parameters.
      # For example, `global_batch_size: 8 * #single_node_bs` for 8 nodes in this case.
-      NNODES=8 \
-      EXP=examples/megatron/configs/MI300X/llama2_7B-FP8-pretrain.yaml \
-      bash ./examples/run_slurm_pretrain.sh \
-          --global_batch_size 2048 \
+      NNODES=8 EXP=examples/megatron/configs/MI300X/llama2_7B-FP8-pretrain.yaml \
+      bash ./examples/run_slurm_pretrain.sh --global_batch_size 2048

 .. container:: model-doc primus_pyt_megatron_lm_train_llama-3.1-70b

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Llama 3.1 70B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To train Llama 3.1 70B FP8 on 8 nodes, run:

   .. code-block:: shell

-      # Adjust the training parameters.
-      # For example, `global_batch_size: 8 * #single_node_bs` for 8 nodes in this case.
-      NNODES=8 \
-      EXP=examples/megatron/configs/MI300X/llama3.1_70B-FP8-pretrain.yaml \
+      NNODES=8 EXP=examples/megatron/configs/MI300X/llama3.1_70B-FP8-pretrain.yaml \
      bash examples/run_slurm_pretrain.sh \
-          --micro_batch_size 4 \
-          --global_batch_size 256 \
-          --recompute_num_layers 80 \
+          --micro_batch_size 4 --global_batch_size 256 --recompute_num_layers 80

   To train Llama 3.1 70B BF16 on 8 nodes, run:

   .. code-block:: shell

-      NNODES=8 \
-      EXP=examples/megatron/configs/MI300X/llama3.1_70B-BF16-pretrain.yaml \
+      NNODES=8 EXP=examples/megatron/configs/MI300X/llama3.1_70B-BF16-pretrain.yaml \
      bash examples/run_slurm_pretrain.sh \
-          --micro_batch_size 1 \
-          --global_batch_size 256 \
-          --recompute_num_layers 12
+          --micro_batch_size 1 --global_batch_size 256 --recompute_num_layers 12

 .. container:: model-doc primus_pyt_megatron_lm_train_llama-2-70b

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Llama 2 70B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To train Llama 2 70B FP8 on 8 nodes, run:

   .. code-block:: shell

-      # Adjust the training parameters.
-      # For example, `global_batch_size: 8 * #single_node_bs` for 8 nodes in this case.
-      NNODES=8 \
-      EXP=examples/megatron/configs/MI300X/llama2_70B-FP8-pretrain.yaml \
+      NNODES=8 EXP=examples/megatron/configs/MI300X/llama2_70B-FP8-pretrain.yaml \
      bash examples/run_slurm_pretrain.sh \
-          --micro_batch_size 10 \
-          --global_batch_size 640 \
-          --recompute_num_layers 80 \
+          --micro_batch_size 10 --global_batch_size 640 --recompute_num_layers 80

   To train Llama 2 70B BF16 on 8 nodes, run:

   .. code-block:: shell

-      NNODES=8 \
-      EXP=examples/megatron/configs/MI300X/llama2_70B-BF16-pretrain.yaml \
+      NNODES=8 EXP=examples/megatron/configs/MI300X/llama2_70B-BF16-pretrain.yaml \
      bash ./examples/run_slurm_pretrain.sh \
-          --micro_batch_size 2 \
-          --global_batch_size 1536 \
-          --recompute_num_layers 12
+          --micro_batch_size 2 --global_batch_size 1536 --recompute_num_layers 12

 .. container:: model-doc primus_pyt_megatron_lm_train_llama-3.3-70b

   Once setup is complete, run the appropriate training command.
   The following run commands are tailored to Llama 3.3 70B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To train Llama 3.3 70B FP8 on 8 nodes, run:

   .. code-block:: shell

-      # Adjust the training parameters.
-      # For example, `global_batch_size: 8 * #single_node_bs` for 8 nodes in this case
-      NNODES=8 \
-      EXP=examples/megatron/configs/MI300X/llama3.3_70B-FP8-pretrain.yaml \
+      NNODES=8 EXP=examples/megatron/configs/MI300X/llama3.3_70B-FP8-pretrain.yaml \
      bash examples/run_slurm_pretrain.sh \
-          --micro_batch_size 4 \
-          --global_batch_size 256 \
-          --recompute_num_layers 80 \
+          --micro_batch_size 4 --global_batch_size 256 --recompute_num_layers 80

   To train Llama 3.3 70B BF16 on 8 nodes, run:

   .. code-block:: shell

-      NNODES=8 \
-      EXP=examples/megatron/configs/MI300X/llama3.3_70B-BF16-pretrain.yaml \
+      NNODES=8 EXP=examples/megatron/configs/MI300X/llama3.3_70B-BF16-pretrain.yaml \
      bash examples/run_slurm_pretrain.sh \
-          --micro_batch_size 1 \
-          --global_batch_size 256 \
-          --recompute_num_layers 12
+          --micro_batch_size 1 --global_batch_size 256 --recompute_num_layers 12

 .. container:: model-doc primus_pyt_megatron_lm_train_mixtral-8x7b

   Once setup is complete, run the appropriate training command.
-   The following run commands are tailored to Llama 2 70B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   The following run commands are tailored to Mixtral 8x7B.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

   To train Mixtral 8x7B BF16 on 8 nodes, run:

   .. code-block:: shell

-      # Adjust the training parameters.
-      # For example, `global_batch_size: 8 * #single_node_bs` for 8 nodes in this case
-      NNODES=8 \
-      EXP=examples/megatron/configs/MI300X/mixtral_8x7B_v0.1-BF16-pretrain.yaml \
+      NNODES=8 EXP=examples/megatron/configs/MI300X/mixtral_8x7B_v0.1-BF16-pretrain.yaml \
      bash examples/run_slurm_pretrain.sh \
-          --micro_batch_size 2 \
-          --global_batch_size 256
+          --micro_batch_size 2 --global_batch_size 256
+
+To train Mixtral 8x22B BF16 on 4 nodes using ``primus-cli`` (recommended), run:
+
+.. code-block:: shell
+
+   # In the Primus directory
+   ./primus-cli slurm srun -N 4 -- train pretrain \
+       --config examples/megatron/configs/MI355X/mixtral_8x22B_v0.1-BF16-pretrain.yaml \
+       --micro_batch_size 1 \
+       --global_batch_size 512 \
+       --num_virtual_stages_per_pipeline_rank 2 \
+       --pipeline_model_parallel_size 4 \
+       --expert_model_parallel_size 8 \
+       --recompute_num_layers 1 \
+       --moe_use_legacy_grouped_gemm True \
+       --gradient_accumulation_fusion True
+
+Alternatively, using the legacy script:
+
+.. code-block:: shell
+
+   NNODES=4 EXP=examples/megatron/configs/MI355X/mixtral_8x22B_v0.1-BF16-pretrain.yaml \
+   bash examples/run_slurm_pretrain.sh \
+       --micro_batch_size 1 \
+       --global_batch_size 512 \
+       --num_virtual_stages_per_pipeline_rank 2 \
+       --pipeline_model_parallel_size 4 \
+       --expert_model_parallel_size 8 \
+       --recompute_num_layers 1 \
+       --moe_use_legacy_grouped_gemm True \
+       --gradient_accumulation_fusion True

 .. container:: model-doc primus_pyt_megatron_lm_train_qwen2.5-72b

   Once setup is complete, run the appropriate training command.
-   The following run commands are tailored to Llama 2 70B.
-   See :ref:`amd-primus-megatron-lm-model-support-v26.2` to switch to another available model.
+   The following run commands are tailored to Qwen 2.5 72B.
+   See :ref:`amd-primus-megatron-lm-model-support-v26.3` to switch to another available model.

-   To train Qwen2.5 72B FP8 on 8 nodes, run:
+   To train Qwen 2.5 72B FP8 on 8 nodes, run:

   .. code-block:: shell

-      # Adjust the training parameters.
-      # For example, `global_batch_size: 8 * #single_node_bs` for 8 nodes in this case
-      NNODES=8 \
-      EXP=examples/megatron/configs/qwen2.5_72B-FP8-pretrain.yaml \
+      NNODES=8 EXP=examples/megatron/configs/MI300X/qwen2.5_72B-FP8-pretrain.yaml \
      bash examples/run_slurm_pretrain.sh \
-          --micro_batch_size 8 \
-          --global_batch_size 512 \
-          --recompute_num_layers 80 \
+          --micro_batch_size 8 --global_batch_size 512 --recompute_num_layers 80

-.. _amd-primus-megatron-lm-benchmark-test-vars-v26.2:
+To train Llama 3.1 405B FP8 on 8 nodes using ``primus-cli`` (recommended), run:
+
+.. code-block:: shell
+
+   # In the Primus directory
+   # TP=8 is used for Llama 3.1 405B on 8 nodes. The model has 126 layers which is not
+   # divisible by 8, so decoder_first_pipeline_num_layers and
+   # decoder_last_pipeline_num_layers must be set explicitly.
+   ./primus-cli slurm srun -N 8 -- train pretrain \
+       --config examples/megatron/configs/MI325X/llama3.1_405B-FP8-pretrain.yaml \
+       --micro_batch_size 1 \
+       --global_batch_size 256 \
+       --decoder_first_pipeline_num_layers 15 \
+       --decoder_last_pipeline_num_layers 15
+
+Alternatively, using the legacy script:
+
+.. code-block:: shell
+
+   NNODES=8 EXP=examples/megatron/configs/MI300X/llama3.1_405B-FP8-pretrain.yaml \
+   bash examples/run_slurm_pretrain.sh \
+       --micro_batch_size 1 \
+       --global_batch_size 256 \
+       --decoder_first_pipeline_num_layers 15 \
+       --decoder_last_pipeline_num_layers 15
+
+.. _amd-primus-megatron-lm-benchmark-test-vars-v26.3:

 Key options
 -----------
@@ -45,7 +45,7 @@ with Primus Turbo optimizations.
              - {{ component_version }}
            {% endfor %}

-.. _amd-primus-pytorch-model-support-v26.2:
+.. _amd-primus-pytorch-model-support-v26.3:

 Supported models
 ================
@@ -91,7 +91,7 @@ vary by model -- select one to get started.
   For additional workloads, including Llama 3.3, Llama 3.2, Llama 2, GPT OSS, Qwen, and Flux models,
   see the documentation :doc:`pytorch-training` (without Primus)

-.. _amd-primus-pytorch-performance-measurements-v26.2:
+.. _amd-primus-pytorch-performance-measurements-v26.3:

 System validation
 =================
@@ -146,7 +146,7 @@ tweak some configurations (such as batch sizes).
         .. container:: model-doc {{ model.mad_tag }}

            The following run commands are tailored to {{ model.model }}.
-            See :ref:`amd-primus-pytorch-model-support-v26.2` to switch to another available model.
+            See :ref:`amd-primus-pytorch-model-support-v26.3` to switch to another available model.

            .. rubric:: Download the Docker image and required packages

@@ -182,10 +182,10 @@ tweak some configurations (such as batch sizes).
                  docker start training_env
                  docker exec -it training_env bash

-               The Docker container hosts verified commit ``9c529cd4`` of the `Primus
-               <https://github.com/AMD-AGI/Primus/tree/9c529cd4a934a68a880ede036c3e97b792e38167/>`__ repository.
+               The Docker container hosts verified commit ``43a6e00`` of the `Primus
+               <https://github.com/AMD-AGI/Primus/tree/43a6e006c419697208295c5523b99070e8198ad9>`__ repository.

-            .. rubric:: Prepare training datasets and dependencies
+            .. rubric:: Setup

            The following benchmarking examples require downloading models and datasets
            from Hugging Face. To ensure successful access to gated repos, set your
@@ -195,8 +195,6 @@ tweak some configurations (such as batch sizes).

               export HF_TOKEN=$your_personal_hugging_face_access_token

-            .. rubric:: Pretraining
-
            To get started, navigate to the ``Primus`` directory in your container.

            .. code-block::
@@ -206,7 +204,7 @@ tweak some configurations (such as batch sizes).
            Now, to start the pretraining benchmark, use the ``run_pretrain.sh`` script
            included with Primus with the appropriate options.

-            .. rubric:: Benchmarking examples
+            .. rubric:: Pretraining examples

            .. container:: model-doc primus_pyt_train_llama-3.1-8b

@@ -344,7 +342,7 @@ tweak some configurations (such as batch sizes).
         .. container:: model-doc {{ model.mad_tag }}

            The following run command is tailored to {{ model.model }}.
-            See :ref:`amd-primus-pytorch-model-support-v26.2` to switch to another available model.
+            See :ref:`amd-primus-pytorch-model-support-v26.3` to switch to another available model.

            1. Clone the ROCm Model Automation and Dashboarding (`<https://github.com/ROCm/MAD>`__) repository to a local
               directory and install the required packages on the host machine.
@@ -374,6 +372,136 @@ tweak some configurations (such as batch sizes).
      {% endfor %}
   {% endfor %}

+Multi-node training examples
+============================
+
+Refer to :doc:`/how-to/rocm-for-ai/system-setup/multi-node-setup` to configure your environment for multi-node
+training.
+
+To run training on multiple nodes, use ``primus-cli`` to launch multi-node workloads. Use the following steps to set up your environment:
+
+.. important::
+
+   **Verify NCCL / network environment first.** The ``primus-cli`` launcher sets sensible
+   ``NCCL_*`` defaults via ``base_env.sh``, but auto-detection can pick the wrong device
+   on multi-NIC nodes. Always confirm ``NCCL_IB_HCA``, ``NCCL_IB_GID_INDEX``,
+   ``NCCL_SOCKET_IFNAME``, and ``GLOO_SOCKET_IFNAME`` (set to the same value as
+   ``NCCL_SOCKET_IFNAME``) are correct for your fabric. If necessary, export these
+   environment variables before running.
+
+.. code-block:: shell
+
+   git clone --recurse-submodules https://github.com/AMD-AGI/Primus.git
+   cd Primus/
+   git checkout release/v26.3
+   git submodule update --init --recursive
+   export DOCKER_IMAGE=rocm/primus:v26.3
+   export HF_TOKEN=<your_HF_token>
+   export NCCL_IB_HCA=<your_NCCL_IB_HCA> # specify which RDMA interfaces to use for communication
+   export NCCL_SOCKET_IFNAME=<your_NCCL_SOCKET_IFNAME> # your Network Interface
+   export GLOO_SOCKET_IFNAME=<your_GLOO_SOCKET_IFNAME> # your Network Interface
+   export NCCL_IB_GID_INDEX=3 # Set InfiniBand GID index for NCCL communication. Default is 3 for ROCE
+
+   # MI300/MI325X only -- for better performance
+   export PRIMUS_TURBO_ATTN_V3_ATOMIC_FP32=1
+   export NVTE_CK_IS_V3_ATOMIC_FP32=1
+
+For clusters using AMD AINIC, also set the following:
+
+.. code-block:: shell
+
+   export USING_AINIC=1
+   export NCCL_PXN_DISABLE=0
+   export NCCL_IB_GID_INDEX=1
+
+.. note::
+
+   * Make sure correct network drivers are installed on the nodes. If inside a Docker, either install the drivers inside the Docker container or pass the network drivers from the host while creating the Docker container.
+   * If ``NCCL_IB_HCA`` and ``NCCL_SOCKET_IFNAME`` are not set, Primus will try to auto-detect. However, since NICs can vary across different clusters, it is encouraged to explicitly export your NCCL parameters for the cluster.
+   * To find your network interface, you can use ``ip a``.
+   * To find RDMA interfaces, you can use ``ibv_devices`` to get the list of all the RDMA/IB devices.
+
+.. container:: model-doc primus_pyt_train_llama-3.1-8b
+
+   Once setup is complete, run the appropriate training command.
+   The following run commands are tailored to Llama 3.1 8B.
+   See :ref:`amd-primus-pytorch-model-support-v26.3` to switch to another available model.
+
+   To train Llama 3.1 8B FP8 on 8 nodes, run:
+
+   .. code-block:: shell
+
+      ./primus-cli slurm srun -N 8 -- train pretrain \
+          --config examples/torchtitan/configs/MI300X/llama3.1_8B-FP8-pretrain.yaml
+
+   To train Llama 3.1 8B BF16 on 8 nodes, run:
+
+   .. code-block:: shell
+
+      ./primus-cli slurm srun -N 8 -- train pretrain \
+          --config examples/torchtitan/configs/MI300X/llama3.1_8B-BF16-pretrain.yaml
+
+.. container:: model-doc primus_pyt_train_llama-3.1-70b
+
+   Once setup is complete, run the appropriate training command.
+   The following run commands are tailored to Llama 3.1 70B.
+   See :ref:`amd-primus-pytorch-model-support-v26.3` to switch to another available model.
+
+   To train Llama 3.1 70B FP8 on 4 nodes using ``primus-cli`` (recommended), run:
+
+   .. code-block:: shell
+
+      # In the Primus directory
+      ./primus-cli slurm srun -N 4 -- train pretrain \
+          --config examples/torchtitan/configs/MI355X/llama3.1_70B-FP8-pretrain.yaml \
+          --training.local_batch_size 6 \
+          --training.global_batch_size 192 \
+          --training.mock_data True
+
+   Alternatively, using the legacy script:
+
+   .. code-block:: shell
+
+      NNODES=4 EXP=examples/torchtitan/configs/MI355X/llama3.1_70B-FP8-pretrain.yaml \
+      bash examples/run_slurm_pretrain.sh \
+          --training.local_batch_size 6 \
+          --training.global_batch_size 192 \
+          --training.mock_data True
+
+.. container:: model-doc primus_pyt_train_deepseek-v3-16b
+
+   Once setup is complete, run the appropriate training command.
+   The following run commands are tailored to DeepSeek V3 16B.
+   See :ref:`amd-primus-pytorch-model-support-v26.3` to switch to another available model.
+
+   To train DeepSeek V3 16B BF16 on 8 nodes, run:
+
+   .. code-block:: shell
+
+      ./primus-cli slurm srun -N 8 -- train pretrain \
+          --config examples/torchtitan/configs/MI300X/deepseek_v3_16b-pretrain.yaml
+
+To train Llama 3.1 405B FP8 on 8 nodes using ``primus-cli`` (recommended), run:
+
+.. code-block:: shell
+
+   # In the Primus directory
+   ./primus-cli slurm srun -N 8 -- train pretrain \
+       --config examples/torchtitan/configs/MI355X/llama3.1_405B-FP8-pretrain.yaml \
+       --training.local_batch_size 3 \
+       --training.global_batch_size 192 \
+       --training.mock_data True
+
+Alternatively, using the legacy script:
+
+.. code-block:: shell
+
+   NNODES=8 EXP=examples/torchtitan/configs/MI355X/llama3.1_405B-FP8-pretrain.yaml \
+   bash examples/run_slurm_pretrain.sh \
+       --training.local_batch_size 3 \
+       --training.global_batch_size 192 \
+       --training.mock_data True
+
 Further reading
 ===============

@@ -1,4 +1,5 @@
 :orphan:
+:no-search:

 .. meta::
   :description: How to train a model using PyTorch for ROCm.
@@ -63,14 +63,8 @@ subtrees:
        - entries:
          - file: how-to/rocm-for-ai/training/benchmark-docker/primus-megatron.rst
            title: Train a model with Primus and Megatron-LM
-            entries:
-            - file: how-to/rocm-for-ai/training/benchmark-docker/megatron-lm.rst
-              title: Train a model with Megatron-LM (legacy)
          - file: how-to/rocm-for-ai/training/benchmark-docker/primus-pytorch.rst
            title: Train a model with Primus and PyTorch
-            entries:
-            - file: how-to/rocm-for-ai/training/benchmark-docker/pytorch-training.rst
-              title: Train a model with PyTorch (legacy)
          - file: how-to/rocm-for-ai/training/benchmark-docker/jax-maxtext.rst
            title: Train a model with Primus and JAX MaxText
          - file: how-to/rocm-for-ai/training/benchmark-docker/mpt-llm-foundry
Author	SHA1	Message	Date
Pratik BasyalandGitHub	a723c34015	AMDGPU Driver version updated (#6311 ) (#6312 )	2026-05-28 17:59:06 -04:00
Peter Parkandpeterjunpark	c507d8c229	Update preview release link in README (cherry picked from commit `89321ae826`)	2026-05-25 09:26:49 -04:00
peterjunparkandpeterjunpark	7cd6916f45	docs: Primus 26.3 (#6300 ) * Archive previous version - 26.2 * Update Megatron data and template ver hist page fix gpt fix * Update torch data and template hist page clean up * Remove legacy (non-Primus) from TOC add :no-search: * Update Primus commit hashes * Update multi-node examples update * fix anchors (cherry picked from commit `3cd1d85020`)	2026-05-25 09:26:49 -04:00
peterjunparkandGitHub	7dfae7ea3d	[docs/7.2.3] Update docs for JAX MaxText training 26.3 (#6276 ) (#6278 ) * Update docs for JAX MaxText training 26.3 (#6276) * archive previous version * update for 26.3 * update col width * add profiling sections * update .wordlist.txt (cherry picked from commit `f5c11d16a5`) * Apply suggestion from @peterjunpark	2026-05-19 13:44:33 -04:00
peterjunpark GitHub Kristoffer Torp	daaea93bb1	Update xDiT diffusion inference page for 26.5 release (#6224 ) (#6241 ) * archive previous version * Update components version * Update run settings for a few models * Update hf download and pull tag * ROCm bump and what's new * Fix double (preview) (preview) text * update --------- (cherry picked from commit `0cc148b308`) Co-authored-by: Kristoffer Torp <kristoffer.torp@amd.com>	2026-05-12 14:56:45 -04:00
Pratik BasyalandGitHub	08b26c579d	Ubuntu support version updated (#6199 ) (#6200 )	2026-05-05 11:46:04 -04:00
Pratik BasyalandGitHub	202c42efaf	7.2.3 OS table updated (#6197 ) (#6198 ) * OS table updated * Minor change	2026-05-04 16:07:29 -04:00
Alex Xu	14f8138863	Merge branch 'develop' into docs/7.2.3	2026-05-04 14:38:42 -04:00
Pratik BasyalandGitHub	f5c2cbcf54	30.30.2 AMD GPU Driver added (#6177 ) (#6179 ) * 30.30.2 AMD GPU Driver added * Table formatting updated * Vertial space optimized * Table optimized	2026-04-23 12:07:02 -04:00
peterjunparkandpeterjunpark	3ecda28ba8	Suppress certain pages in RTD search results (#6174 ) * add `/previous-versions/` to RTD search ignore list * add `:nosearch:` to archived pages * rename :nosearch: to :no-search: (cherry picked from commit `8686138dea`)	2026-04-21 18:02:46 -04:00
peterjunparkandGitHub	36f9dc9d89	docs: update `huggingface-cli` cmds to `hf` equivalents (#6169 ) (#6171 ) (cherry picked from commit `f544433d88`)	2026-04-21 11:49:27 -04:00
Pratik BasyalandGitHub	b1fe357181	MI325X PLDM and 30.30.2 driver update (#6160 ) (#6161 )	2026-04-17 18:43:21 -04:00
Pratik BasyalandGitHub	31decb4afd	AMD Radeon and Ryzen reference note updated [Develop] (#6158 ) (#6159 ) * AMD Radeon and Ryzen reference note updated * Posession updated	2026-04-17 08:52:56 -04:00
Alex Xu	f7f1ec6ba1	gMerge branch 'roc-7.2.x' into docs/7.2.2	2026-04-14 15:27:04 -04:00
Alex Xu	6848f66b26	Merge branch 'roc-7.2.x' into docs/7.2.2	2026-04-14 15:17:09 -04:00
Istvan KissandGitHub	840f956558	RDNA 3.5 page update based on provided feedbacks. (#6134 )	2026-04-08 20:26:18 +02:00
Istvan KissandGitHub	f84c5e4004	Add the AMD ROCm Programming Guide link (#6128 )	2026-04-08 13:09:02 +02:00