Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Istvan Kiss
|
c93901117c |
@@ -1,112 +0,0 @@
|
||||
Precision support delta check — SHA-filtered. Arguments: $ARGUMENTS = "previous_version current_version" (e.g. 7.1.1 7.2.0).
|
||||
|
||||
Only checks libraries where the source file itself changed between the two releases (SHA comparison). Libraries with an unchanged source file are skipped automatically — no manifest diff is performed.
|
||||
|
||||
**Output rule:** Do not write any text until all tool calls are complete. When you are ready to respond, write exactly: "X of N libraries need verification." then the summary table, then detail blocks, then the "Changes written:" block. Nothing else. No narration, no transitions, no status updates, no reasoning.
|
||||
|
||||
## Step 1 — Fetch source files
|
||||
|
||||
```bash
|
||||
cd ~/projects/ROCm-internal/tools/autotag && python3 precision_fetch.py -t "$GITHUB_TOKEN" --previous PREVIOUS --current CURRENT --sha-filter
|
||||
```
|
||||
|
||||
Replace PREVIOUS and CURRENT from $ARGUMENTS.
|
||||
|
||||
## Step 2 — Read library list
|
||||
|
||||
Read `/tmp/precision_libs.txt`.
|
||||
|
||||
## Step 3 — Parse each library
|
||||
|
||||
For each library in the list:
|
||||
|
||||
- Source file exists (`/tmp/precision_{lib}_source.txt`): read it and `/tmp/precision_{lib}_yaml.txt`, then compare.
|
||||
- URL file exists (`/tmp/precision_{lib}_url.txt`): read it — this is the GitHub link to the source file, used in the detail block Source field.
|
||||
- Skip file exists (`/tmp/precision_{lib}_skip.txt`): note the reason.
|
||||
|
||||
**Do not use GitHub MCP tools, WebFetch, or any network call.** All content is already in `/tmp/`. Use only the Read tool on those files.
|
||||
|
||||
You are the parser. Read the source file directly — do not infer from filenames or prior knowledge.
|
||||
|
||||
- RST files: look for list-tables, type listings, and support level indicators.
|
||||
- C headers: look for enums and macros that define supported types.
|
||||
- AMD/NVIDIA split tables: use the AMD column only.
|
||||
|
||||
## Step 4 — Known mismatches (do not flag)
|
||||
|
||||
These entries are **specific** — they exempt only the named type for the named library. They do not mark the entire library as clean. Continue checking all other types in every library.
|
||||
|
||||
- **hipBLAS float16**: YAML=⚠️ correct — only AXPY and Dot support it.
|
||||
- **hipBLAS bfloat16**: YAML=⚠️ correct — only Dot supports it.
|
||||
- **rocBLAS float16**: same partial-support pattern as hipBLAS — do not flag.
|
||||
- **rocBLAS bfloat16**: same partial-support pattern as hipBLAS — do not flag.
|
||||
- **MIGraphX int8**: YAML=⚠️ correct — quantization-only via `quantize_int8()`.
|
||||
- **rocFFT float16**: YAML already includes float16 ✅ — do not flag as missing.
|
||||
|
||||
**Do not dismiss any other gaps on your own judgment.** If a type appears in the source but not in the YAML, flag it — even if you think it might be intentional or covered by convention. That call belongs to the human reviewer, not you.
|
||||
|
||||
## Step 5 — Classify and update (silent)
|
||||
|
||||
For each finding, decide **auto-update** or **flag**:
|
||||
|
||||
**Default: auto-update.** If a type is absent from YAML and appears in the source, add it — unless one of the following exceptions applies.
|
||||
|
||||
**Flag for human review if ANY are true:**
|
||||
|
||||
- Found only via macro expansion (e.g., `MIGRAPHX_SHAPE_VISIT_TYPES`).
|
||||
- Found only in a combination table (Ti/To/Tc triplets) — not a canonical per-type list.
|
||||
- Finding is a support level mismatch (✅ vs ⚠️) — the type is already in YAML but at the wrong level.
|
||||
- Source contains only typedef declarations with no support table or ✅ indicators of any kind.
|
||||
- Type is explicitly marked AMD ❌ in an AMD/NVIDIA split table.
|
||||
|
||||
For auto-update findings, add the missing types to `~/projects/ROCm-internal/docs/data/reference/precision-support/precision-support.yaml`. Match the existing format exactly (type + support fields).
|
||||
|
||||
Then write a log to `~/projects/ROCm-internal/tools/autotag/precision-update-log/PREVIOUS-CURRENT-YYYYMMDD-HHMMSS.md` (replace PREVIOUS/CURRENT from $ARGUMENTS, timestamp from `date +%Y%m%d-%H%M%S`). Always create a new file — never read or overwrite an existing log. Log format:
|
||||
|
||||
```markdown
|
||||
# Precision support audit: ROCm PREVIOUS → CURRENT
|
||||
|
||||
**Date:** YYYY-MM-DD
|
||||
|
||||
## Auto-updated
|
||||
(table of types added, or "None")
|
||||
|
||||
## Flagged for human review
|
||||
(table of flags)
|
||||
|
||||
## Clean / skipped
|
||||
(table)
|
||||
```
|
||||
|
||||
## Step 6 — Output
|
||||
|
||||
Write your response now. Start with: "X of N libraries need verification."
|
||||
|
||||
Then the summary table. **This MUST be a markdown table — no lists, no separators, no other format.** One row per library, all libraries included:
|
||||
|
||||
| Library | Finding | Action | Source |
|
||||
|---------|---------|--------|--------|
|
||||
| ... | ... | ... | filename.ext |
|
||||
|
||||
Use the URL from `/tmp/precision_{lib}_url.txt`. In the Source column, write only the filename (last path segment, plain text — no markdown link). For skipped libraries, leave Source blank.
|
||||
|
||||
Then, **for flagged libraries only**, a detail block:
|
||||
|
||||
<!-- markdownlint-disable MD036 -->
|
||||
**{Library}**
|
||||
<!-- markdownlint-enable MD036 -->
|
||||
|
||||
- Finding: ...
|
||||
- Details: ...
|
||||
- Source: [filename](url) — full clickable link using the URL from `/tmp/precision_{lib}_url.txt`.
|
||||
- Recommended action: ...
|
||||
|
||||
Clean and skipped libraries appear only in the table — no detail block.
|
||||
|
||||
End with:
|
||||
|
||||
```text
|
||||
Changes written:
|
||||
- {library}: {types added}
|
||||
(or "None" if no auto-updates)
|
||||
```
|
||||
@@ -1,110 +0,0 @@
|
||||
Precision support delta check. Arguments: $ARGUMENTS = "previous_version current_version" (e.g. 7.1.1 7.2.0).
|
||||
|
||||
**Output rule:** Do not write any text until all tool calls are complete. When you are ready to respond, write exactly: "X of N libraries need verification." then the summary table, then detail blocks, then the "Changes written:" block. Nothing else. No narration, no transitions, no status updates, no reasoning.
|
||||
|
||||
## Step 1 — Fetch source files
|
||||
|
||||
```bash
|
||||
cd ~/projects/ROCm-internal/tools/precision-support && python3 precision_fetch.py -t "$GITHUB_TOKEN" --previous PREVIOUS --current CURRENT
|
||||
```
|
||||
|
||||
Replace PREVIOUS and CURRENT from $ARGUMENTS.
|
||||
|
||||
## Step 2 — Read library list
|
||||
|
||||
Read `/tmp/precision_libs.txt`.
|
||||
|
||||
## Step 3 — Parse each library
|
||||
|
||||
For each library in the list:
|
||||
|
||||
- Source file exists (`/tmp/precision_{lib}_source.txt`): read it and `/tmp/precision_{lib}_yaml.txt`, then compare.
|
||||
- URL file exists (`/tmp/precision_{lib}_url.txt`): read it — this is the GitHub link to the source file, used in the detail block Source field.
|
||||
- Skip file exists (`/tmp/precision_{lib}_skip.txt`): note the reason.
|
||||
|
||||
**Do not use GitHub MCP tools, WebFetch, or any network call.** All content is already in `/tmp/`. Use only the Read tool on those files.
|
||||
|
||||
You are the parser. Read the source file directly — do not infer from filenames or prior knowledge.
|
||||
|
||||
- RST files: look for list-tables, type listings, and support level indicators.
|
||||
- C headers: look for enums and macros that define supported types.
|
||||
- AMD/NVIDIA split tables: use the AMD column only.
|
||||
|
||||
## Step 4 — Known mismatches (do not flag)
|
||||
|
||||
These entries are **specific** — they exempt only the named type for the named library. They do not mark the entire library as clean. Continue checking all other types in every library.
|
||||
|
||||
- **hipBLAS float16**: YAML=⚠️ correct — only AXPY and Dot support it.
|
||||
- **hipBLAS bfloat16**: YAML=⚠️ correct — only Dot supports it.
|
||||
- **rocBLAS float16**: same partial-support pattern as hipBLAS — do not flag.
|
||||
- **rocBLAS bfloat16**: same partial-support pattern as hipBLAS — do not flag.
|
||||
- **MIGraphX int8**: YAML=⚠️ correct — quantization-only via `quantize_int8()`.
|
||||
- **rocFFT float16**: YAML already includes float16 ✅ — do not flag as missing.
|
||||
|
||||
**Do not dismiss any other gaps on your own judgment.** If a type appears in the source but not in the YAML, flag it — even if you think it might be intentional or covered by convention. That call belongs to the human reviewer, not you.
|
||||
|
||||
## Step 5 — Classify and update (silent)
|
||||
|
||||
For each finding, decide **auto-update** or **flag**:
|
||||
|
||||
**Default: auto-update.** If a type is absent from YAML and appears in the source, add it — unless one of the following exceptions applies.
|
||||
|
||||
**Flag for human review if ANY are true:**
|
||||
|
||||
- Found only via macro expansion (e.g., `MIGRAPHX_SHAPE_VISIT_TYPES`).
|
||||
- Found only in a combination table (Ti/To/Tc triplets) — not a canonical per-type list.
|
||||
- Finding is a support level mismatch (✅ vs ⚠️) — the type is already in YAML but at the wrong level.
|
||||
- Source contains only typedef declarations with no support table or ✅ indicators of any kind.
|
||||
- Type is explicitly marked AMD ❌ in an AMD/NVIDIA split table.
|
||||
|
||||
For auto-update findings, add the missing types to `~/projects/ROCm-internal/docs/data/reference/precision-support/precision-support.yaml`. Match the existing format exactly (type + support fields).
|
||||
|
||||
Then write a log to `~/projects/ROCm-internal/tools/precision-support/precision-update-log/PREVIOUS-CURRENT-YYYYMMDD-HHMMSS.md` (replace PREVIOUS/CURRENT from $ARGUMENTS, timestamp from `date +%Y%m%d-%H%M%S`). Always create a new file — never read or overwrite an existing log. Log format:
|
||||
|
||||
```markdown
|
||||
# Precision support audit: ROCm PREVIOUS → CURRENT
|
||||
|
||||
**Date:** YYYY-MM-DD
|
||||
|
||||
## Auto-updated
|
||||
(table of types added, or "None")
|
||||
|
||||
## Flagged for human review
|
||||
(table of flags)
|
||||
|
||||
## Clean / skipped
|
||||
(table)
|
||||
```
|
||||
|
||||
## Step 6 — Output
|
||||
|
||||
Write your response now. Start with: "X of N libraries need verification."
|
||||
|
||||
Then the summary table. **This MUST be a markdown table — no lists, no separators, no other format.** One row per library, all libraries included:
|
||||
|
||||
| Library | Finding | Action | Source |
|
||||
|---------|---------|--------|--------|
|
||||
| ... | ... | ... | filename.ext |
|
||||
|
||||
Use the URL from `/tmp/precision_{lib}_url.txt`. In the Source column, write only the filename (last path segment, plain text — no markdown link). For skipped libraries, leave Source blank.
|
||||
|
||||
Then, **for flagged libraries only**, a detail block:
|
||||
|
||||
<!-- markdownlint-disable MD036 -->
|
||||
**{Library}**
|
||||
<!-- markdownlint-enable MD036 -->
|
||||
|
||||
- Finding: ...
|
||||
- Details: ...
|
||||
- Source: [filename](url) — full clickable link using the URL from `/tmp/precision_{lib}_url.txt`.
|
||||
- Recommended action: ...
|
||||
|
||||
Clean and skipped libraries appear only in the table — no detail block.
|
||||
|
||||
End with:
|
||||
|
||||
```text
|
||||
Changes written:
|
||||
- {library}: {types added}
|
||||
(or "None" if no auto-updates)
|
||||
```
|
||||
@@ -1,12 +0,0 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"Bash(cd ~/projects/ROCm-internal/tools/autotag && python3 precision_fetch.py*)",
|
||||
"Read(/tmp/precision_*)",
|
||||
"Edit(docs/data/reference/precision-support/precision-support.yaml)",
|
||||
"Write(tools/autotag/precision-update-log/*)",
|
||||
"Write(precision-update-log/*)",
|
||||
"Bash(date +%Y%m%d-%H%M%S)"
|
||||
]
|
||||
}
|
||||
}
|
||||
+4
-4
@@ -1,8 +1,8 @@
|
||||
* @ROCm/rocm-documentation
|
||||
* @amd-aakash @jlgreathouse @samjwu @yhuiYH @ROCm/rocm-documentation
|
||||
# Documentation files
|
||||
docs/ @ROCm/rocm-documentation
|
||||
*.md @ROCm/rocm-documentation
|
||||
*.rst @ROCm/rocm-documentation
|
||||
docs/ @amd-aakash @jlgreathouse @samjwu @yhuiYH @ROCm/rocm-documentation
|
||||
*.md @amd-aakash @jlgreathouse @samjwu @yhuiYH @ROCm/rocm-documentation
|
||||
*.rst @amd-aakash @jlgreathouse @samjwu @yhuiYH @ROCm/rocm-documentation
|
||||
# External CI
|
||||
/.azuredevops/ @ROCm/external-ci
|
||||
tools/rocm-build/ @ROCm/rocm-devops
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="1280" height="640" viewBox="0 0 1280 640" role="img" aria-label="ROCm">
|
||||
<rect width="1280" height="640" fill="#0A0A0A"/>
|
||||
<svg x="96" y="215" width="210" height="210" viewBox="0 0 67 67"><path d="M22.21 67V44.6369H0V67H22.21Z" fill="#fff"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z" fill="#fff"/><path d="M22.21 0H0V22.3184H22.21V0Z" fill="#fff"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z" fill="#fff"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z" fill="#fff"/></svg>
|
||||
<text x="378" y="276" font-family="Inter,system-ui,-apple-system,sans-serif" font-size="78" font-weight="800" letter-spacing="-2" fill="#ffffff">ROCm</text>
|
||||
<text x="378" y="322" font-family="Inter,system-ui,sans-serif" font-size="30" fill="#ffffff" opacity=".66">AMD ROCm™ Software - GitHub Home</text>
|
||||
<rect x="378" y="338" width="806" height="3" rx="1.5" fill="#ffffff" opacity=".9"/>
|
||||
<text x="378" y="390" font-family="Inter,system-ui,sans-serif" font-size="24" font-weight="600" fill="#ffffff" opacity=".5">github.com/hanzoai</text>
|
||||
<text x="1184" y="390" text-anchor="end" font-family="Inter,system-ui,sans-serif" font-size="24" font-weight="600" fill="#ffffff" opacity=".5">hanzo.ai</text>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 1.2 KiB |
@@ -19,4 +19,3 @@ __pycache__/
|
||||
docs/contribute/index.md
|
||||
docs/about/release-notes.md
|
||||
docs/release/changelog.md
|
||||
.claude/settings.local.json
|
||||
|
||||
@@ -20,7 +20,3 @@ build:
|
||||
- "doxygen"
|
||||
- "gfortran" # For pre-processing fortran sources
|
||||
- "graphviz" # For dot graphs in doxygen
|
||||
|
||||
search:
|
||||
ignore:
|
||||
- "**/previous-versions/**"
|
||||
|
||||
+127
-170
@@ -3,14 +3,11 @@ ABI
|
||||
ACE
|
||||
ACEs
|
||||
ACS
|
||||
AccVGPR
|
||||
AccVGPRs
|
||||
AITER
|
||||
ALU
|
||||
AllReduce
|
||||
AllToAll
|
||||
AGPR
|
||||
AGPRs
|
||||
AITER
|
||||
ALU
|
||||
AMD
|
||||
AMDGPU
|
||||
AMDGPUs
|
||||
@@ -25,35 +22,31 @@ APIC
|
||||
APIs
|
||||
APU
|
||||
APUs
|
||||
ASAN
|
||||
ASIC
|
||||
ASICs
|
||||
ASan
|
||||
ASAN
|
||||
ASm
|
||||
Async
|
||||
ATI
|
||||
AccVGPR
|
||||
AccVGPRs
|
||||
atomicRMW
|
||||
AddressSanitizer
|
||||
AlexNet
|
||||
AllReduce
|
||||
Andrej
|
||||
Arb
|
||||
Async
|
||||
Autocast
|
||||
autograd
|
||||
Backported
|
||||
BARs
|
||||
BatchNorm
|
||||
BKC
|
||||
BLAS
|
||||
BMC
|
||||
BNXT
|
||||
BabelStream
|
||||
Backported
|
||||
BatchNorm
|
||||
Bitwise
|
||||
Blit
|
||||
Blockwise
|
||||
Bluefield
|
||||
Bootloader
|
||||
BrainFloat
|
||||
Broadcom
|
||||
CAS
|
||||
CCD
|
||||
@@ -61,18 +54,19 @@ CDNA
|
||||
CGUI
|
||||
CHTML
|
||||
CIFAR
|
||||
CNP
|
||||
CLI
|
||||
CLion
|
||||
CMake
|
||||
CMakeLists
|
||||
CMakePackage
|
||||
CNP
|
||||
CP
|
||||
CPC
|
||||
CPF
|
||||
CPP
|
||||
CPU
|
||||
CPUs
|
||||
Cron
|
||||
CSC
|
||||
CSDATA
|
||||
CSE
|
||||
@@ -83,12 +77,13 @@ CTests
|
||||
CU
|
||||
CUDA
|
||||
CUs
|
||||
CX
|
||||
CXX
|
||||
CX
|
||||
Cavium
|
||||
CentOS
|
||||
ChatGPT
|
||||
Cholesky
|
||||
cholesky
|
||||
CoRR
|
||||
Codespaces
|
||||
Commitizen
|
||||
@@ -97,8 +92,12 @@ Concretized
|
||||
Conda
|
||||
ConnectX
|
||||
CountOnes
|
||||
Cron
|
||||
CuPy
|
||||
customizable
|
||||
da
|
||||
Dashboarding
|
||||
Dataloading
|
||||
dataflows
|
||||
DBRX
|
||||
DCQCN
|
||||
DDR
|
||||
@@ -106,27 +105,27 @@ DF
|
||||
DGEMM
|
||||
DGL
|
||||
DGLGraph
|
||||
dGPU
|
||||
dGPUs
|
||||
DIMM
|
||||
DKMS
|
||||
DL
|
||||
DMA
|
||||
DOMContentLoaded
|
||||
DNN
|
||||
DNNL
|
||||
DOCA
|
||||
DOMContentLoaded
|
||||
DPM
|
||||
DRI
|
||||
DSA
|
||||
DSCP
|
||||
DW
|
||||
DWORD
|
||||
Dashboarding
|
||||
Dask
|
||||
DataFrame
|
||||
DataLoader
|
||||
DataParallel
|
||||
Dataloading
|
||||
Debian
|
||||
decompositions
|
||||
DeepSeek
|
||||
DeepSpeed
|
||||
Dependabot
|
||||
@@ -134,33 +133,40 @@ Deprecations
|
||||
DevCap
|
||||
DirectX
|
||||
Disaggregated
|
||||
disagg
|
||||
disaggregated
|
||||
disaggregation
|
||||
Dockerfile
|
||||
Dockerized
|
||||
Doxygen
|
||||
dropless
|
||||
ELMo
|
||||
ENDPGM
|
||||
EP
|
||||
EPYC
|
||||
ESXi
|
||||
EP
|
||||
EoS
|
||||
etcd
|
||||
equalto
|
||||
fas
|
||||
FBGEMM
|
||||
FiLM
|
||||
FIFOs
|
||||
FFT
|
||||
FFTs
|
||||
FFmpeg
|
||||
FHS
|
||||
FIFOs
|
||||
FIXME
|
||||
FMA
|
||||
FNUZ
|
||||
FP
|
||||
FX
|
||||
FiLM
|
||||
Filesystem
|
||||
FindDb
|
||||
Flang
|
||||
FlashAttention
|
||||
FlashInfer
|
||||
FlashInfer’s
|
||||
FlashInfer
|
||||
flashinfer
|
||||
FluxBenchmark
|
||||
Fortran
|
||||
Fuyu
|
||||
@@ -180,17 +186,18 @@ GDS
|
||||
GEMM
|
||||
GEMMs
|
||||
GFLOPS
|
||||
GFXIP
|
||||
GFortran
|
||||
GFXIP
|
||||
GGUF
|
||||
GID
|
||||
Gbps
|
||||
Gemma
|
||||
GiB
|
||||
GIM
|
||||
GL
|
||||
Glibc
|
||||
GLM
|
||||
GIM
|
||||
GL
|
||||
GLXT
|
||||
Gloo
|
||||
GMI
|
||||
GNN
|
||||
GNNs
|
||||
@@ -202,30 +209,27 @@ GPU's
|
||||
GPUDirect
|
||||
GPUVM
|
||||
GPUs
|
||||
GraphBolt
|
||||
GraphSage
|
||||
GRBM
|
||||
GRE
|
||||
GTT
|
||||
Gbps
|
||||
Gemma
|
||||
GenAI
|
||||
GenZ
|
||||
GiB
|
||||
GitHub
|
||||
Gitpod
|
||||
Glibc
|
||||
Gloo
|
||||
Gluon
|
||||
GraphBolt
|
||||
GraphSage
|
||||
hardcoded
|
||||
HBM
|
||||
HCA
|
||||
HGX
|
||||
HLO
|
||||
HIPCC
|
||||
hipDataType
|
||||
HIPExtension
|
||||
HIPIFY
|
||||
HIPification
|
||||
hipification
|
||||
HIPify
|
||||
HLO
|
||||
HPC
|
||||
HPCG
|
||||
HPE
|
||||
@@ -236,24 +240,27 @@ HWE
|
||||
HWS
|
||||
Haswell
|
||||
Higgs
|
||||
href
|
||||
Hyperparameters
|
||||
HybridEngine
|
||||
Huggingface
|
||||
Hunyuan
|
||||
HunyuanVideo
|
||||
HybridEngine
|
||||
Hyperparameters
|
||||
IB
|
||||
ICD
|
||||
InternVL
|
||||
ICT
|
||||
ICV
|
||||
IDE
|
||||
IDEs
|
||||
IFWI
|
||||
IMDb
|
||||
IncDec
|
||||
instrSize
|
||||
interpolators
|
||||
IOMMU
|
||||
IOP
|
||||
IOPM
|
||||
IOPS
|
||||
IOPM
|
||||
IOV
|
||||
IPs
|
||||
IRQ
|
||||
@@ -262,7 +269,6 @@ ISV
|
||||
ISVs
|
||||
ITL
|
||||
ImageNet
|
||||
IncDec
|
||||
InfiniBand
|
||||
Inlines
|
||||
IntelliSense
|
||||
@@ -272,8 +278,9 @@ Intra
|
||||
Ioffe
|
||||
JAX's
|
||||
JAXLIB
|
||||
JSON
|
||||
Jinja
|
||||
js
|
||||
JSON
|
||||
Jupyter
|
||||
KFD
|
||||
KFDTest
|
||||
@@ -281,13 +288,10 @@ KMD
|
||||
KV
|
||||
KVM
|
||||
Karpathy's
|
||||
Kimi
|
||||
KiB
|
||||
Kineto
|
||||
Keras
|
||||
Khronos
|
||||
KiB
|
||||
Kineto
|
||||
LAPACK
|
||||
LCLK
|
||||
LDS
|
||||
@@ -295,27 +299,27 @@ LLM
|
||||
LLMs
|
||||
LLVM
|
||||
LM
|
||||
logsumexp
|
||||
LPDDR
|
||||
LRU
|
||||
LSE
|
||||
LSAN
|
||||
LSTMs
|
||||
LSan
|
||||
lstsq
|
||||
LTS
|
||||
LSTMs
|
||||
LteAll
|
||||
LanguageCrossEntropy
|
||||
LoRA
|
||||
LteAll
|
||||
MBT
|
||||
MECO
|
||||
MEM
|
||||
MERCHANTABILITY
|
||||
MFMA
|
||||
MiB
|
||||
MIGraphX
|
||||
MIOpen
|
||||
MIOpen's
|
||||
MIOpenGEMM
|
||||
MIOpen's
|
||||
MIVisionX
|
||||
MLA
|
||||
MLM
|
||||
MMA
|
||||
MMIO
|
||||
@@ -325,36 +329,36 @@ MNIST
|
||||
MPI
|
||||
MPT
|
||||
MSVC
|
||||
MTP
|
||||
MTU
|
||||
mul
|
||||
MVAPICH
|
||||
MVFFR
|
||||
MXFP
|
||||
Makefile
|
||||
Makefiles
|
||||
Matplotlib
|
||||
Matrox
|
||||
MaxText
|
||||
MBT
|
||||
Megablocks
|
||||
Megatrends
|
||||
Megatron
|
||||
Mellanox
|
||||
Mellanox's
|
||||
Meta's
|
||||
MiB
|
||||
Miniconda
|
||||
MirroredStrategy
|
||||
Mixtral
|
||||
MoEs
|
||||
MoRI
|
||||
Mooncake
|
||||
MLA
|
||||
MosaicML
|
||||
MoEs
|
||||
Mooncake
|
||||
MoRI
|
||||
Mpops
|
||||
Multicore
|
||||
Multimodal
|
||||
multimodal
|
||||
multihost
|
||||
Multithreaded
|
||||
mx
|
||||
MXFP
|
||||
MyEnvironment
|
||||
MyST
|
||||
NANOO
|
||||
@@ -362,28 +366,24 @@ NBIO
|
||||
NBIOs
|
||||
NCCL
|
||||
NCF
|
||||
NCS
|
||||
NFS
|
||||
NIC
|
||||
NICs
|
||||
NLI
|
||||
NLP
|
||||
NN
|
||||
NOP
|
||||
NPKit
|
||||
NPS
|
||||
NSP
|
||||
NUMA
|
||||
NVCC
|
||||
NVIDIA
|
||||
NVLink
|
||||
NVPTX
|
||||
NaN
|
||||
NaNs
|
||||
Nano
|
||||
Navi
|
||||
NoReturn
|
||||
Noncoherently
|
||||
NoReturn
|
||||
NousResearch's
|
||||
NumPy
|
||||
OAM
|
||||
@@ -394,13 +394,12 @@ OFED
|
||||
OMM
|
||||
OMP
|
||||
OMPI
|
||||
OOM
|
||||
OMPT
|
||||
OMPX
|
||||
ONNX
|
||||
OSL
|
||||
OSS
|
||||
OSU
|
||||
OOM
|
||||
OpenCL
|
||||
OpenCV
|
||||
OpenFabrics
|
||||
@@ -412,10 +411,13 @@ OpenVX
|
||||
OpenXLA
|
||||
Optim
|
||||
Oversubscription
|
||||
PagedAttention
|
||||
Pallas
|
||||
PCC
|
||||
PCI
|
||||
PCIe
|
||||
PEFT
|
||||
perf
|
||||
PEQT
|
||||
PIL
|
||||
PILImage
|
||||
@@ -428,30 +430,27 @@ PSID
|
||||
PTPC
|
||||
PaLM
|
||||
Pageable
|
||||
PagedAttention
|
||||
Pallas
|
||||
PeerDirect
|
||||
Pensando
|
||||
PerfDb
|
||||
Perfetto
|
||||
PipelineParallel
|
||||
Pipelining
|
||||
PnP
|
||||
Pollara
|
||||
PowerEdge
|
||||
PowerShell
|
||||
Preshuffled
|
||||
Pretrained
|
||||
Pretraining
|
||||
Primus
|
||||
Profiler's
|
||||
PyPi
|
||||
PyTorch
|
||||
Pytest
|
||||
PyTorch
|
||||
QPS
|
||||
Qcycles
|
||||
QoS
|
||||
Qwen
|
||||
Radix
|
||||
RAII
|
||||
RAS
|
||||
RCCL
|
||||
@@ -460,16 +459,16 @@ RDC's
|
||||
RDMA
|
||||
RDNA
|
||||
README
|
||||
Recomputation
|
||||
RHEL
|
||||
RLHF
|
||||
RMW
|
||||
RNN
|
||||
RNNs
|
||||
ROC
|
||||
ROCProfiler
|
||||
ROCT
|
||||
ROCTracer
|
||||
ROCTx
|
||||
ROCTracer
|
||||
ROCclr
|
||||
ROCdbgapi
|
||||
ROCgdb
|
||||
@@ -484,20 +483,17 @@ RPP
|
||||
RST
|
||||
RW
|
||||
Radeon
|
||||
Radix
|
||||
Recomputation
|
||||
Redfish
|
||||
RelWithDebInfo
|
||||
Req
|
||||
ResNet
|
||||
Rickle
|
||||
RoCE
|
||||
Runfile
|
||||
Ryzen
|
||||
SALU
|
||||
safetensors
|
||||
SBIOS
|
||||
SCA
|
||||
ScaledGEMM
|
||||
SDK
|
||||
SDMA
|
||||
SDPA
|
||||
@@ -515,6 +511,7 @@ SKU
|
||||
SKUs
|
||||
SLES
|
||||
SLURM
|
||||
Slurm
|
||||
SMEM
|
||||
SMFMA
|
||||
SMI
|
||||
@@ -525,19 +522,18 @@ SRAM
|
||||
SRAMECC
|
||||
SVD
|
||||
SWE
|
||||
ScaledGEMM
|
||||
SerDes
|
||||
Shardy
|
||||
ShareGPT
|
||||
Shlens
|
||||
simd
|
||||
Skylake
|
||||
Slurm
|
||||
Softmax
|
||||
Spack
|
||||
SplitK
|
||||
Strix
|
||||
Supermicro
|
||||
Szegedy
|
||||
TagRAM
|
||||
TCA
|
||||
TCC
|
||||
TCCs
|
||||
@@ -545,32 +541,35 @@ TCI
|
||||
TCIU
|
||||
TCP
|
||||
TCR
|
||||
TTM
|
||||
TVM
|
||||
THREADGROUPS
|
||||
threadgroups
|
||||
TensorRT
|
||||
TensorFloat
|
||||
TF
|
||||
TFLOPS
|
||||
THREADGROUPS
|
||||
TP
|
||||
TPS
|
||||
TPU
|
||||
TPUs
|
||||
TSME
|
||||
TTM
|
||||
TVM
|
||||
TagRAM
|
||||
Tagram
|
||||
TensileLite
|
||||
TensorBoard
|
||||
TensorFloat
|
||||
TensorFlow
|
||||
TensorParallel
|
||||
TensorRT
|
||||
TheRock
|
||||
ToC
|
||||
TopK
|
||||
TorchAudio
|
||||
torchaudio
|
||||
TorchElastic
|
||||
TorchMIGraphX
|
||||
torchrec
|
||||
TorchScript
|
||||
TorchServe
|
||||
torchserve
|
||||
torchtext
|
||||
TorchVision
|
||||
TransferBench
|
||||
TrapStatus
|
||||
@@ -578,32 +577,30 @@ UAC
|
||||
UC
|
||||
UCC
|
||||
UCX
|
||||
ud
|
||||
UE
|
||||
UI
|
||||
UIF
|
||||
UMC
|
||||
USM
|
||||
USM
|
||||
UTCL
|
||||
UTCL
|
||||
UTIL
|
||||
UTIL
|
||||
UX
|
||||
UltraChat
|
||||
Uncached
|
||||
Unhandled
|
||||
Unittests
|
||||
Unhandled
|
||||
unwindowed
|
||||
VALU
|
||||
VBIOS
|
||||
VCN
|
||||
verl's
|
||||
VGPR
|
||||
VGPRs
|
||||
VM
|
||||
VMEM
|
||||
VMID
|
||||
VMIDs
|
||||
VMWare
|
||||
VMs
|
||||
VMEM
|
||||
VMWare
|
||||
VRAM
|
||||
VSIX
|
||||
VSkipped
|
||||
@@ -617,6 +614,7 @@ WikiText
|
||||
Wojna
|
||||
Workgroups
|
||||
Writebacks
|
||||
xcc
|
||||
XCD
|
||||
XCDs
|
||||
XGBoost
|
||||
@@ -636,8 +634,8 @@ ZeRO
|
||||
ZenDNN
|
||||
accuracies
|
||||
activations
|
||||
addEventListener
|
||||
addr
|
||||
addEventListener
|
||||
ade
|
||||
ai
|
||||
alloc
|
||||
@@ -646,15 +644,11 @@ allocator
|
||||
allocators
|
||||
amdgpu
|
||||
api
|
||||
async
|
||||
aten
|
||||
atmi
|
||||
atomicRMW
|
||||
atomics
|
||||
autogenerated
|
||||
autograd
|
||||
autotune
|
||||
autotuning
|
||||
avx
|
||||
awk
|
||||
az
|
||||
@@ -668,17 +662,17 @@ bilinear
|
||||
bitcode
|
||||
bitsandbytes
|
||||
bitwise
|
||||
Bitwise
|
||||
blit
|
||||
bootloader
|
||||
boson
|
||||
bosons
|
||||
bottlenecked
|
||||
br
|
||||
BrainFloat
|
||||
btn
|
||||
buildable
|
||||
bursty
|
||||
bzip
|
||||
cTDP
|
||||
cacheable
|
||||
carveout
|
||||
cd
|
||||
@@ -687,7 +681,6 @@ centric
|
||||
changelog
|
||||
checkpointing
|
||||
chiplet
|
||||
cholesky
|
||||
classList
|
||||
cmake
|
||||
cmd
|
||||
@@ -711,20 +704,16 @@ copyable
|
||||
cpp
|
||||
csn
|
||||
cuBLAS
|
||||
cuda
|
||||
cuDNN
|
||||
cudnn
|
||||
cuFFT
|
||||
cuLIB
|
||||
cuRAND
|
||||
cuSOLVER
|
||||
cuSPARSE
|
||||
cuda
|
||||
cudnn
|
||||
customizable
|
||||
customizations
|
||||
dGPU
|
||||
dGPUs
|
||||
da
|
||||
dataflows
|
||||
cTDP
|
||||
dataset
|
||||
datasets
|
||||
dataspace
|
||||
@@ -734,9 +723,8 @@ datatypes
|
||||
dbgapi
|
||||
de
|
||||
deallocation
|
||||
debian
|
||||
debuggability
|
||||
decompositions
|
||||
debian
|
||||
deepseek
|
||||
denoise
|
||||
denoised
|
||||
@@ -752,14 +740,10 @@ devicelibs
|
||||
devsel
|
||||
dgl
|
||||
dimensionality
|
||||
disagg
|
||||
disaggregated
|
||||
disaggregation
|
||||
disambiguates
|
||||
distro
|
||||
distros
|
||||
dkms
|
||||
dropless
|
||||
dtype
|
||||
eb
|
||||
el
|
||||
@@ -772,19 +756,14 @@ endpgm
|
||||
enqueue
|
||||
env
|
||||
epilog
|
||||
equalto
|
||||
etcd
|
||||
etcetera
|
||||
eth
|
||||
ethernet
|
||||
exascale
|
||||
executables
|
||||
fas
|
||||
ffmpeg
|
||||
filesystem
|
||||
flashinfer
|
||||
forEach
|
||||
foreach
|
||||
fortran
|
||||
fp
|
||||
framebuffer
|
||||
@@ -803,15 +782,13 @@ gnupg
|
||||
gpu
|
||||
grayscale
|
||||
gx
|
||||
gz
|
||||
gzip
|
||||
hardcoded
|
||||
heterogenous
|
||||
hipBLAS
|
||||
hipBLASLt
|
||||
hipBLASLt's
|
||||
hipblaslt
|
||||
hipCUB
|
||||
hipDataType
|
||||
hipFFT
|
||||
hipFORT
|
||||
hipLIB
|
||||
@@ -822,12 +799,10 @@ hipSPARSELt
|
||||
hipTensor
|
||||
hipamd
|
||||
hipblas
|
||||
hipblaslt
|
||||
hipcc
|
||||
hipcub
|
||||
hipfft
|
||||
hipfort
|
||||
hipification
|
||||
hipify
|
||||
hipsolver
|
||||
hipsparse
|
||||
@@ -836,7 +811,6 @@ hostname
|
||||
hotspotting
|
||||
hpc
|
||||
hpp
|
||||
href
|
||||
hsa
|
||||
hsakmt
|
||||
hyperparameter
|
||||
@@ -852,9 +826,7 @@ init
|
||||
initializer
|
||||
inlining
|
||||
installable
|
||||
instrSize
|
||||
interop
|
||||
interpolators
|
||||
interprocedural
|
||||
intra
|
||||
intrinsics
|
||||
@@ -862,7 +834,6 @@ invariants
|
||||
invocating
|
||||
ipo
|
||||
jax
|
||||
js
|
||||
json
|
||||
kdb
|
||||
kfd
|
||||
@@ -882,10 +853,8 @@ llvm
|
||||
lm
|
||||
localscratch
|
||||
logits
|
||||
logsumexp
|
||||
loopback
|
||||
lossy
|
||||
lstsq
|
||||
macOS
|
||||
matchers
|
||||
maxtext
|
||||
@@ -902,25 +871,25 @@ mjx
|
||||
mkdir
|
||||
mlirmiopen
|
||||
mtypes
|
||||
mul
|
||||
multihost
|
||||
mutex
|
||||
mvffr
|
||||
mx
|
||||
namespace
|
||||
namespaces
|
||||
nanoGPT
|
||||
NCS
|
||||
NOP
|
||||
NVLink
|
||||
netplan
|
||||
num
|
||||
numref
|
||||
ocl
|
||||
ol
|
||||
openai
|
||||
opencl
|
||||
opencv
|
||||
openmp
|
||||
openssl
|
||||
optimizers
|
||||
ol
|
||||
os
|
||||
oversubscription
|
||||
pageable
|
||||
@@ -928,18 +897,14 @@ pallas
|
||||
parallelization
|
||||
parallelizing
|
||||
param
|
||||
parameterization
|
||||
params
|
||||
parameterization
|
||||
passthrough
|
||||
pb
|
||||
pe
|
||||
perf
|
||||
perfcounter
|
||||
performant
|
||||
perl
|
||||
piecewise
|
||||
pipelined
|
||||
pipelining
|
||||
perl
|
||||
pragma
|
||||
pre
|
||||
prebuild
|
||||
@@ -969,6 +934,8 @@ pseudorandom
|
||||
px
|
||||
py
|
||||
pytorch
|
||||
recommender
|
||||
recommenders
|
||||
quantile
|
||||
quantizer
|
||||
quasirandom
|
||||
@@ -981,10 +948,8 @@ rc
|
||||
rccl
|
||||
rdc
|
||||
rdma
|
||||
reStructuredText
|
||||
reachability
|
||||
recommender
|
||||
recommenders
|
||||
reStructuredText
|
||||
redirections
|
||||
refactorization
|
||||
reformats
|
||||
@@ -998,6 +963,7 @@ rescaling
|
||||
reusability
|
||||
rhel
|
||||
rl
|
||||
RLHF
|
||||
roadmap
|
||||
roc
|
||||
rocAL
|
||||
@@ -1024,8 +990,8 @@ rocm
|
||||
rocminfo
|
||||
rocprim
|
||||
rocprof
|
||||
rocprofiler
|
||||
rocprofv
|
||||
rocprofiler
|
||||
rocr
|
||||
rocrand
|
||||
rocsolver
|
||||
@@ -1035,6 +1001,7 @@ roctracer
|
||||
rst
|
||||
runtime
|
||||
runtimes
|
||||
ResNet
|
||||
sL
|
||||
scalability
|
||||
scalable
|
||||
@@ -1050,7 +1017,6 @@ sglang
|
||||
shader
|
||||
sharding
|
||||
sigmoid
|
||||
simd
|
||||
sles
|
||||
slurm
|
||||
sm
|
||||
@@ -1071,7 +1037,6 @@ submatrix
|
||||
submodule
|
||||
submodules
|
||||
subnet
|
||||
subnets
|
||||
supercomputing
|
||||
symlink
|
||||
symlinks
|
||||
@@ -1082,7 +1047,6 @@ td
|
||||
tensorfloat
|
||||
tf
|
||||
th
|
||||
threadgroups
|
||||
tokenization
|
||||
tokenize
|
||||
tokenized
|
||||
@@ -1090,39 +1054,36 @@ tokenizer
|
||||
tokenizes
|
||||
toolchain
|
||||
toolchains
|
||||
topk
|
||||
toolset
|
||||
toolsets
|
||||
topk
|
||||
torchaudio
|
||||
torchrec
|
||||
torchserve
|
||||
torchtext
|
||||
torchtitan
|
||||
torchvision
|
||||
tp
|
||||
tqdm
|
||||
tracebacks
|
||||
txt
|
||||
TopK
|
||||
uarch
|
||||
ud
|
||||
uncacheable
|
||||
uncached
|
||||
uncacheable
|
||||
uncorrectable
|
||||
underoptimized
|
||||
unfused
|
||||
unhandled
|
||||
unfused
|
||||
uninstallation
|
||||
unmapped
|
||||
unpadded
|
||||
unsqueeze
|
||||
unstacking
|
||||
unswitching
|
||||
unswizzled
|
||||
untrusted
|
||||
untuned
|
||||
unwindowed
|
||||
upvote
|
||||
USM
|
||||
UTCL
|
||||
UTIL
|
||||
utils
|
||||
UX
|
||||
vL
|
||||
variational
|
||||
vdi
|
||||
@@ -1133,7 +1094,6 @@ vectorized
|
||||
vectorizer
|
||||
vectorizes
|
||||
verl
|
||||
verl's
|
||||
virtualize
|
||||
virtualized
|
||||
vjxb
|
||||
@@ -1141,7 +1101,6 @@ vllm
|
||||
voxel
|
||||
walkthrough
|
||||
walkthroughs
|
||||
warmup
|
||||
watchpoints
|
||||
wavefront
|
||||
wavefronts
|
||||
@@ -1153,13 +1112,11 @@ writeback
|
||||
writebacks
|
||||
wrreq
|
||||
wzo
|
||||
xargs
|
||||
xdit
|
||||
xDiT
|
||||
xGMI
|
||||
xPacked
|
||||
xargs
|
||||
xcc
|
||||
xdit
|
||||
xplane
|
||||
xz
|
||||
yaml
|
||||
ysvmadyb
|
||||
|
||||
@@ -4,44 +4,6 @@ This page is a historical overview of changes made to ROCm components. This
|
||||
consolidated changelog documents key modifications and improvements across
|
||||
different versions of the ROCm software stack and its components.
|
||||
|
||||
## ROCm 7.2.3
|
||||
|
||||
See the [ROCm 7.2.3 release notes](https://rocm.docs.amd.com/en/docs-7.2.3/about/release-notes.html#rocm-7-2-3-release-notes)
|
||||
for a complete overview of this release.
|
||||
|
||||
### **MIGraphX** (2.15.0)
|
||||
|
||||
#### Added
|
||||
|
||||
* External stream support to the MIGraphX context, allowing external HIP streams to be used during execution.
|
||||
* Ability to return a vector for output alias, supporting operators like `make_tuple`.
|
||||
|
||||
#### Changed
|
||||
|
||||
* Refactored `move_output_instructions_after` into the module class.
|
||||
* Updated rocMLIR to fix `bert_squad` and `bert_tf` regressions.
|
||||
|
||||
#### Optimized
|
||||
|
||||
* Rewrote the `gather` operator to use `transpose`/`reshape`/`broadcast`/`slice` for improved performance.
|
||||
* Horizontally fuse cross-embedding `gather` operators.
|
||||
* Improved tuning for Split-K.
|
||||
* Removed extra assignments and inserts in `find_nop_reshapes` to reduce overhead.
|
||||
|
||||
#### Resolved issues
|
||||
|
||||
The following issues have been fixed:
|
||||
|
||||
* `int` to `bf16`/`fp16` conversion errors.
|
||||
* Comparison logic in `find_concat_op` to match the correct I/O.
|
||||
* `shape_transform_descriptor::rebase` when flattening a broadcasted dimension.
|
||||
* An error with `rewrite_reshapes`.
|
||||
* A gather rewrite crash by validating strided view element count.
|
||||
* A bug in gather rewrite with NHWC shapes.
|
||||
* A crash in rocMLIR with Inception v3 on RDNA3 architecture-based Radeon GPUs.
|
||||
* Filter zero-argument operators during ONNX parsing to prevent errors.
|
||||
* Conflict for missing `no_broadcast` parameter on ROCm 7.2.x.
|
||||
|
||||
## ROCm 7.2.1
|
||||
|
||||
See the [ROCm 7.2.1 release notes](https://rocm.docs.amd.com/en/docs-7.2.1/about/release-notes.html#rocm-7-2-1-release-notes)
|
||||
|
||||
@@ -1,18 +0,0 @@
|
||||
Hanzo ROCm
|
||||
Copyright (c) 2026 Hanzo AI, Inc.
|
||||
|
||||
This product includes software from AMD ROCm (https://github.com/ROCm/ROCm),
|
||||
licensed under the MIT License:
|
||||
|
||||
Copyright (c) 2023 - 2025 Advanced Micro Devices, Inc. All rights reserved.
|
||||
|
||||
This repository is the AMD ROCm meta/manifest repository: documentation, build
|
||||
tooling, and the repo manifest (default.xml). This meta-repository is MIT-licensed
|
||||
and its upstream MIT license is retained in LICENSE.
|
||||
|
||||
The individual ROCm components referenced by the manifest and built by the tooling
|
||||
carry their own upstream licenses, including the MIT License, the Apache License 2.0,
|
||||
and the University of Illinois/NCSA Open Source License. Note that the ROCgdb
|
||||
component (AMD's fork of GNU GDB, packaged by tools/rocm-build/build_rocm-gdb.sh) is
|
||||
licensed under the GNU General Public License (GPL) — a copyleft license. Each
|
||||
component is governed by its own license; consult that component's repository.
|
||||
@@ -1,5 +1,3 @@
|
||||
<p align="center"><img src=".github/hero.svg" alt="ROCm" width="880"></p>
|
||||
|
||||
<div align="center">
|
||||
<img src="docs/data/amd-rocm-logo.png" width="200px" alt="ROCm logo">
|
||||
|
||||
@@ -138,7 +136,7 @@ For a complete list of ROCm components and version information, see the
|
||||
## Release notes
|
||||
|
||||
- [Latest version of ROCm](https://rocm.docs.amd.com/en/latest/about/release-notes.html) - production
|
||||
- [ROCm 7.13.0](https://rocm.docs.amd.com/en/7.13.0-preview/about/release-notes.html) – preview stream
|
||||
- [ROCm 7.12.0](https://rocm.docs.amd.com/en/7.12.0-preview/about/release-notes.html) – preview stream
|
||||
|
||||
---
|
||||
|
||||
|
||||
+254
-279
@@ -10,159 +10,11 @@
|
||||
<!-- markdownlint-disable reference-links-images -->
|
||||
<!-- markdownlint-disable no-missing-space-atx -->
|
||||
<!-- spellcheck-disable -->
|
||||
|
||||
# ROCm 7.2.4 release notes
|
||||
|
||||
ROCm 7.2.4 is a quality release focused on performance and stability fixes for AI inference workloads on AMD Instinct GPUs.
|
||||
|
||||
## Release highlights
|
||||
|
||||
The following are the notable changes in ROCm 7.2.4.
|
||||
|
||||
### Reduced hipGraphLaunch latency for multi-list graphs
|
||||
|
||||
The HIP runtime's graph dispatch mechanism has been optimized, reducing launch latency for workloads using `hipGraphLaunch` with multi-list graph topologies.
|
||||
|
||||
### Fixed H2D memory copy latency regression in CPX mode
|
||||
|
||||
HIP runtime synchronization behavior has been corrected on AMD Instinct MI300 Series GPUs in CPX mode, restoring latency to previous levels for inference workloads that run multiple HIP streams with concurrent memory copies.
|
||||
|
||||
### Reduced ROCprofiler-SDK profiling overhead
|
||||
|
||||
Profiling stability has been improved for vLLM workloads traced with PyTorch `torch.profiler` using the ROCprofiler-SDK backend. The large, sporadic idle gaps that previously appeared between GPU kernels in the trace have been substantially reduced in common configurations, and the traces now more accurately reflect actual runtime behavior. Coverage may vary depending on model and parallelism settings.
|
||||
|
||||
### Reduced copy overhead in MIGraphX concat operations
|
||||
|
||||
MIGraphX now recognizes ONNX models that concatenate the same tensor multiple times and avoids redundant device-side copies, improving inference throughput at small batch sizes for the affected model class on AMD Instinct MI300X GPUs.
|
||||
|
||||
### User space, driver, and firmware dependent changes
|
||||
|
||||
The software for AMD Data Center GPU products requires maintaining a hardware
|
||||
and software stack with interdependencies among the GPU and baseboard
|
||||
firmware, AMD GPU drivers, and the ROCm user space software. While AMD publishes drivers and ROCm user space components, your server or infrastructure provider publishes the GPU and baseboard firmware by bundling AMD’s firmware releases via the AMD Platform Level Data Model (PLDM) bundle, which includes the Integrated Firmware Image (IFWI).
|
||||
|
||||
GPU and baseboard firmware versioning might differ across GPU families.
|
||||
|
||||
<div class="pst-scrollable-table-container">
|
||||
<table class="table table--middle-left">
|
||||
<thead>
|
||||
<tr>
|
||||
<th class="head">
|
||||
<p>ROCm Version</p>
|
||||
</th>
|
||||
<th class="head">
|
||||
<p>GPU</p>
|
||||
</th>
|
||||
<th class="head">
|
||||
<p>PLDM Bundle (Firmware)</p>
|
||||
</th>
|
||||
<th class="head">
|
||||
<p>AMD GPU Driver (amdgpu)</p>
|
||||
</th>
|
||||
<th class="head">
|
||||
<p>AMD GPU <br>
|
||||
Virtualization Driver (GIM)</p>
|
||||
</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<style>
|
||||
tbody#virtualization-support-instinct tr:last-child {
|
||||
border-bottom: 2px solid var(--pst-color-primary);
|
||||
}
|
||||
</style>
|
||||
<tr>
|
||||
<td rowspan="9" style="vertical-align: middle;">ROCm 7.2.4</td>
|
||||
<td>MI355X</td>
|
||||
<td>
|
||||
01.26.00.02<br>
|
||||
01.25.17.07<br>
|
||||
01.25.16.03
|
||||
</td>
|
||||
<td>
|
||||
30.30.x where x (0-4)<br>
|
||||
30.20.x where x (0-1)<br>
|
||||
30.10.x where x (0-2)
|
||||
</td>
|
||||
<td rowspan="3" style="vertical-align: middle;">8.7.1.K</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MI350X</td>
|
||||
<td>
|
||||
01.26.00.02<br>
|
||||
01.25.17.07<br>
|
||||
01.25.16.03
|
||||
</td>
|
||||
<td>
|
||||
30.30.x where x (0-4)<br>
|
||||
30.20.x where x (0-1)<br>
|
||||
30.10.x where x (0-2)
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MI325X<a href="#footnote1"><sup>[1]</sup></a></td>
|
||||
<td>
|
||||
01.25.06.08<br>
|
||||
01.25.04.02
|
||||
</td>
|
||||
<td>30.30.x where x (0-4)<br>
|
||||
30.20.x where x (0-1)<a href="#footnote1"><sup>[1]</sup></a><br>
|
||||
30.10.x where x (0-2)<br>
|
||||
6.4.z where z (0-3)<br>
|
||||
6.3.3
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MI300X<a href="#footnote2"><sup>[2]</sup></a></td>
|
||||
<td>01.25.06.04<br>
|
||||
01.25.03.12<br>
|
||||
01.25.02.04</td>
|
||||
<td rowspan="6" style="vertical-align: middle;">
|
||||
30.30.x where x (0-4)<br>
|
||||
30.20.x where x (0-1)<br>
|
||||
30.10.x where x (0-2)<br>
|
||||
6.4.z where z (0–3)<br>
|
||||
6.3.3
|
||||
</td>
|
||||
<td>8.7.1.K</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MI300A</td>
|
||||
<td>BKC 26.1</td>
|
||||
<td rowspan="3" style="vertical-align: middle;">Not Applicable</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MI250X</td>
|
||||
<td>IFWI 47 (or later)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MI250</td>
|
||||
<td>MU5 w/ IFWI 75 (or later)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MI210</td>
|
||||
<td>MU5 w/ IFWI 75 (or later)</td>
|
||||
<td>8.7.1.K</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MI100</td>
|
||||
<td>VBIOS D3430401-037</td>
|
||||
<td>Not Applicable</td>
|
||||
</tr>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<p id="footnote1">[1]: For AMD Instinct MI325X KVM SR-IOV users, don't use AMD GPU driver (amdgpu) 30.20.0.</p>
|
||||
<p id="footnote2">[2]: AMD Instinct MI300X KVM SR-IOV with Multi-VF (8 VF) support requires a compatible firmware BKC bundle, which will be released in the coming months.</p>
|
||||
|
||||
```{note}
|
||||
ROCm 7.2.4 doesn't include any other significant changes or feature additions. For comprehensive changes, new features, and enhancements in ROCm 7.2.3, refer to the [ROCm 7.2.3 release notes](#rocm-7-2-3-release-notes) below.
|
||||
```
|
||||
|
||||
## ROCm 7.2.3 release notes
|
||||
# ROCm 7.2.1 release notes
|
||||
|
||||
The release notes provide a summary of notable changes since the previous ROCm release.
|
||||
|
||||
- [Release highlights](#id1)
|
||||
- [Release highlights](#release-highlights)
|
||||
|
||||
- [Supported hardware, operating system, and virtualization changes](#supported-hardware-operating-system-and-virtualization-changes)
|
||||
|
||||
@@ -174,34 +26,41 @@ The release notes provide a summary of notable changes since the previous ROCm r
|
||||
|
||||
- [ROCm known issues](#rocm-known-issues)
|
||||
|
||||
- [ROCm resolved issues](#rocm-resolved-issues)
|
||||
|
||||
- [ROCm upcoming changes](#rocm-upcoming-changes)
|
||||
|
||||
```{note}
|
||||
If you’re using AMD Radeon™ GPUs or Ryzen™ for graphics workloads, see the [Use ROCm on Radeon and Ryzen](https://rocm.docs.amd.com/projects/radeon-ryzen/en/latest/index.html) documentation to verify compatibility and system requirements.
|
||||
If you’re using AMD Radeon GPUs or Ryzen APUs in a workstation setting with a display connected, see the [Use ROCm on Radeon and Ryzen](https://rocm.docs.amd.com/projects/radeon-ryzen/en/latest/index.html)
|
||||
documentation to verify compatibility and system requirements.
|
||||
```
|
||||
|
||||
### Release highlights
|
||||
## Release highlights
|
||||
|
||||
The following are notable new features and improvements in ROCm 7.2.3. For changes to individual components, see
|
||||
The following are notable new features and improvements in ROCm 7.2.1. For changes to individual components, see
|
||||
[Detailed component changes](#detailed-component-changes).
|
||||
|
||||
#### Supported hardware, operating system, and virtualization changes
|
||||
### Supported hardware, operating system, and virtualization changes
|
||||
|
||||
Hardware, operating system, and virtualization support remains unchanged in this release.
|
||||
Hardware support remains unchanged in this release.
|
||||
|
||||
ROCm 7.2.1 adds support for Ubuntu 24.04.4 (kernel: 6.8 [GA], 6.17 [HWE]) and marks end of support (EoS) for Ubuntu 24.04.3. For more information, see [Ubuntu installation](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.1/install/install-methods/package-manager/package-manager-ubuntu.html).
|
||||
|
||||
For more information about:
|
||||
|
||||
* AMD hardware, see [Supported GPUs (Linux)](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.3/reference/system-requirements.html#supported-gpus).
|
||||
* AMD hardware, see [Supported GPUs (Linux)](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.1/reference/system-requirements.html#supported-gpus).
|
||||
|
||||
* Operating systems, see [Supported operating systems](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.3/reference/system-requirements.html#supported-operating-systems) and [ROCm installation for Linux](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.3/).
|
||||
* Operating systems, see [Supported operating systems](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.1/reference/system-requirements.html#supported-operating-systems) and [ROCm installation for Linux](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.1/).
|
||||
|
||||
* Virtualization support, see [Virtualization support](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.3/reference/system-requirements.html#virtualization-support).
|
||||
#### Virtualization support
|
||||
|
||||
#### User space, driver, and firmware dependent changes
|
||||
Virtualization support remains unchanged in this release. For more information, see [Virtualization support](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.1/reference/system-requirements.html#virtualization-support).
|
||||
|
||||
### User space, driver, and firmware dependent changes
|
||||
|
||||
The software for AMD Data Center GPU products requires maintaining a hardware
|
||||
and software stack with interdependencies among the GPU and baseboard
|
||||
firmware, AMD GPU drivers, and the ROCm user space software. While AMD publishes drivers and ROCm user space components, your server or infrastructure provider publishes the GPU and baseboard firmware by bundling AMD’s firmware releases via the AMD Platform Level Data Model (PLDM) bundle, which includes the Integrated Firmware Image (IFWI).
|
||||
firmware, AMD GPU drivers, and the ROCm user space software. While AMD publishes drivers and ROCm user space components, your server or infrastructure provider publishes the GPU and baseboard firmware by bundling AMD’s firmware releases via AMD’s Platform Level Data Model (PLDM) bundle, which includes the Integrated Firmware Image (IFWI).
|
||||
|
||||
GPU and baseboard firmware versioning might differ across GPU families.
|
||||
|
||||
@@ -233,7 +92,7 @@ GPU and baseboard firmware versioning might differ across GPU families.
|
||||
}
|
||||
</style>
|
||||
<tr>
|
||||
<td rowspan="9" style="vertical-align: middle;">ROCm 7.2.3</td>
|
||||
<td rowspan="9" style="vertical-align: middle;">ROCm 7.2.1</td>
|
||||
<td>MI355X</td>
|
||||
<td>
|
||||
01.26.00.02<br>
|
||||
@@ -241,9 +100,13 @@ GPU and baseboard firmware versioning might differ across GPU families.
|
||||
01.25.16.03
|
||||
</td>
|
||||
<td>
|
||||
30.30.x where x (0-3)<br>
|
||||
30.20.x where x (0-1)<br>
|
||||
30.10.x where x (0-2)
|
||||
30.30.1<br>
|
||||
30.30.0<br>
|
||||
30.20.1<br>
|
||||
30.20.0<br>
|
||||
30.10.2<br>
|
||||
30.10.1<br>
|
||||
30.10
|
||||
</td>
|
||||
<td rowspan="3" style="vertical-align: middle;">8.7.1.K</td>
|
||||
</tr>
|
||||
@@ -255,20 +118,27 @@ GPU and baseboard firmware versioning might differ across GPU families.
|
||||
01.25.16.03
|
||||
</td>
|
||||
<td>
|
||||
30.30.x where x (0-3)<br>
|
||||
30.20.x where x (0-1)<br>
|
||||
30.10.x where x (0-2)
|
||||
30.30.1<br>
|
||||
30.30.0<br>
|
||||
30.20.1<br>
|
||||
30.20.0<br>
|
||||
30.10.2<br>
|
||||
30.10.1<br>
|
||||
30.10
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MI325X<a href="#footnote1"><sup>[1]</sup></a></td>
|
||||
<td>
|
||||
01.25.06.08<br>
|
||||
01.25.04.02
|
||||
</td>
|
||||
<td>30.30.x where x (0-3)<br>
|
||||
30.20.x where x (0-1)<a href="#footnote1"><sup>[1]</sup></a><br>
|
||||
30.10.x where x (0-2)<br>
|
||||
<td>30.30.1<br>
|
||||
30.30.0<br>
|
||||
30.20.1<br>
|
||||
30.20.0<a href="#footnote1"><sup>[1]</sup></a><br>
|
||||
30.10.2<br>
|
||||
30.10.1<br>
|
||||
30.10<br>
|
||||
6.4.z where z (0-3)<br>
|
||||
6.3.3
|
||||
</td>
|
||||
@@ -279,9 +149,13 @@ GPU and baseboard firmware versioning might differ across GPU families.
|
||||
01.25.03.12<br>
|
||||
01.25.02.04</td>
|
||||
<td rowspan="6" style="vertical-align: middle;">
|
||||
30.30.x where x (0-3)<br>
|
||||
30.20.x where x (0-1)<br>
|
||||
30.10.x where x (0-2)<br>
|
||||
30.30.1<br>
|
||||
30.30.0<br>
|
||||
30.20.1<br>
|
||||
30.20.0<br>
|
||||
30.10.2<br>
|
||||
30.10.1<br>
|
||||
30.10<br>
|
||||
6.4.z where z (0–3)<br>
|
||||
6.3.3
|
||||
</td>
|
||||
@@ -314,33 +188,53 @@ GPU and baseboard firmware versioning might differ across GPU families.
|
||||
</div>
|
||||
|
||||
<p id="footnote1">[1]: For AMD Instinct MI325X KVM SR-IOV users, don't use AMD GPU driver (amdgpu) 30.20.0.</p>
|
||||
<p id="footnote2">[2]: AMD Instinct MI300X KVM SR-IOV with Multi-VF (8 VF) support requires a compatible firmware BKC bundle, which will be released in the coming months.</p>
|
||||
<p id="footnote2">[2]: For AMD Instinct MI300X KVM SR-IOV with Multi-VF (8 VF) support requires a compatible firmware BKC bundle which will be released in coming months.</p>
|
||||
|
||||
#### Improved profiling accuracy for vLLM workloads
|
||||
### hipBLASLt updates
|
||||
|
||||
ROCm 7.2.3 improves profiling stability for vLLM workloads traced with PyTorch `torch.profiler`. The large, sporadic idle gaps that previously appeared between GPU kernels in the trace have been substantially reduced in common configurations, and the traces now more accurately reflect actual runtime behavior. Coverage may vary depending on model and parallelism settings; additional improvements are in progress.
|
||||
hipBLASLt has improved performance for MXFP8 and MXFP4 GEMMs.
|
||||
|
||||
#### MIGraphX update
|
||||
### Deep learning and AI framework updates
|
||||
|
||||
[MIGraphX](https://rocm.docs.amd.com/projects/AMDMIGraphX/en/docs-7.2.3/index.html) has the following enhancements:
|
||||
ROCm provides a comprehensive ecosystem for deep learning development. For more information, see [Deep learning frameworks for ROCm](../../docs/how-to/deep-learning-rocm.rst) and the [Compatibility
|
||||
matrix](../../docs/compatibility/compatibility-matrix.rst) for the complete list of Deep learning and AI framework versions tested for compatibility with ROCm. AMD ROCm has officially updated support for the following Deep learning and AI frameworks:
|
||||
|
||||
##### Improved performance of the Gather operator
|
||||
#### JAX
|
||||
|
||||
Performance for embedding‑heavy inference workloads is improved by merging multiple independent gather operations from similar embedding tables into a single batched operation. Multi‑gather workloads now run more efficiently with fewer kernel launches and reduced memory traffic by adding horizontal fusion for cross-embedding gather operators. These gather operators have been updated to use `transpose`/`reshape`/`broadcast`/`slice`, enabling better optimization across different backends and data layouts.
|
||||
ROCm 7.2.1 enables support for JAX 0.8.2. For more information, see [JAX compatibility](../../docs/compatibility/ml-compatibility/jax-compatibility.rst).
|
||||
|
||||
##### ONNX Runtime reliability improvement
|
||||
#### ROCm Offline Installer Creator discontinuation
|
||||
|
||||
ONNX Runtime workloads accelerated with MIGraphX now provide a more reliable experience through external stream support in the MIGraphX Execution Provider, with improved memory allocation and deallocation for multi-stream inference.
|
||||
The ROCm Offline Installer Creator is discontinued in ROCm 7.2.1. Equivalent installation capabilities are available through the ROCm Runfile Installer, a self-extracting installer that is not based on OS package managers. For more information, see [ROCm Runfile Installer](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.1/install/rocm-runfile-installer.html).
|
||||
|
||||
#### ROCm documentation updates
|
||||
### ROCm documentation updates
|
||||
|
||||
ROCm documentation has been updated with ROCm XIO documentation. ROCm XIO provides an API for Accelerator-Initiated IO (XIO) for an AMD GPU `__device__` code. It enables AMD GPUs to perform direct IO operations to hardware devices without CPU intervention. ROCm XIO was initially released in April 2026 as an early-access software technology preview. Running production workloads is not recommended.
|
||||
For more information, see the [ROCm XIO documentation](https://rocm.docs.amd.com/projects/rocm-xio/en/beta-0.1.0/index.html) and {fab}`github` [ROCm/rocm-xio](https://github.com/ROCm/rocm-xio) GitHub repository.
|
||||
ROCm documentation continues to be updated to provide clearer and more comprehensive guidance for a wider range of user needs and use cases.
|
||||
|
||||
### ROCm components
|
||||
* [Tutorials for AI developers](https://rocm.docs.amd.com/projects/ai-developer-hub/en/latest/) have been expanded with the following two new tutorials:
|
||||
* Pretraining tutorial: [SE(3)-Transformer overview](https://rocm.docs.amd.com/projects/ai-developer-hub/en/latest/notebooks/pretrain/se3transform_intro.html)
|
||||
* Fine-tuning tutorial: [GRPO with slime](https://rocm.docs.amd.com/projects/ai-developer-hub/en/latest/notebooks/fine_tune/slime_qwen3_4B_GRPO.html)
|
||||
|
||||
The following table lists the versions of ROCm components for ROCm 7.2.3, including any version
|
||||
changes from 7.2.2/7.2.1 to 7.2.3. Click the component's updated version to go to a list of its changes.
|
||||
For more information about the changes, see the [Changelog for the AI Developer Hub](https://rocm.docs.amd.com/projects/ai-developer-hub/en/latest/changelog.html).
|
||||
|
||||
* HIP documentation has been expanded with additional context and in-depth explanations across several core topics in the Programming Guide section. The following topics have been significantly enhanced:
|
||||
|
||||
* [Compilers](https://rocm.docs.amd.com/projects/HIP/en/docs-7.2.1/understand/compilers.html)
|
||||
* [Programming model](https://rocm.docs.amd.com/projects/HIP/en/docs-7.2.1/understand/programming_model.html)
|
||||
* [Performance guidelines](https://rocm.docs.amd.com/projects/HIP/en/docs-7.2.1/how-to/performance_guidelines.html)
|
||||
* [Performance optimization](https://rocm.docs.amd.com/projects/HIP/en/docs-7.2.1/understand/performance_optimization.html)
|
||||
|
||||
* [ROCm glossary](https://rocm.docs.amd.com/en/docs-7.2.1/reference/glossary.html) to provide concise definitions of AMD ROCm key terms and concepts has been added. The glossary is organized into:
|
||||
|
||||
* [Device hardware glossary](https://rocm.docs.amd.com/en/docs-7.2.1/reference/glossary/device-hardware.html): Provides brief definitions of hardware components and architectural features of AMD GPUs.
|
||||
* [Device software glossary](https://rocm.docs.amd.com/en/docs-7.2.1/reference/glossary/device-software.html): Provides brief definitions of software abstractions and programming models that run on AMD GPUs.
|
||||
* [Host software glossary](https://rocm.docs.amd.com/en/docs-7.2.1/reference/glossary/host-software.html): Provides brief definitions of development tools, compilers, libraries, and runtime environments for programming AMD GPUs.
|
||||
* [Performance glossary](https://rocm.docs.amd.com/en/docs-7.2.1/reference/glossary/performance.html): Provides brief definitions of performance analysis concepts and optimization techniques.
|
||||
|
||||
## ROCm components
|
||||
|
||||
The following table lists the versions of ROCm components for ROCm 7.2.1, including any version
|
||||
changes from 7.2.0 to 7.2.1. Click the component's updated version to go to a list of its changes.
|
||||
|
||||
Click {fab}`github` to go to the component's source code on GitHub.
|
||||
|
||||
@@ -363,48 +257,48 @@ Click {fab}`github` to go to the component's source code on GitHub.
|
||||
<tr>
|
||||
<th rowspan="9">Libraries</th>
|
||||
<th rowspan="9">Machine learning and computer vision</th>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/composable_kernel/en/docs-7.2.3/index.html">Composable Kernel</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/composable_kernel/en/docs-7.2.1/index.html">Composable Kernel</a></td>
|
||||
<td>1.2.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/composablekernel"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/AMDMIGraphX/en/docs-7.2.3/index.html">MIGraphX</a></td>
|
||||
<td>2.15.0 ⇒ <a href="#migraphx-2-15-0">2.15.0</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/AMDMIGraphX/en/docs-7.2.1/index.html">MIGraphX</a></td>
|
||||
<td>2.15.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/AMDMIGraphX"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/MIOpen/en/docs-7.2.3/index.html">MIOpen</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/MIOpen/en/docs-7.2.1/index.html">MIOpen</a></td>
|
||||
<td>3.5.1</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/miopen"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/MIVisionX/en/docs-7.2.3/index.html">MIVisionX</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/MIVisionX/en/docs-7.2.1/index.html">MIVisionX</a></td>
|
||||
<td>3.5.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/MIVisionX"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocAL/en/docs-7.2.3/index.html">rocAL</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocAL/en/docs-7.2.1/index.html">rocAL</a></td>
|
||||
<td>2.5.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocAL"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocDecode/en/docs-7.2.3/index.html">rocDecode</a></td>
|
||||
<td>1.7.0</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocDecode/en/docs-7.2.1/index.html">rocDecode</a></td>
|
||||
<td>1.5.0 ⇒ <a href="#rocdecode-1-7-0">1.7.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocDecode"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocJPEG/en/docs-7.2.3/index.html">rocJPEG</a></td>
|
||||
<td>1.4.0</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocJPEG/en/docs-7.2.1/index.html">rocJPEG</a></td>
|
||||
<td>1.3.0 ⇒ <a href="#rocjpeg-1-4-0">1.4.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocJPEG"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocPyDecode/en/docs-7.2.3/index.html">rocPyDecode</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocPyDecode/en/docs-7.2.1/index.html">rocPyDecode</a></td>
|
||||
<td>0.8.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocPyDecode"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rpp/en/docs-7.2.3/index.html">RPP</a></td>
|
||||
<td>2.2.1</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rpp/en/docs-7.2.1/index.html">RPP</a></td>
|
||||
<td>2.2.0 ⇒ <a href="#rpp-2-2-1">2.2.1</a></td>
|
||||
<td><a href="https://github.com/ROCm/rpp"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
@@ -412,13 +306,13 @@ Click {fab}`github` to go to the component's source code on GitHub.
|
||||
<tr>
|
||||
<th rowspan="2"></th>
|
||||
<th rowspan="2">Communication</th>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rccl/en/docs-7.2.3/index.html">RCCL</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rccl/en/docs-7.2.1/index.html">RCCL</a></td>
|
||||
<td>2.27.7</a></td>
|
||||
<td><a href="https://github.com/ROCm/rccl"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocSHMEM/en/docs-7.1.0/index.html">rocSHMEM</a></td>
|
||||
<td>3.2.0</a></td>
|
||||
<td>3.2.0 ⇒ <a href="#rocshmem-3-2-0">3.2.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocSHMEM"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
@@ -426,82 +320,82 @@ Click {fab}`github` to go to the component's source code on GitHub.
|
||||
<tr>
|
||||
<th rowspan="16"></th>
|
||||
<th rowspan="16">Math</th>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipBLAS/en/docs-7.2.3/index.html">hipBLAS</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipBLAS/en/docs-7.2.1/index.html">hipBLAS</a></td>
|
||||
<td>3.2.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipblas"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipBLASLt/en/docs-7.2.3/index.html">hipBLASLt</a></td>
|
||||
<td>1.2.2</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipBLASLt/en/docs-7.2.1/index.html">hipBLASLt</a></td>
|
||||
<td>1.2.1 ⇒ <a href="#hipblaslt-1-2-2">1.2.2</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipblaslt"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipFFT/en/docs-7.2.3/index.html">hipFFT</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipFFT/en/docs-7.2.1/index.html">hipFFT</a></td>
|
||||
<td>1.0.22</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipfft"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipfort/en/docs-7.2.3/index.html">hipfort</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipfort/en/docs-7.2.1/index.html">hipfort</a></td>
|
||||
<td>0.7.1</a></td>
|
||||
<td><a href="https://github.com/ROCm/hipfort"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipRAND/en/docs-7.2.3/index.html">hipRAND</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipRAND/en/docs-7.2.1/index.html">hipRAND</a></td>
|
||||
<td>3.1.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/hiprand"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipSOLVER/en/docs-7.2.3/index.html">hipSOLVER</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipSOLVER/en/docs-7.2.1/index.html">hipSOLVER</a></td>
|
||||
<td>3.2.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipsolver"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipSPARSE/en/docs-7.2.3/index.html">hipSPARSE</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipSPARSE/en/docs-7.2.1/index.html">hipSPARSE</a></td>
|
||||
<td>4.2.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipsparse"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipSPARSELt/en/docs-7.2.3/index.html">hipSPARSELt</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipSPARSELt/en/docs-7.2.1/index.html">hipSPARSELt</a></td>
|
||||
<td>0.2.6</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipsparselt"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocALUTION/en/docs-7.2.3/index.html">rocALUTION</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocALUTION/en/docs-7.2.1/index.html">rocALUTION</a></td>
|
||||
<td>4.1.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocALUTION"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocBLAS/en/docs-7.2.3/index.html">rocBLAS</a></td>
|
||||
<td>5.2.0</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocBLAS/en/docs-7.2.1/index.html">rocBLAS</a></td>
|
||||
<td>5.2.0</a></td></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/rocblas"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocFFT/en/docs-7.2.3/index.html">rocFFT</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocFFT/en/docs-7.2.1/index.html">rocFFT</a></td>
|
||||
<td>1.0.36</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/rocfft"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocRAND/en/docs-7.2.3/index.html">rocRAND</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocRAND/en/docs-7.2.1/index.html">rocRAND</a></td>
|
||||
<td>4.2.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/rocrand"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocSOLVER/en/docs-7.2.3/index.html">rocSOLVER</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocSOLVER/en/docs-7.2.1/index.html">rocSOLVER</a></td>
|
||||
<td>3.32.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/rocsolver"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocSPARSE/en/docs-7.2.3/index.html">rocSPARSE</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocSPARSE/en/docs-7.2.1/index.html">rocSPARSE</a></td>
|
||||
<td>4.2.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/rocsparse"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocWMMA/en/docs-7.2.3/index.html">rocWMMA</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocWMMA/en/docs-7.2.1/index.html">rocWMMA</a></td>
|
||||
<td>2.2.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/rocwmma"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/Tensile/en/docs-7.2.3/src/index.html">Tensile</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/Tensile/en/docs-7.2.1/src/index.html">Tensile</a></td>
|
||||
<td>4.45.0</td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/shared/tensile"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
@@ -510,22 +404,22 @@ Click {fab}`github` to go to the component's source code on GitHub.
|
||||
<tr>
|
||||
<th rowspan="4"></th>
|
||||
<th rowspan="4">Primitives</th>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipCUB/en/docs-7.2.3/index.html">hipCUB</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipCUB/en/docs-7.2.1/index.html">hipCUB</a></td>
|
||||
<td>4.2.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipcub"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipTensor/en/docs-7.2.3/index.html">hipTensor</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/hipTensor/en/docs-7.2.1/index.html">hipTensor</a></td>
|
||||
<td>2.2.0</td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/hiptensor"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocPRIM/en/docs-7.2.3/index.html">rocPRIM</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocPRIM/en/docs-7.2.1/index.html">rocPRIM</a></td>
|
||||
<td>4.2.0</td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/rocprim"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocThrust/en/docs-7.2.3/index.html">rocThrust</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocThrust/en/docs-7.2.1/index.html">rocThrust</a></td>
|
||||
<td>4.2.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-libraries/tree/develop/projects/rocthrust"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
@@ -534,27 +428,27 @@ Click {fab}`github` to go to the component's source code on GitHub.
|
||||
<tr>
|
||||
<th rowspan="7">Tools</th>
|
||||
<th rowspan="7">System management</th>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/amdsmi/en/docs-7.2.3/index.html">AMD SMI</a></td>
|
||||
<td>26.2.2</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/amdsmi/en/docs-7.2.1/index.html">AMD SMI</a></td>
|
||||
<td>26.2.1 ⇒ <a href="#amd-smi-26-2-2">26.2.2</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-systems/tree/develop/projects/amdsmi"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rdc/en/docs-7.2.3/index.html">ROCm Data Center Tool</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rdc/en/docs-7.2.1/index.html">ROCm Data Center Tool</a></td>
|
||||
<td>1.2.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-systems/tree/develop/projects/rdc/"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocminfo/en/docs-7.2.3/index.html">rocminfo</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocminfo/en/docs-7.2.1/index.html">rocminfo</a></td>
|
||||
<td>1.0.0</td>
|
||||
<td><a href="https://github.com/ROCm/rocm-systems/tree/develop/projects/rocminfo/"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocm_smi_lib/en/docs-7.2.3/index.html">ROCm SMI</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocm_smi_lib/en/docs-7.2.1/index.html">ROCm SMI</a></td>
|
||||
<td>7.8.0</td>
|
||||
<td><a href="https://github.com/ROCm/rocm-systems/tree/develop/projects/rocm-smi-lib/"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/ROCmValidationSuite/en/docs-7.2.3/index.html">ROCm Validation Suite</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/ROCmValidationSuite/en/docs-7.2.1/index.html">ROCm Validation Suite</a></td>
|
||||
<td>1.3.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/ROCmValidationSuite"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
@@ -563,38 +457,38 @@ Click {fab}`github` to go to the component's source code on GitHub.
|
||||
<tr>
|
||||
<th rowspan="6"></th>
|
||||
<th rowspan="6">Performance</th>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocm_bandwidth_test/en/docs-7.2.3/index.html">ROCm Bandwidth
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocm_bandwidth_test/en/docs-7.2.1/index.html">ROCm Bandwidth
|
||||
Test</a></td>
|
||||
<td>2.6.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm_bandwidth_test/"><i
|
||||
class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocprofiler-compute/en/docs-7.2.3/index.html">ROCm Compute Profiler</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocprofiler-compute/en/docs-7.2.1/index.html">ROCm Compute Profiler</a></td>
|
||||
<td>3.4.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-systems/tree/develop/projects/rocprofiler-compute"><i
|
||||
class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocprofiler-systems/en/docs-7.2.3/index.html">ROCm Systems Profiler</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocprofiler-systems/en/docs-7.2.1/index.html">ROCm Systems Profiler</a></td>
|
||||
<td>1.3.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-systems/tree/develop/projects/rocprofiler-systems/"><i
|
||||
class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocprofiler/en/docs-7.2.3/index.html">ROCProfiler</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocprofiler/en/docs-7.2.1/index.html">ROCProfiler</a></td>
|
||||
<td>2.0.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-systems/tree/develop/projects/rocprofiler/"><i
|
||||
class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocprofiler-sdk/en/docs-7.2.3/index.html">ROCprofiler-SDK</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocprofiler-sdk/en/docs-7.2.1/index.html">ROCprofiler-SDK</a></td>
|
||||
<td>1.1.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-systems/tree/develop/projects/rocprofiler-sdk/"><i
|
||||
class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr >
|
||||
<td><a href="https://rocm.docs.amd.com/projects/roctracer/en/docs-7.2.3/index.html">ROCTracer</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/roctracer/en/docs-7.2.1/index.html">ROCTracer</a></td>
|
||||
<td>4.1.0</td>
|
||||
<td><a href="https://github.com/ROCm/rocm-systems/tree/develop/projects/roctracer/"><i
|
||||
class="fab fa-github fa-lg"></i></a></td>
|
||||
@@ -604,32 +498,32 @@ Click {fab}`github` to go to the component's source code on GitHub.
|
||||
<tr>
|
||||
<th rowspan="5"></th>
|
||||
<th rowspan="5">Development</th>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/HIPIFY/en/docs-7.2.3/index.html">HIPIFY</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/HIPIFY/en/docs-7.2.1/index.html">HIPIFY</a></td>
|
||||
<td>22.0.0</td>
|
||||
<td><a href="https://github.com/ROCm/HIPIFY/"><i
|
||||
class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/ROCdbgapi/en/docs-7.2.3/index.html">ROCdbgapi</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/ROCdbgapi/en/docs-7.2.1/index.html">ROCdbgapi</a></td>
|
||||
<td>0.77.4</a></td>
|
||||
<td><a href="https://github.com/ROCm/ROCdbgapi/"><i
|
||||
class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/ROCmCMakeBuildTools/en/docs-7.2.3/index.html">ROCm CMake</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/ROCmCMakeBuildTools/en/docs-7.2.1/index.html">ROCm CMake</a></td>
|
||||
<td>0.14.0</td>
|
||||
<td><a href="https://github.com/ROCm/rocm-cmake/"><i
|
||||
class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/ROCgdb/en/docs-7.2.3/index.html">ROCm Debugger (ROCgdb)</a>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/ROCgdb/en/docs-7.2.1/index.html">ROCm Debugger (ROCgdb)</a>
|
||||
</td>
|
||||
<td>16.3</a></td>
|
||||
<td><a href="https://github.com/ROCm/ROCgdb/"><i
|
||||
class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocr_debug_agent/en/docs-7.2.3/index.html">ROCr Debug Agent</a>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/rocr_debug_agent/en/docs-7.2.1/index.html">ROCr Debug Agent</a>
|
||||
</td>
|
||||
<td>2.1.0</td>
|
||||
<td><a href="https://github.com/ROCm/rocr_debug_agent/"><i
|
||||
@@ -639,13 +533,13 @@ Click {fab}`github` to go to the component's source code on GitHub.
|
||||
<tbody class="rocm-components-compilers tbody-reverse-zebra">
|
||||
<tr>
|
||||
<th rowspan="2" colspan="2">Compilers</th>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/HIPCC/en/docs-7.2.3/index.html">HIPCC</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/HIPCC/en/docs-7.2.1/index.html">HIPCC</a></td>
|
||||
<td>1.1.1</td>
|
||||
<td><a href="https://github.com/ROCm/llvm-project/tree/amd-staging/amd/hipcc"><i
|
||||
class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/llvm-project/en/docs-7.2.3/index.html">llvm-project</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/llvm-project/en/docs-7.2.1/index.html">llvm-project</a></td>
|
||||
<td>22.0.0</a></td>
|
||||
<td><a href="https://github.com/ROCm/llvm-project/"><i
|
||||
class="fab fa-github fa-lg"></i></a></td>
|
||||
@@ -654,12 +548,12 @@ Click {fab}`github` to go to the component's source code on GitHub.
|
||||
<tbody class="rocm-components-runtimes tbody-reverse-zebra">
|
||||
<tr>
|
||||
<th rowspan="2" colspan="2">Runtimes</th>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/HIP/en/docs-7.2.3/index.html">HIP</a></td>
|
||||
<td>7.2.1</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/HIP/en/docs-7.2.1/index.html">HIP</a></td>
|
||||
<td>7.2.0 ⇒ <a href="#hip-7-2-1">7.2.1</a></td>
|
||||
<td><a href="https://github.com/ROCm/rocm-systems/tree/develop/projects/hip"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/ROCR-Runtime/en/docs-7.2.3/index.html">ROCr Runtime</a></td>
|
||||
<td><a href="https://rocm.docs.amd.com/projects/ROCR-Runtime/en/docs-7.2.1/index.html">ROCr Runtime</a></td>
|
||||
<td>1.18.0</td>
|
||||
<td><a href="https://github.com/ROCm/rocm-systems/tree/develop/projects/rocr-runtime"><i class="fab fa-github fa-lg"></i></a></td>
|
||||
</tr>
|
||||
@@ -667,7 +561,7 @@ Click {fab}`github` to go to the component's source code on GitHub.
|
||||
</table>
|
||||
</div>
|
||||
|
||||
### Detailed component changes
|
||||
## Detailed component changes
|
||||
|
||||
The following sections describe key changes to ROCm components.
|
||||
|
||||
@@ -675,53 +569,134 @@ The following sections describe key changes to ROCm components.
|
||||
For a historical overview of ROCm component updates, see the {doc}`ROCm consolidated changelog </release/changelog>`.
|
||||
```
|
||||
|
||||
#### **MIGraphX** (2.15.0)
|
||||
### **AMD SMI** (26.2.2)
|
||||
|
||||
##### Added
|
||||
#### Added
|
||||
|
||||
* External stream support to the MIGraphX context, allowing external HIP streams to be used during execution.
|
||||
* Ability to return a vector for output alias, supporting operators like `make_tuple`.
|
||||
* GPU board and base board temperature sensors to `amd-smi monitor` command.
|
||||
|
||||
##### Changed
|
||||
#### Resolved issues
|
||||
|
||||
* Refactored `move_output_instructions_after` into the module class.
|
||||
* Updated rocMLIR to fix `bert_squad` and `bert_tf` regressions.
|
||||
* JSON output was not formatted correctly when using watch mode with metrics.
|
||||
* Output was not properly redirected to file when using JSON format.
|
||||
* CPER component output was not redirected when using the `--follow` option.
|
||||
* Invalid CPER files caused garbage output for AFID lists.
|
||||
* JSON output was not formatted correctly for reset commands.
|
||||
|
||||
##### Optimized
|
||||
### **HIP** (7.2.1)
|
||||
|
||||
* Rewrote the `gather` operator to use `transpose`/`reshape`/`broadcast`/`slice` for improved performance.
|
||||
* Horizontally fuse cross-embedding `gather` operators.
|
||||
* Improved tuning for Split-K.
|
||||
* Removed extra assignments and inserts in `find_nop_reshapes` to reduce overhead.
|
||||
#### Resolved issues
|
||||
|
||||
##### Resolved issues
|
||||
* Corrected the validation of stream capture in global‑capture mode. It is no longer affected by any thread‑local capture‑mode sequences occurring in other threads.
|
||||
* Corrected the return value of `hipEventQuery` and `hipEventSynchronize`. The HIP runtime now properly handles and restricts stream capture within these APIs.
|
||||
* Corrected an issue in the batch-dispatch doorbell for AQL packets to avoid a potential CPU hang.
|
||||
* To address potential delays in memory‑object destruction that could affect application logic, the HIP runtime disables memory‑object reference counting in direct‑dispatch mode.
|
||||
|
||||
The following issues have been fixed:
|
||||
#### Changed
|
||||
|
||||
* `int` to `bf16`/`fp16` conversion errors.
|
||||
* Comparison logic in `find_concat_op` to match the correct I/O.
|
||||
* `shape_transform_descriptor::rebase` when flattening a broadcasted dimension.
|
||||
* An error with `rewrite_reshapes`.
|
||||
* A gather rewrite crash by validating the strided view element count.
|
||||
* A bug in gather rewrite with NHWC shapes.
|
||||
* A crash in rocMLIR with Inception v3 on RDNA3 architecture-based Radeon GPUs.
|
||||
* Filter zero-argument operators during ONNX parsing to prevent errors.
|
||||
* Conflict for missing `no_broadcast` parameter on ROCm 7.2.x.
|
||||
* The `AMD_DIRECT_DISPATCH` environment variable has been deprecated in the HIP runtime.
|
||||
|
||||
### ROCm known issues
|
||||
### **hipBLASLt** (1.2.2)
|
||||
|
||||
#### Changed
|
||||
|
||||
* Enumeration value update for the Sigmoid Activation Function feature.
|
||||
|
||||
### **rocDecode** (1.7.0)
|
||||
|
||||
#### Upcoming changes
|
||||
|
||||
* The rocDecode GitHub repository will be officially moved to [https://github.com/ROCm/rocm-systems/tree/develop/projects/rocdecode](https://github.com/ROCm/rocm-systems/tree/develop/projects/rocdecode) in an upcoming release.
|
||||
|
||||
### **rocJPEG** (1.4.0)
|
||||
|
||||
#### Changed
|
||||
|
||||
* Bug fixes and performance improvements.
|
||||
|
||||
#### Upcoming changes
|
||||
|
||||
* The rocJPEG GitHub repository will be officially moved to [https://github.com/ROCm/rocm-systems/tree/develop/projects/rocjpeg](https://github.com/ROCm/rocm-systems/tree/develop/projects/rocjpeg) in an upcoming release.
|
||||
|
||||
### **rocSHMEM** (3.2.0)
|
||||
|
||||
#### Added
|
||||
* Warnings to notify if large BAR is not available.
|
||||
|
||||
#### Resolved issues
|
||||
|
||||
* GDA Backend will disable itself when no GDA compatible NICs are available rather than crashing.
|
||||
* Fix memory coherency issues on gfx1201.
|
||||
|
||||
#### Known issues
|
||||
|
||||
* Only 64-bit rocSHMEM atomic APIs are implemented for the GDA conduit.
|
||||
|
||||
### **RPP** (2.2.1)
|
||||
|
||||
#### Added
|
||||
|
||||
* Error-code capture in test scripts for all C++ tests.
|
||||
|
||||
#### Optimized
|
||||
|
||||
* Optimized F16 variants by replacing scalar load/store operations with AVX2 intrinsics for spatter, log, blend, color_cast, flip, crop_mirror_normalize, and exposure kernels.
|
||||
|
||||
## ROCm known issues
|
||||
|
||||
ROCm known issues are noted on {fab}`github` [GitHub](https://github.com/ROCm/ROCm/labels/Verified%20Issue). For known
|
||||
issues related to individual components, review the [Detailed component changes](#detailed-component-changes).
|
||||
|
||||
#### Minor performance regression for MIGraphX with int8-quantized models
|
||||
### hipBLASLt performance regression for specific GEMM configurations
|
||||
|
||||
You might observe a slight performance regression when running int8-quantized models with MIGraphX. This impact is generally minimal and does not affect correctness. However, workloads sensitive to peak throughput might have reduced performance when compared to non-quantized or alternative execution paths. This issue is currently under investigation and will be fixed in a future ROCm release. See [GitHub issue #6195](https://github.com/ROCm/ROCm/issues/6195).
|
||||
You might observe a noticeable performance regression if you’re using hipBLASLt with the following GPUs for LLMs with specific GEMM configurations:
|
||||
|
||||
### ROCm upcoming changes
|
||||
#### AMD Instinct MI300X and MI325X GPUs
|
||||
|
||||
Affected GEMM configurations:
|
||||
|
||||
* 16384 × 16384 × 6656 (BBS, TN)
|
||||
|
||||
* 32768 × 8192 × 3072 (BBS, TN)
|
||||
|
||||
* 9728 × 8192 × 65536 (F8F8S, TN)
|
||||
|
||||
#### AMD Instinct MI350 Series GPUs
|
||||
|
||||
Affected GEMM configurations:
|
||||
|
||||
* 4096 × 4096 × 1 × 8192
|
||||
|
||||
* 4096 × 4096 × 1 × 16384
|
||||
|
||||
* 8192 × 8192 × 1 × 8192
|
||||
|
||||
* 8192 × 8192 × 1 × 16384
|
||||
|
||||
Due to this issue, you might also observe a slight increase in the test or inference time. This issue is resolved in the {fab}`github`[hipBLASLt develop branch](https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipblaslt) and will be part of a future ROCm release. See [GitHub issue #6065](https://github.com/ROCm/ROCm/issues/6065).
|
||||
|
||||
### Longer runtime for hipBLASLt GEMM operations on Instinct MI300X GPUs in partitioned mode
|
||||
|
||||
GEMM operations using hipBLASLt might result in longer runtime on AMD Instinct MI300X GPUs configured in CPX or NPS4 partition mode (38 control units or CUs). This issue occurs when hipBLASLt fails to find applicable pre-tuned kernels. As a result, it performs an extensive kernel search, which increases both search time and the overall operation runtime. This issue is resolved in the {fab}`github`[hipBLASLt develop branch](https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipblaslt) and will be part of a future ROCm release. See [GitHub issue #6066](https://github.com/ROCm/ROCm/issues/6066).
|
||||
|
||||
### ROCTracer might fail to report kernel operations
|
||||
|
||||
Applications that use [ROCTracer](https://rocm.docs.amd.com/projects/roctracer/en/latest/index.html) might fail to receive some or all kernel operation events due to a ROCTracer reporting failure. ROCTracer is already deprecated and is scheduled to reach end of support (EoS) by the end of 2026 Q2. For more details on ROCTracer deprecation, see [ROCm upcoming changes](#roctracer-rocprofiler-rocprof-and-rocprofv2-deprecation). This issue will be resolved in a future PyTorch on ROCm release that replaces ROCTracer with [ROCprofiler-SDK](https://rocm.docs.amd.com/projects/rocprofiler-sdk/en/latest/). See [GitHub issue #6102](https://github.com/ROCm/ROCm/issues/6102).
|
||||
|
||||
## ROCm resolved issues
|
||||
|
||||
The following are previously known issues resolved in this release. For resolved issues related to
|
||||
individual components, review the [Detailed component changes](#detailed-component-changes).
|
||||
|
||||
### Increased runtime latency of the HIP hipStreamCreate API
|
||||
|
||||
As issue that resulted in doubling of the runtime latency of the [HIP](https://rocmdocs.amd.com/projects/HIP/en/latest/doxygen/html/group___stream.html) `hipStreamCreate` API has been resolved. See [GitHub issue #5978](https://github.com/ROCm/ROCm/issues/5978).
|
||||
|
||||
## ROCm upcoming changes
|
||||
|
||||
The following changes to the ROCm software stack are anticipated for future releases.
|
||||
|
||||
#### ROCTracer, ROCProfiler, rocprof, and rocprofv2 deprecation
|
||||
### ROCTracer, ROCProfiler, rocprof, and rocprofv2 deprecation
|
||||
|
||||
ROCTracer, ROCProfiler, `rocprof`, and `rocprofv2` are deprecated. It's strongly recommended to upgrade to the latest version of the [ROCprofiler-SDK](https://rocm.docs.amd.com/projects/rocprofiler-sdk/en/latest/) library and the (`rocprofv3`) tool to ensure continued support and access to new features.
|
||||
|
||||
@@ -729,7 +704,7 @@ To learn about key feature improvements and benefits of ROCprofiler-SDK over the
|
||||
|
||||
It's anticipated that ROCTracer, ROCProfiler, `rocprof`, and `rocprofv2` will reach end of support (EoS) by the end of 2026 Q2.
|
||||
|
||||
#### ROCm SMI deprecation
|
||||
### ROCm SMI deprecation
|
||||
|
||||
[ROCm SMI](https://github.com/ROCm/rocm_smi_lib) will be phased out in an
|
||||
upcoming ROCm release and will enter maintenance mode. After this transition,
|
||||
@@ -742,7 +717,7 @@ includes all the features of the ROCm SMI and will continue to receive regular
|
||||
updates, new functionality, and ongoing support. For more information on AMD
|
||||
SMI, see the [AMD SMI documentation](https://rocm.docs.amd.com/projects/amdsmi/en/latest/).
|
||||
|
||||
#### Changes to ROCm Object Tooling
|
||||
### Changes to ROCm Object Tooling
|
||||
|
||||
ROCm Object Tooling tools ``roc-obj-ls``, ``roc-obj-extract``, and ``roc-obj`` were
|
||||
deprecated in ROCm 6.4, and will be removed in a future release. Functionality
|
||||
|
||||
@@ -1,130 +1,130 @@
|
||||
ROCm Version,7.2.4/7.2.3,7.2.2/7.2.1,7.2.0,7.1.1,7.1.0,7.0.2,7.0.1/7.0.0,6.4.3,6.4.2,6.4.1,6.4.0,6.3.3,6.3.2,6.3.1,6.3.0,6.2.4,6.2.2,6.2.1,6.2.0, 6.1.5, 6.1.2, 6.1.1, 6.1.0, 6.0.2, 6.0.0
|
||||
:ref:`Operating systems & kernels <OS-kernel-versions>` [#os-compatibility-past-60]_,Ubuntu 24.04.4,Ubuntu 24.04.4,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,"Ubuntu 24.04.1, 24.04","Ubuntu 24.04.1, 24.04","Ubuntu 24.04.1, 24.04",Ubuntu 24.04,,,,,,
|
||||
,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,"Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3, 22.04.2","Ubuntu 22.04.4, 22.04.3, 22.04.2"
|
||||
,,,,,,,,,,,,,,,,,,,,"Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5"
|
||||
,"RHEL 10.1, 10.0, 9.7, 9.6, 9.4","RHEL 10.1, 10.0, 9.7, 9.6, 9.4","RHEL 10.1, 10.0, 9.7, 9.6, 9.4","RHEL 10.1, 10.0, 9.7, 9.6, 9.4","RHEL 10.0, 9.6, 9.4","RHEL 10.0, 9.6, 9.4","RHEL 9.6, 9.4","RHEL 9.6, 9.4","RHEL 9.6, 9.4","RHEL 9.6, 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4, 9.3, 9.2","RHEL 9.4, 9.3, 9.2","RHEL 9.4, 9.3, 9.2","RHEL 9.4, 9.3, 9.2","RHEL 9.3, 9.2","RHEL 9.3, 9.2"
|
||||
,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,"RHEL 8.10, 8.9","RHEL 8.10, 8.9","RHEL 8.10, 8.9","RHEL 8.10, 8.9","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8"
|
||||
,SLES 15 SP7,SLES 15 SP7,SLES 15 SP7,SLES 15 SP7,SLES 15 SP7,SLES 15 SP7,SLES 15 SP7,"SLES 15 SP7, SP6","SLES 15 SP7, SP6",SLES 15 SP6,SLES 15 SP6,"SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4"
|
||||
,,,,,,,,,,,,,,,,,,,,,CentOS 7.9,CentOS 7.9,CentOS 7.9,CentOS 7.9,CentOS 7.9
|
||||
,"Oracle Linux 10, 9, 8","Oracle Linux 10, 9, 8","Oracle Linux 10, 9, 8","Oracle Linux 10, 9, 8","Oracle Linux 10, 9, 8","Oracle Linux 10, 9, 8","Oracle Linux 9, 8","Oracle Linux 9, 8","Oracle Linux 9, 8","Oracle Linux 9, 8","Oracle Linux 9, 8",Oracle Linux 8.10,Oracle Linux 8.10,Oracle Linux 8.10,Oracle Linux 8.10,Oracle Linux 8.9,Oracle Linux 8.9,Oracle Linux 8.9,Oracle Linux 8.9,Oracle Linux 8.9,Oracle Linux 8.9,Oracle Linux 8.9,,,
|
||||
,"Debian 13, 12","Debian 13, 12","Debian 13, 12","Debian 13, 12","Debian 13, 12","Debian 13, 12",Debian 12,Debian 12,Debian 12,Debian 12,Debian 12,Debian 12,Debian 12,Debian 12,,,,,,,,,,,
|
||||
,,,,,,Azure Linux 3.0,Azure Linux 3.0,Azure Linux 3.0,Azure Linux 3.0,Azure Linux 3.0,Azure Linux 3.0,Azure Linux 3.0,Azure Linux 3.0,,,,,,,,,,,,
|
||||
,Rocky Linux 9,Rocky Linux 9,Rocky Linux 9,Rocky Linux 9,Rocky Linux 9,Rocky Linux 9,Rocky Linux 9,,,,,,,,,,,,,,,,,,
|
||||
,.. _architecture-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`Architecture <rocm-install-on-linux:reference/system-requirements>`,CDNA4,CDNA4,CDNA4,CDNA4,CDNA4,CDNA4,CDNA4,,,,,,,,,,,,,,,,,,
|
||||
,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3
|
||||
,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2
|
||||
,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA
|
||||
,RDNA4,RDNA4,RDNA4,RDNA4,RDNA4,RDNA4,RDNA4,RDNA4,RDNA4,RDNA4,,,,,,,,,,,,,,,
|
||||
,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3
|
||||
,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2
|
||||
,.. _gpu-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`GPU / LLVM target <rocm-install-on-linux:reference/system-requirements>` [#gpu-compatibility-past-60]_,gfx950,gfx950,gfx950,gfx950,gfx950,gfx950,gfx950,,,,,,,,,,,,,,,,,,
|
||||
,gfx1201,gfx1201,gfx1201,gfx1201,gfx1201,gfx1201,gfx1201,gfx1201,gfx1201,gfx1201,,,,,,,,,,,,,,,
|
||||
,gfx1200,gfx1200,gfx1200,gfx1200,gfx1200,gfx1200,gfx1200,gfx1200,gfx1200,gfx1200,,,,,,,,,,,,,,,
|
||||
,gfx1101,gfx1101,gfx1101,gfx1101,gfx1101,gfx1101,gfx1101,gfx1101,gfx1101,gfx1101,,,,,,,,,,,,,,,
|
||||
,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100
|
||||
,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030
|
||||
,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942, gfx942, gfx942, gfx942, gfx942, gfx942, gfx942
|
||||
,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a
|
||||
,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
FRAMEWORK SUPPORT,.. _framework-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`PyTorch <../compatibility/ml-compatibility/pytorch-compatibility>`,"2.9.1, 2.8.0, 2.7.1","2.9.1, 2.8.0, 2.7.1","2.9.1, 2.8.0, 2.7.1","2.9, 2.8, 2.7","2.8, 2.7, 2.6","2.8, 2.7, 2.6","2.7, 2.6, 2.5","2.6, 2.5, 2.4, 2.3","2.6, 2.5, 2.4, 2.3","2.6, 2.5, 2.4, 2.3","2.6, 2.5, 2.4, 2.3","2.4, 2.3, 2.2, 1.13","2.4, 2.3, 2.2, 1.13","2.4, 2.3, 2.2, 1.13","2.4, 2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13"
|
||||
:doc:`TensorFlow <../compatibility/ml-compatibility/tensorflow-compatibility>`,"2.20.0, 2.19.1, 2.18.1","2.20.0, 2.19.1, 2.18.1","2.20.0, 2.19.1, 2.18.1","2.20.0, 2.19.1, 2.18.1","2.20.0, 2.19.1, 2.18.1","2.19.1, 2.18.1, 2.17.1 [#tf-mi350-past-60]_","2.19.1, 2.18.1, 2.17.1 [#tf-mi350-past-60]_","2.18.1, 2.17.1, 2.16.2","2.18.1, 2.17.1, 2.16.2","2.18.1, 2.17.1, 2.16.2","2.18.1, 2.17.1, 2.16.2","2.17.0, 2.16.2, 2.15.1","2.17.0, 2.16.2, 2.15.1","2.17.0, 2.16.2, 2.15.1","2.17.0, 2.16.2, 2.15.1","2.16.1, 2.15.1, 2.14.1","2.16.1, 2.15.1, 2.14.1","2.16.1, 2.15.1, 2.14.1","2.16.1, 2.15.1, 2.14.1","2.15.0, 2.14.0, 2.13.1","2.15.0, 2.14.0, 2.13.1","2.15.0, 2.14.0, 2.13.1","2.15.0, 2.14.0, 2.13.1","2.14.0, 2.13.1, 2.12.1","2.14.0, 2.13.1, 2.12.1"
|
||||
:doc:`JAX <../compatibility/ml-compatibility/jax-compatibility>`,0.8.2,0.8.2,0.8.0,0.7.1,0.7.1,0.6.0,0.6.0,0.4.35,0.4.35,0.4.35,0.4.35,0.4.31,0.4.31,0.4.31,0.4.31,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26
|
||||
:doc:`DGL <../compatibility/ml-compatibility/dgl-compatibility>` [#dgl_compat-past-60]_,N/A,N/A,N/A,N/A,N/A,N/A,2.4.0,2.4.0,N/A,N/A,2.4.0,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
`ONNX Runtime <https://onnxruntime.ai/docs/build/eps.html#amd-migraphx>`_,1.23.2,1.23.2,1.23.2,1.23.1,1.22.0,1.22.0,1.22.0,1.20.0,1.20.0,1.20.0,1.20.0,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.14.1,1.14.1
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
THIRD PARTY COMMS,.. _thirdpartycomms-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
`UCC <https://github.com/ROCm/ucc>`_,>=1.6.0,>=1.6.0,>=1.4.0,>=1.4.0,>=1.4.0,>=1.4.0,>=1.4.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.2.0,>=1.2.0
|
||||
`UCX <https://github.com/ROCm/ucx>`_,>=1.17.0,>=1.17.0,>=1.17.0,>=1.17.0,>=1.17.0,>=1.17.0,>=1.17.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.14.1,>=1.14.1,>=1.14.1,>=1.14.1,>=1.14.1,>=1.14.1
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
THIRD PARTY ALGORITHM,.. _thirdpartyalgorithm-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
Thrust,2.8.5,2.8.5,2.8.5,2.8.5,2.8.5,2.6.0,2.6.0,2.5.0,2.5.0,2.5.0,2.5.0,2.3.2,2.3.2,2.3.2,2.3.2,2.2.0,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.1.0,2.0.1,2.0.1
|
||||
CUB,2.8.5,2.8.5,2.8.5,2.8.5,2.8.5,2.6.0,2.6.0,2.5.0,2.5.0,2.5.0,2.5.0,2.3.2,2.3.2,2.3.2,2.3.2,2.2.0,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.1.0,2.0.1,2.0.1
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
DRIVER & USER SPACE [#kfd_support-past-60]_,.. _kfd-userspace-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`AMD GPU Driver <rocm-install-on-linux:reference/user-kernel-space-compat-matrix>`,"30.30.x, 30.20.x [#mi325x_KVM-past-60]_, 30.10.x [#driver_patch-past-60]_, 6.4.x","30.30.x, 30.20.x [#mi325x_KVM-past-60]_, 30.10.x [#driver_patch-past-60]_, 6.4.x","30.30.0, 30.20.1, 30.20.0 [#mi325x_KVM-past-60]_, 30.10.2, 30.10.1 [#driver_patch-past-60]_, 30.10, 6.4.x","30.20.1, 30.20.0 [#mi325x_KVM-past-60]_, 30.10.2, 30.10.1 [#driver_patch-past-60]_, 30.10, 6.4.x","30.20.0 [#mi325x_KVM-past-60]_, 30.10.2, 30.10.1 [#driver_patch-past-60]_, 30.10, 6.4.x","30.10.2, 30.10.1 [#driver_patch-past-60]_, 30.10, 6.4.x, 6.3.x","30.10.1 [#driver_patch-past-60]_, 30.10, 6.4.x, 6.3.x, 6.2.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x, 5.7.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x, 5.7.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x, 5.7.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x, 5.7.x","6.2.x, 6.1.x, 6.0.x, 5.7.x, 5.6.x","6.2.x, 6.1.x, 6.0.x, 5.7.x, 5.6.x"
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
ML & COMPUTER VISION,.. _mllibs-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`Composable Kernel <composable_kernel:index>`,1.2.0,1.2.0,1.2.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0
|
||||
:doc:`MIGraphX <amdmigraphx:index>`,2.15.0,2.15.0,2.15.0,2.14.0,2.14.0,2.13.0,2.13.0,2.12.0,2.12.0,2.12.0,2.12.0,2.11.0,2.11.0,2.11.0,2.11.0,2.10.0,2.10.0,2.10.0,2.10.0,2.9.0,2.9.0,2.9.0,2.9.0,2.8.0,2.8.0
|
||||
:doc:`MIOpen <miopen:index>`,3.5.1,3.5.1,3.5.1,3.5.1,3.5.1,3.5.0,3.5.0,3.4.0,3.4.0,3.4.0,3.4.0,3.3.0,3.3.0,3.3.0,3.3.0,3.2.0,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0
|
||||
:doc:`MIVisionX <mivisionx:index>`,3.5.0,3.5.0,3.5.0,3.4.0,3.4.0,3.3.0,3.3.0,3.2.0,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0,3.0.0,3.0.0,2.5.0,2.5.0,2.5.0,2.5.0,2.5.0,2.5.0
|
||||
:doc:`rocAL <rocal:index>`,2.5.0,2.5.0,2.5.0,2.4.0,2.4.0,2.3.0,2.3.0,2.2.0,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.1.0,2.0.0,2.0.0,2.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0
|
||||
:doc:`rocDecode <rocdecode:index>`,1.7.0,1.7.0,1.5.0,1.4.0,1.4.0,1.0.0,1.0.0,0.10.0,0.10.0,0.10.0,0.10.0,0.8.0,0.8.0,0.8.0,0.8.0,0.6.0,0.6.0,0.6.0,0.6.0,0.6.0,0.6.0,0.5.0,0.5.0,N/A,N/A
|
||||
:doc:`rocJPEG <rocjpeg:index>`,1.4.0,1.4.0,1.3.0,1.2.0,1.2.0,1.1.0,1.1.0,0.8.0,0.8.0,0.8.0,0.8.0,0.6.0,0.6.0,0.6.0,0.6.0,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
:doc:`rocPyDecode <rocpydecode:index>`,0.8.0,0.8.0,0.8.0,0.7.0,0.7.0,0.6.0,0.6.0,0.3.1,0.3.1,0.3.1,0.3.1,0.2.0,0.2.0,0.2.0,0.2.0,0.1.0,0.1.0,0.1.0,0.1.0,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
:doc:`RPP <rpp:index>`,2.2.1,2.2.1,2.2.0,2.1.0,2.1.0,2.0.0,2.0.0,1.9.10,1.9.10,1.9.10,1.9.10,1.9.1,1.9.1,1.9.1,1.9.1,1.8.0,1.8.0,1.8.0,1.8.0,1.5.0,1.5.0,1.5.0,1.5.0,1.4.0,1.4.0
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
COMMUNICATION,.. _commlibs-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`RCCL <rccl:index>`,2.27.7,2.27.7,2.27.7,2.27.7,2.27.7,2.26.6,2.26.6,2.22.3,2.22.3,2.22.3,2.22.3,2.21.5,2.21.5,2.21.5,2.21.5,2.20.5,2.20.5,2.20.5,2.20.5,2.18.6,2.18.6,2.18.6,2.18.6,2.18.3,2.18.3
|
||||
:doc:`rocSHMEM <rocshmem:index>`,3.2.0,3.2.0,3.2.0,3.1.0,3.0.0,3.0.0,3.0.0,2.0.1,2.0.1,2.0.0,2.0.0,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
MATH LIBS,.. _mathlibs-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
`half <https://github.com/ROCm/half>`_ ,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0
|
||||
:doc:`hipBLAS <hipblas:index>`,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.0.2,3.0.0,2.4.0,2.4.0,2.4.0,2.4.0,2.3.0,2.3.0,2.3.0,2.3.0,2.2.0,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.1.0,2.0.0,2.0.0
|
||||
:doc:`hipBLASLt <hipblaslt:index>`,1.2.2,1.2.2,1.2.1,1.1.0,1.1.0,1.0.0,1.0.0,0.12.1,0.12.1,0.12.1,0.12.0,0.10.0,0.10.0,0.10.0,0.10.0,0.8.0,0.8.0,0.8.0,0.8.0,0.7.0,0.7.0,0.7.0,0.7.0,0.6.0,0.6.0
|
||||
:doc:`hipFFT <hipfft:index>`,1.0.22,1.0.22,1.0.22,1.0.21,1.0.21,1.0.20,1.0.20,1.0.18,1.0.18,1.0.18,1.0.18,1.0.17,1.0.17,1.0.17,1.0.17,1.0.16,1.0.15,1.0.15,1.0.14,1.0.14,1.0.14,1.0.14,1.0.14,1.0.13,1.0.13
|
||||
:doc:`hipfort <hipfort:index>`,0.7.1,0.7.1,0.7.1,0.7.1,0.7.1,0.7.0,0.7.0,0.6.0,0.6.0,0.6.0,0.6.0,0.5.1,0.5.1,0.5.0,0.5.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0
|
||||
:doc:`hipRAND <hiprand:index>`,3.1.0,3.1.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0,2.12.0,2.12.0,2.12.0,2.12.0,2.11.1,2.11.1,2.11.1,2.11.0,2.11.1,2.11.0,2.11.0,2.11.0,2.10.16,2.10.16,2.10.16,2.10.16,2.10.16,2.10.16
|
||||
:doc:`hipSOLVER <hipsolver:index>`,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.0.0,3.0.0,2.4.0,2.4.0,2.4.0,2.4.0,2.3.0,2.3.0,2.3.0,2.3.0,2.2.0,2.2.0,2.2.0,2.2.0,2.1.1,2.1.1,2.1.1,2.1.0,2.0.0,2.0.0
|
||||
:doc:`hipSPARSE <hipsparse:index>`,4.2.0,4.2.0,4.2.0,4.1.0,4.1.0,4.0.1,4.0.1,3.2.0,3.2.0,3.2.0,3.2.0,3.1.2,3.1.2,3.1.2,3.1.2,3.1.1,3.1.1,3.1.1,3.1.1,3.0.1,3.0.1,3.0.1,3.0.1,3.0.0,3.0.0
|
||||
:doc:`hipSPARSELt <hipsparselt:index>`,0.2.6,0.2.6,0.2.6,0.2.5,0.2.5,0.2.4,0.2.4,0.2.3,0.2.3,0.2.3,0.2.3,0.2.2,0.2.2,0.2.2,0.2.2,0.2.1,0.2.1,0.2.1,0.2.1,0.2.0,0.2.0,0.1.0,0.1.0,0.1.0,0.1.0
|
||||
:doc:`rocALUTION <rocalution:index>`,4.1.0,4.1.0,4.1.0,4.0.1,4.0.1,4.0.0,4.0.0,3.2.3,3.2.3,3.2.3,3.2.2,3.2.1,3.2.1,3.2.1,3.2.1,3.2.1,3.2.0,3.2.0,3.2.0,3.1.1,3.1.1,3.1.1,3.1.1,3.0.3,3.0.3
|
||||
:doc:`rocBLAS <rocblas:index>`,5.2.0,5.2.0,5.2.0,5.1.1,5.1.0,5.0.2,5.0.0,4.4.1,4.4.1,4.4.0,4.4.0,4.3.0,4.3.0,4.3.0,4.3.0,4.2.4,4.2.1,4.2.1,4.2.0,4.1.2,4.1.2,4.1.0,4.1.0,4.0.0,4.0.0
|
||||
:doc:`rocFFT <rocfft:index>`,1.0.36,1.0.36,1.0.36,1.0.35,1.0.35,1.0.34,1.0.34,1.0.32,1.0.32,1.0.32,1.0.32,1.0.31,1.0.31,1.0.31,1.0.31,1.0.30,1.0.29,1.0.29,1.0.28,1.0.27,1.0.27,1.0.27,1.0.26,1.0.25,1.0.23
|
||||
:doc:`rocRAND <rocrand:index>`,4.2.0,4.2.0,4.2.0,4.1.0,4.1.0,4.0.0,4.0.0,3.3.0,3.3.0,3.3.0,3.3.0,3.2.0,3.2.0,3.2.0,3.2.0,3.1.1,3.1.0,3.1.0,3.1.0,3.0.1,3.0.1,3.0.1,3.0.1,3.0.0,2.10.17
|
||||
:doc:`rocSOLVER <rocsolver:index>`,3.32.0,3.32.0,3.32.0,3.31.0,3.31.0,3.30.1,3.30.0,3.28.2,3.28.2,3.28.0,3.28.0,3.27.0,3.27.0,3.27.0,3.27.0,3.26.2,3.26.0,3.26.0,3.26.0,3.25.0,3.25.0,3.25.0,3.25.0,3.24.0,3.24.0
|
||||
:doc:`rocSPARSE <rocsparse:index>`,4.2.0,4.2.0,4.2.0,4.1.0,4.1.0,4.0.2,4.0.2,3.4.0,3.4.0,3.4.0,3.4.0,3.3.0,3.3.0,3.3.0,3.3.0,3.2.1,3.2.0,3.2.0,3.2.0,3.1.2,3.1.2,3.1.2,3.1.2,3.0.2,3.0.2
|
||||
:doc:`rocWMMA <rocwmma:index>`,2.2.0,2.2.0,2.2.0,2.1.0,2.0.0,2.0.0,2.0.0,1.7.0,1.7.0,1.7.0,1.7.0,1.6.0,1.6.0,1.6.0,1.6.0,1.5.0,1.5.0,1.5.0,1.5.0,1.4.0,1.4.0,1.4.0,1.4.0,1.3.0,1.3.0
|
||||
:doc:`Tensile <tensile:src/index>`,4.45.0,4.45.0,4.45.0,4.44.0,4.44.0,4.44.0,4.44.0,4.43.0,4.43.0,4.43.0,4.43.0,4.42.0,4.42.0,4.42.0,4.42.0,4.41.0,4.41.0,4.41.0,4.41.0,4.40.0,4.40.0,4.40.0,4.40.0,4.39.0,4.39.0
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
PRIMITIVES,.. _primitivelibs-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`hipCUB <hipcub:index>`,4.2.0,4.2.0,4.2.0,4.1.0,4.1.0,4.0.0,4.0.0,3.4.0,3.4.0,3.4.0,3.4.0,3.3.0,3.3.0,3.3.0,3.3.0,3.2.1,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0
|
||||
:doc:`hipTensor <hiptensor:index>`,2.2.0,2.2.0,2.2.0,2.0.0,2.0.0,2.0.0,2.0.0,1.5.0,1.5.0,1.5.0,1.5.0,1.4.0,1.4.0,1.4.0,1.4.0,1.3.0,1.3.0,1.3.0,1.3.0,1.2.0,1.2.0,1.2.0,1.2.0,1.1.0,1.1.0
|
||||
:doc:`rocPRIM <rocprim:index>`,4.2.0,4.2.0,4.2.0,4.1.0,4.1.0,4.0.1,4.0.0,3.4.1,3.4.1,3.4.0,3.4.0,3.3.0,3.3.0,3.3.0,3.3.0,3.2.2,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0
|
||||
:doc:`rocThrust <rocthrust:index>`,4.2.0,4.2.0,4.2.0,4.1.0,4.1.0,4.0.0,4.0.0,3.3.0,3.3.0,3.3.0,3.3.0,3.3.0,3.3.0,3.3.0,3.3.0,3.1.1,3.1.0,3.1.0,3.0.1,3.0.1,3.0.1,3.0.1,3.0.1,3.0.0,3.0.0
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
SUPPORT LIBS,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
`hipother <https://github.com/ROCm/hipother>`_,7.2.53211,7.2.53211,7.2.26015,7.1.52802,7.1.25424,7.0.51831,7.0.51830,6.4.43483,6.4.43483,6.4.43483,6.4.43482,6.3.42134,6.3.42134,6.3.42133,6.3.42131,6.2.41134,6.2.41134,6.2.41134,6.2.41133,6.1.40093,6.1.40093,6.1.40092,6.1.40091,6.1.32831,6.1.32830
|
||||
`rocm-core <https://github.com/ROCm/rocm-core>`_,7.2.4/7.2.3,7.2.2/7.2.1,7.2.0,7.1.1,7.1.0,7.0.2,7.0.1/7.0.0,6.4.3,6.4.2,6.4.1,6.4.0,6.3.3,6.3.2,6.3.1,6.3.0,6.2.4,6.2.2,6.2.1,6.2.0,6.1.5,6.1.2,6.1.1,6.1.0,6.0.2,6.0.0
|
||||
`ROCT-Thunk-Interface <https://github.com/ROCm/ROCT-Thunk-Interface>`_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,20240607.5.7,20240607.5.7,20240607.4.05,20240607.1.4246,20240125.5.08,20240125.5.08,20240125.5.08,20240125.3.30,20231016.2.245,20231016.2.245
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
SYSTEM MGMT TOOLS,.. _tools-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`AMD SMI <amdsmi:index>`,26.2.2,26.2.2,26.2.1,26.2.0,26.1.0,26.0.2,26.0.0,25.5.1,25.5.1,25.4.2,25.3.0,24.7.1,24.7.1,24.7.1,24.7.1,24.6.3,24.6.3,24.6.3,24.6.2,24.5.1,24.5.1,24.5.1,24.4.1,23.4.2,23.4.2
|
||||
:doc:`ROCm Data Center Tool <rdc:index>`,1.2.0,1.2.0,1.2.0,1.2.0,1.2.0,1.1.0,1.1.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0
|
||||
:doc:`rocminfo <rocminfo:index>`,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0
|
||||
:doc:`ROCm SMI <rocm_smi_lib:index>`,7.8.0,7.8.0,7.8.0,7.8.0,7.8.0,7.8.0,7.8.0,7.7.0,7.5.0,7.5.0,7.5.0,7.4.0,7.4.0,7.4.0,7.4.0,7.3.0,7.3.0,7.3.0,7.3.0,7.2.0,7.2.0,7.0.0,7.0.0,6.0.2,6.0.0
|
||||
:doc:`ROCm Validation Suite <rocmvalidationsuite:index>`,1.3.0,1.3.0,1.3.0,1.3.0,1.2.0,1.2.0,1.2.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.0.60204,1.0.60202,1.0.60201,1.0.60200,1.0.60105,1.0.60102,1.0.60101,1.0.60100,1.0.60002,1.0.60000
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
PERFORMANCE TOOLS,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`ROCm Bandwidth Test <rocm_bandwidth_test:index>`,2.6.0,2.6.0,2.6.0,2.6.0,2.6.0,2.6.0,2.6.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0
|
||||
:doc:`ROCm Compute Profiler <rocprofiler-compute:index>`,3.4.0,3.4.0,3.4.0,3.3.1,3.3.0,3.2.3,3.2.3,3.1.1,3.1.1,3.1.0,3.1.0,3.0.0,3.0.0,3.0.0,3.0.0,2.0.1,2.0.1,2.0.1,2.0.1,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
:doc:`ROCm Systems Profiler <rocprofiler-systems:index>`,1.3.0,1.3.0,1.3.0,1.2.1,1.2.0,1.1.1,1.1.0,1.0.2,1.0.2,1.0.1,1.0.0,0.1.2,0.1.1,0.1.0,0.1.0,1.11.2,1.11.2,1.11.2,1.11.2,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
:doc:`ROCProfiler <rocprofiler:index>`,2.0.70204/2.0.70203,2.0.70202/2.0.70201,2.0.70200,2.0.70101,2.0.70100,2.0.70002,2.0.70000,2.0.60403,2.0.60402,2.0.60401,2.0.60400,2.0.60303,2.0.60302,2.0.60301,2.0.60300,2.0.60204,2.0.60202,2.0.60201,2.0.60200,2.0.60105,2.0.60102,2.0.60101,2.0.60100,2.0.60002,2.0.60000
|
||||
:doc:`ROCprofiler-SDK <rocprofiler-sdk:index>`,1.1.0,1.1.0,1.1.0,1.0.0,1.0.0,1.0.0,1.0.0,0.6.0,0.6.0,0.6.0,0.6.0,0.5.0,0.5.0,0.5.0,0.5.0,0.4.0,0.4.0,0.4.0,0.4.0,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
:doc:`ROCTracer <roctracer:index>`,4.1.70204/4.1.70203,4.1.70202/4.1.70201,4.1.70200,4.1.70101,4.1.70100,4.1.70002,4.1.70000,4.1.60403,4.1.60402,4.1.60401,4.1.60400,4.1.60303,4.1.60302,4.1.60301,4.1.60300,4.1.60204,4.1.60202,4.1.60201,4.1.60200,4.1.60105,4.1.60102,4.1.60101,4.1.60100,4.1.60002,4.1.60000
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
DEVELOPMENT TOOLS,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`HIPIFY <hipify:index>`,22.0.0,22.0.0,22.0.0,20.0.0,20.0.0,20.0.0,20.0.0,19.0.0,19.0.0,19.0.0,19.0.0,18.0.0.25012,18.0.0.25012,18.0.0.24491,18.0.0.24455,18.0.0.24392,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483
|
||||
:doc:`ROCm CMake <rocmcmakebuildtools:index>`,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.13.0,0.13.0,0.13.0,0.13.0,0.12.0,0.12.0,0.12.0,0.12.0,0.11.0,0.11.0
|
||||
:doc:`ROCdbgapi <rocdbgapi:index>`,0.77.4,0.77.4,0.77.4,0.77.4,0.77.4,0.77.4,0.77.3,0.77.2,0.77.2,0.77.2,0.77.2,0.77.0,0.77.0,0.77.0,0.77.0,0.76.0,0.76.0,0.76.0,0.76.0,0.71.0,0.71.0,0.71.0,0.71.0,0.71.0,0.71.0
|
||||
:doc:`ROCm Debugger (ROCgdb) <rocgdb:index>`,16.3.0,16.3.0,16.3.0,16.3.0,16.3.0,16.3.0,16.3.0,15.2.0,15.2.0,15.2.0,15.2.0,15.2.0,15.2.0,15.2.0,15.2.0,14.2.0,14.2.0,14.2.0,14.2.0,14.1.0,14.1.0,14.1.0,14.1.0,13.2.0,13.2.0
|
||||
`rocprofiler-register <https://github.com/ROCm/rocprofiler-register>`_,0.5.0,0.5.0,0.5.0,0.5.0,0.5.0,0.5.0,0.5.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.3.0,0.3.0,0.3.0,0.3.0,N/A,N/A
|
||||
:doc:`ROCr Debug Agent <rocr_debug_agent:index>`,2.1.0,2.1.0,2.1.0,2.1.0,2.1.0,2.1.0,2.1.0,2.0.4,2.0.4,2.0.4,2.0.4,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
COMPILERS,.. _compilers-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
`clang-ocl <https://github.com/ROCm/clang-ocl>`_,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,0.5.0,0.5.0,0.5.0,0.5.0,0.5.0,0.5.0
|
||||
:doc:`hipCC <hipcc:index>`,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0
|
||||
`Flang <https://github.com/ROCm/flang>`_,22.0.0.26084,22.0.0.26084,22.0.0.26014,20.0.025444,20.0.025425,20.0.0.25385,20.0.0.25314,19.0.0.25224,19.0.0.25224,19.0.0.25184,19.0.0.25133,18.0.0.25012,18.0.0.25012,18.0.0.24491,18.0.0.24455,18.0.0.24392,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483
|
||||
:doc:`llvm-project <llvm-project:index>`,22.0.0.26084,22.0.0.26084,22.0.0.26014,20.0.025444,20.0.025425,20.0.0.25385,20.0.0.25314,19.0.0.25224,19.0.0.25224,19.0.0.25184,19.0.0.25133,18.0.0.25012,18.0.0.25012,18.0.0.24491,18.0.0.24491,18.0.0.24392,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483
|
||||
`OpenMP <https://github.com/ROCm/llvm-project/tree/amd-staging/openmp>`_,22.0.0.26084,22.0.0.26084,22.0.0.26014,20.0.025444,20.0.025425,20.0.0.25385,20.0.0.25314,19.0.0.25224,19.0.0.25224,19.0.0.25184,19.0.0.25133,18.0.0.25012,18.0.0.25012,18.0.0.24491,18.0.0.24491,18.0.0.24392,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
RUNTIMES,.. _runtime-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`AMD CLR <hip:understand/amd_clr>`,7.2.53211,7.2.53211,7.2.26015,7.1.52802,7.1.25424,7.0.51831,7.0.51830,6.4.43484,6.4.43484,6.4.43483,6.4.43482,6.3.42134,6.3.42134,6.3.42133,6.3.42131,6.2.41134,6.2.41134,6.2.41134,6.2.41133,6.1.40093,6.1.40093,6.1.40092,6.1.40091,6.1.32831,6.1.32830
|
||||
:doc:`HIP <hip:index>`,7.2.53211,7.2.53211,7.2.26015,7.1.52802,7.1.25424,7.0.51831,7.0.51830,6.4.43484,6.4.43484,6.4.43483,6.4.43482,6.3.42134,6.3.42134,6.3.42133,6.3.42131,6.2.41134,6.2.41134,6.2.41134,6.2.41133,6.1.40093,6.1.40093,6.1.40092,6.1.40091,6.1.32831,6.1.32830
|
||||
`OpenCL Runtime <https://github.com/ROCm/clr/tree/develop/opencl>`_,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0
|
||||
:doc:`ROCr Runtime <rocr-runtime:index>`,1.18.0,1.18.0,1.18.0,1.18.0,1.18.0,1.18.0,1.18.0,1.15.0,1.15.0,1.15.0,1.15.0,1.14.0,1.14.0,1.14.0,1.14.0,1.14.0,1.14.0,1.14.0,1.13.0,1.13.0,1.13.0,1.13.0,1.13.0,1.12.0,1.12.0
|
||||
ROCm Version,7.2.1,7.2.0,7.1.1,7.1.0,7.0.2,7.0.1/7.0.0,6.4.3,6.4.2,6.4.1,6.4.0,6.3.3,6.3.2,6.3.1,6.3.0,6.2.4,6.2.2,6.2.1,6.2.0, 6.1.5, 6.1.2, 6.1.1, 6.1.0, 6.0.2, 6.0.0
|
||||
:ref:`Operating systems & kernels <OS-kernel-versions>` [#os-compatibility-past-60]_,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.3,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,Ubuntu 24.04.2,"Ubuntu 24.04.1, 24.04","Ubuntu 24.04.1, 24.04","Ubuntu 24.04.1, 24.04",Ubuntu 24.04,,,,,,
|
||||
,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5,"Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4","Ubuntu 22.04.5, 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3","Ubuntu 22.04.4, 22.04.3, 22.04.2","Ubuntu 22.04.4, 22.04.3, 22.04.2"
|
||||
,,,,,,,,,,,,,,,,,,,"Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5","Ubuntu 20.04.6, 20.04.5"
|
||||
,"RHEL 10.1, 10.0, 9.7, 9.6, 9.4","RHEL 10.1, 10.0, 9.7, 9.6, 9.4","RHEL 10.1, 10.0, 9.7, 9.6, 9.4","RHEL 10.0, 9.6, 9.4","RHEL 10.0, 9.6, 9.4","RHEL 9.6, 9.4","RHEL 9.6, 9.4","RHEL 9.6, 9.4","RHEL 9.6, 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.5, 9.4","RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4, 9.3","RHEL 9.4, 9.3, 9.2","RHEL 9.4, 9.3, 9.2","RHEL 9.4, 9.3, 9.2","RHEL 9.4, 9.3, 9.2","RHEL 9.3, 9.2","RHEL 9.3, 9.2"
|
||||
,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,RHEL 8.10,"RHEL 8.10, 8.9","RHEL 8.10, 8.9","RHEL 8.10, 8.9","RHEL 8.10, 8.9","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8","RHEL 8.9, 8.8"
|
||||
,SLES 15 SP7,SLES 15 SP7,SLES 15 SP7,SLES 15 SP7,SLES 15 SP7,SLES 15 SP7,"SLES 15 SP7, SP6","SLES 15 SP7, SP6",SLES 15 SP6,SLES 15 SP6,"SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP6, SP5","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4","SLES 15 SP5, SP4"
|
||||
,,,,,,,,,,,,,,,,,,,,CentOS 7.9,CentOS 7.9,CentOS 7.9,CentOS 7.9,CentOS 7.9
|
||||
,"Oracle Linux 10, 9, 8","Oracle Linux 10, 9, 8","Oracle Linux 10, 9, 8","Oracle Linux 10, 9, 8","Oracle Linux 10, 9, 8","Oracle Linux 9, 8","Oracle Linux 9, 8","Oracle Linux 9, 8","Oracle Linux 9, 8","Oracle Linux 9, 8",Oracle Linux 8.10,Oracle Linux 8.10,Oracle Linux 8.10,Oracle Linux 8.10,Oracle Linux 8.9,Oracle Linux 8.9,Oracle Linux 8.9,Oracle Linux 8.9,Oracle Linux 8.9,Oracle Linux 8.9,Oracle Linux 8.9,,,
|
||||
,"Debian 13, 12","Debian 13, 12","Debian 13, 12","Debian 13, 12","Debian 13, 12",Debian 12,Debian 12,Debian 12,Debian 12,Debian 12,Debian 12,Debian 12,Debian 12,,,,,,,,,,,
|
||||
,,,,,Azure Linux 3.0,Azure Linux 3.0,Azure Linux 3.0,Azure Linux 3.0,Azure Linux 3.0,Azure Linux 3.0,Azure Linux 3.0,Azure Linux 3.0,,,,,,,,,,,,
|
||||
,Rocky Linux 9,Rocky Linux 9,Rocky Linux 9,Rocky Linux 9,Rocky Linux 9,Rocky Linux 9,,,,,,,,,,,,,,,,,,
|
||||
,.. _architecture-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`Architecture <rocm-install-on-linux:reference/system-requirements>`,CDNA4,CDNA4,CDNA4,CDNA4,CDNA4,CDNA4,,,,,,,,,,,,,,,,,,
|
||||
,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3,CDNA3
|
||||
,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2,CDNA2
|
||||
,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA,CDNA
|
||||
,RDNA4,RDNA4,RDNA4,RDNA4,RDNA4,RDNA4,RDNA4,RDNA4,RDNA4,,,,,,,,,,,,,,,
|
||||
,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3,RDNA3
|
||||
,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2,RDNA2
|
||||
,.. _gpu-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`GPU / LLVM target <rocm-install-on-linux:reference/system-requirements>` [#gpu-compatibility-past-60]_,gfx950,gfx950,gfx950,gfx950,gfx950,gfx950,,,,,,,,,,,,,,,,,,
|
||||
,gfx1201,gfx1201,gfx1201,gfx1201,gfx1201,gfx1201,gfx1201,gfx1201,gfx1201,,,,,,,,,,,,,,,
|
||||
,gfx1200,gfx1200,gfx1200,gfx1200,gfx1200,gfx1200,gfx1200,gfx1200,gfx1200,,,,,,,,,,,,,,,
|
||||
,gfx1101,gfx1101,gfx1101,gfx1101,gfx1101,gfx1101,gfx1101,gfx1101,gfx1101,,,,,,,,,,,,,,,
|
||||
,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100,gfx1100
|
||||
,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030,gfx1030
|
||||
,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942,gfx942, gfx942, gfx942, gfx942, gfx942, gfx942, gfx942
|
||||
,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a,gfx90a
|
||||
,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908,gfx908
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
FRAMEWORK SUPPORT,.. _framework-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`PyTorch <../compatibility/ml-compatibility/pytorch-compatibility>`,"2.9.1, 2.8.0, 2.7.1","2.9.1, 2.8.0, 2.7.1","2.9, 2.8, 2.7","2.8, 2.7, 2.6","2.8, 2.7, 2.6","2.7, 2.6, 2.5","2.6, 2.5, 2.4, 2.3","2.6, 2.5, 2.4, 2.3","2.6, 2.5, 2.4, 2.3","2.6, 2.5, 2.4, 2.3","2.4, 2.3, 2.2, 1.13","2.4, 2.3, 2.2, 1.13","2.4, 2.3, 2.2, 1.13","2.4, 2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.3, 2.2, 2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13","2.1, 2.0, 1.13"
|
||||
:doc:`TensorFlow <../compatibility/ml-compatibility/tensorflow-compatibility>`,"2.20.0, 2.19.1, 2.18.1","2.20.0, 2.19.1, 2.18.1","2.20.0, 2.19.1, 2.18.1","2.20.0, 2.19.1, 2.18.1","2.19.1, 2.18.1, 2.17.1 [#tf-mi350-past-60]_","2.19.1, 2.18.1, 2.17.1 [#tf-mi350-past-60]_","2.18.1, 2.17.1, 2.16.2","2.18.1, 2.17.1, 2.16.2","2.18.1, 2.17.1, 2.16.2","2.18.1, 2.17.1, 2.16.2","2.17.0, 2.16.2, 2.15.1","2.17.0, 2.16.2, 2.15.1","2.17.0, 2.16.2, 2.15.1","2.17.0, 2.16.2, 2.15.1","2.16.1, 2.15.1, 2.14.1","2.16.1, 2.15.1, 2.14.1","2.16.1, 2.15.1, 2.14.1","2.16.1, 2.15.1, 2.14.1","2.15.0, 2.14.0, 2.13.1","2.15.0, 2.14.0, 2.13.1","2.15.0, 2.14.0, 2.13.1","2.15.0, 2.14.0, 2.13.1","2.14.0, 2.13.1, 2.12.1","2.14.0, 2.13.1, 2.12.1"
|
||||
:doc:`JAX <../compatibility/ml-compatibility/jax-compatibility>`,0.8.2,0.8.0,0.7.1,0.7.1,0.6.0,0.6.0,0.4.35,0.4.35,0.4.35,0.4.35,0.4.31,0.4.31,0.4.31,0.4.31,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26,0.4.26
|
||||
:doc:`DGL <../compatibility/ml-compatibility/dgl-compatibility>` [#dgl_compat-past-60]_,N/A,N/A,N/A,N/A,N/A,2.4.0,2.4.0,N/A,N/A,2.4.0,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
`ONNX Runtime <https://onnxruntime.ai/docs/build/eps.html#amd-migraphx>`_,1.23.2,1.23.2,1.23.1,1.22.0,1.22.0,1.22.0,1.20.0,1.20.0,1.20.0,1.20.0,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.17.3,1.14.1,1.14.1
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
THIRD PARTY COMMS,.. _thirdpartycomms-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,
|
||||
`UCC <https://github.com/ROCm/ucc>`_,>=1.6.0,>=1.4.0,>=1.4.0,>=1.4.0,>=1.4.0,>=1.4.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.3.0,>=1.2.0,>=1.2.0
|
||||
`UCX <https://github.com/ROCm/ucx>`_,>=1.17.0,>=1.17.0,>=1.17.0,>=1.17.0,>=1.17.0,>=1.17.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.15.0,>=1.14.1,>=1.14.1,>=1.14.1,>=1.14.1,>=1.14.1,>=1.14.1
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
THIRD PARTY ALGORITHM,.. _thirdpartyalgorithm-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,
|
||||
Thrust,2.8.5,2.8.5,2.8.5,2.8.5,2.6.0,2.6.0,2.5.0,2.5.0,2.5.0,2.5.0,2.3.2,2.3.2,2.3.2,2.3.2,2.2.0,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.1.0,2.0.1,2.0.1
|
||||
CUB,2.8.5,2.8.5,2.8.5,2.8.5,2.6.0,2.6.0,2.5.0,2.5.0,2.5.0,2.5.0,2.3.2,2.3.2,2.3.2,2.3.2,2.2.0,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.1.0,2.0.1,2.0.1
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
DRIVER & USER SPACE [#kfd_support-past-60]_,.. _kfd-userspace-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`AMD GPU Driver <rocm-install-on-linux:reference/user-kernel-space-compat-matrix>`,"30.30.1, 30.30.0, 30.20.1, 30.20.0 [#mi325x_KVM-past-60]_, 30.10.2, 30.10.1 [#driver_patch-past-60]_, 30.10, 6.4.x","30.30.0, 30.20.1, 30.20.0 [#mi325x_KVM-past-60]_, 30.10.2, 30.10.1 [#driver_patch-past-60]_, 30.10, 6.4.x","30.20.1, 30.20.0 [#mi325x_KVM-past-60]_, 30.10.2, 30.10.1 [#driver_patch-past-60]_, 30.10, 6.4.x","30.20.0 [#mi325x_KVM-past-60]_, 30.10.2, 30.10.1 [#driver_patch-past-60]_, 30.10, 6.4.x","30.10.2, 30.10.1 [#driver_patch-past-60]_, 30.10, 6.4.x, 6.3.x","30.10.1 [#driver_patch-past-60]_, 30.10, 6.4.x, 6.3.x, 6.2.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x, 5.7.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x, 5.7.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x, 5.7.x","6.4.x, 6.3.x, 6.2.x, 6.1.x, 6.0.x, 5.7.x","6.2.x, 6.1.x, 6.0.x, 5.7.x, 5.6.x","6.2.x, 6.1.x, 6.0.x, 5.7.x, 5.6.x"
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
ML & COMPUTER VISION,.. _mllibs-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`Composable Kernel <composable_kernel:index>`,1.2.0,1.2.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0
|
||||
:doc:`MIGraphX <amdmigraphx:index>`,2.15.0,2.15.0,2.14.0,2.14.0,2.13.0,2.13.0,2.12.0,2.12.0,2.12.0,2.12.0,2.11.0,2.11.0,2.11.0,2.11.0,2.10.0,2.10.0,2.10.0,2.10.0,2.9.0,2.9.0,2.9.0,2.9.0,2.8.0,2.8.0
|
||||
:doc:`MIOpen <miopen:index>`,3.5.1,3.5.1,3.5.1,3.5.1,3.5.0,3.5.0,3.4.0,3.4.0,3.4.0,3.4.0,3.3.0,3.3.0,3.3.0,3.3.0,3.2.0,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0
|
||||
:doc:`MIVisionX <mivisionx:index>`,3.5.0,3.5.0,3.4.0,3.4.0,3.3.0,3.3.0,3.2.0,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0,3.0.0,3.0.0,2.5.0,2.5.0,2.5.0,2.5.0,2.5.0,2.5.0
|
||||
:doc:`rocAL <rocal:index>`,2.5.0,2.5.0,2.4.0,2.4.0,2.3.0,2.3.0,2.2.0,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.1.0,2.0.0,2.0.0,2.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0
|
||||
:doc:`rocDecode <rocdecode:index>`,1.7.0,1.5.0,1.4.0,1.4.0,1.0.0,1.0.0,0.10.0,0.10.0,0.10.0,0.10.0,0.8.0,0.8.0,0.8.0,0.8.0,0.6.0,0.6.0,0.6.0,0.6.0,0.6.0,0.6.0,0.5.0,0.5.0,N/A,N/A
|
||||
:doc:`rocJPEG <rocjpeg:index>`,1.4.0,1.3.0,1.2.0,1.2.0,1.1.0,1.1.0,0.8.0,0.8.0,0.8.0,0.8.0,0.6.0,0.6.0,0.6.0,0.6.0,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
:doc:`rocPyDecode <rocpydecode:index>`,0.8.0,0.8.0,0.7.0,0.7.0,0.6.0,0.6.0,0.3.1,0.3.1,0.3.1,0.3.1,0.2.0,0.2.0,0.2.0,0.2.0,0.1.0,0.1.0,0.1.0,0.1.0,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
:doc:`RPP <rpp:index>`,2.2.1,2.2.0,2.1.0,2.1.0,2.0.0,2.0.0,1.9.10,1.9.10,1.9.10,1.9.10,1.9.1,1.9.1,1.9.1,1.9.1,1.8.0,1.8.0,1.8.0,1.8.0,1.5.0,1.5.0,1.5.0,1.5.0,1.4.0,1.4.0
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
COMMUNICATION,.. _commlibs-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`RCCL <rccl:index>`,2.27.7,2.27.7,2.27.7,2.27.7,2.26.6,2.26.6,2.22.3,2.22.3,2.22.3,2.22.3,2.21.5,2.21.5,2.21.5,2.21.5,2.20.5,2.20.5,2.20.5,2.20.5,2.18.6,2.18.6,2.18.6,2.18.6,2.18.3,2.18.3
|
||||
:doc:`rocSHMEM <rocshmem:index>`,3.2.0,3.2.0,3.1.0,3.0.0,3.0.0,3.0.0,2.0.1,2.0.1,2.0.0,2.0.0,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
MATH LIBS,.. _mathlibs-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,
|
||||
`half <https://github.com/ROCm/half>`_ ,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0,1.12.0
|
||||
:doc:`hipBLAS <hipblas:index>`,3.2.0,3.2.0,3.1.0,3.1.0,3.0.2,3.0.0,2.4.0,2.4.0,2.4.0,2.4.0,2.3.0,2.3.0,2.3.0,2.3.0,2.2.0,2.2.0,2.2.0,2.2.0,2.1.0,2.1.0,2.1.0,2.1.0,2.0.0,2.0.0
|
||||
:doc:`hipBLASLt <hipblaslt:index>`,1.2.2,1.2.1,1.1.0,1.1.0,1.0.0,1.0.0,0.12.1,0.12.1,0.12.1,0.12.0,0.10.0,0.10.0,0.10.0,0.10.0,0.8.0,0.8.0,0.8.0,0.8.0,0.7.0,0.7.0,0.7.0,0.7.0,0.6.0,0.6.0
|
||||
:doc:`hipFFT <hipfft:index>`,1.0.22,1.0.22,1.0.21,1.0.21,1.0.20,1.0.20,1.0.18,1.0.18,1.0.18,1.0.18,1.0.17,1.0.17,1.0.17,1.0.17,1.0.16,1.0.15,1.0.15,1.0.14,1.0.14,1.0.14,1.0.14,1.0.14,1.0.13,1.0.13
|
||||
:doc:`hipfort <hipfort:index>`,0.7.1,0.7.1,0.7.1,0.7.1,0.7.0,0.7.0,0.6.0,0.6.0,0.6.0,0.6.0,0.5.1,0.5.1,0.5.0,0.5.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0
|
||||
:doc:`hipRAND <hiprand:index>`,3.1.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0,2.12.0,2.12.0,2.12.0,2.12.0,2.11.1,2.11.1,2.11.1,2.11.0,2.11.1,2.11.0,2.11.0,2.11.0,2.10.16,2.10.16,2.10.16,2.10.16,2.10.16,2.10.16
|
||||
:doc:`hipSOLVER <hipsolver:index>`,3.2.0,3.2.0,3.1.0,3.1.0,3.0.0,3.0.0,2.4.0,2.4.0,2.4.0,2.4.0,2.3.0,2.3.0,2.3.0,2.3.0,2.2.0,2.2.0,2.2.0,2.2.0,2.1.1,2.1.1,2.1.1,2.1.0,2.0.0,2.0.0
|
||||
:doc:`hipSPARSE <hipsparse:index>`,4.2.0,4.2.0,4.1.0,4.1.0,4.0.1,4.0.1,3.2.0,3.2.0,3.2.0,3.2.0,3.1.2,3.1.2,3.1.2,3.1.2,3.1.1,3.1.1,3.1.1,3.1.1,3.0.1,3.0.1,3.0.1,3.0.1,3.0.0,3.0.0
|
||||
:doc:`hipSPARSELt <hipsparselt:index>`,0.2.6,0.2.6,0.2.5,0.2.5,0.2.4,0.2.4,0.2.3,0.2.3,0.2.3,0.2.3,0.2.2,0.2.2,0.2.2,0.2.2,0.2.1,0.2.1,0.2.1,0.2.1,0.2.0,0.2.0,0.1.0,0.1.0,0.1.0,0.1.0
|
||||
:doc:`rocALUTION <rocalution:index>`,4.1.0,4.1.0,4.0.1,4.0.1,4.0.0,4.0.0,3.2.3,3.2.3,3.2.3,3.2.2,3.2.1,3.2.1,3.2.1,3.2.1,3.2.1,3.2.0,3.2.0,3.2.0,3.1.1,3.1.1,3.1.1,3.1.1,3.0.3,3.0.3
|
||||
:doc:`rocBLAS <rocblas:index>`,5.2.0,5.2.0,5.1.1,5.1.0,5.0.2,5.0.0,4.4.1,4.4.1,4.4.0,4.4.0,4.3.0,4.3.0,4.3.0,4.3.0,4.2.4,4.2.1,4.2.1,4.2.0,4.1.2,4.1.2,4.1.0,4.1.0,4.0.0,4.0.0
|
||||
:doc:`rocFFT <rocfft:index>`,1.0.36,1.0.36,1.0.35,1.0.35,1.0.34,1.0.34,1.0.32,1.0.32,1.0.32,1.0.32,1.0.31,1.0.31,1.0.31,1.0.31,1.0.30,1.0.29,1.0.29,1.0.28,1.0.27,1.0.27,1.0.27,1.0.26,1.0.25,1.0.23
|
||||
:doc:`rocRAND <rocrand:index>`,4.2.0,4.2.0,4.1.0,4.1.0,4.0.0,4.0.0,3.3.0,3.3.0,3.3.0,3.3.0,3.2.0,3.2.0,3.2.0,3.2.0,3.1.1,3.1.0,3.1.0,3.1.0,3.0.1,3.0.1,3.0.1,3.0.1,3.0.0,2.10.17
|
||||
:doc:`rocSOLVER <rocsolver:index>`,3.32.0,3.32.0,3.31.0,3.31.0,3.30.1,3.30.0,3.28.2,3.28.2,3.28.0,3.28.0,3.27.0,3.27.0,3.27.0,3.27.0,3.26.2,3.26.0,3.26.0,3.26.0,3.25.0,3.25.0,3.25.0,3.25.0,3.24.0,3.24.0
|
||||
:doc:`rocSPARSE <rocsparse:index>`,4.2.0,4.2.0,4.1.0,4.1.0,4.0.2,4.0.2,3.4.0,3.4.0,3.4.0,3.4.0,3.3.0,3.3.0,3.3.0,3.3.0,3.2.1,3.2.0,3.2.0,3.2.0,3.1.2,3.1.2,3.1.2,3.1.2,3.0.2,3.0.2
|
||||
:doc:`rocWMMA <rocwmma:index>`,2.2.0,2.2.0,2.1.0,2.0.0,2.0.0,2.0.0,1.7.0,1.7.0,1.7.0,1.7.0,1.6.0,1.6.0,1.6.0,1.6.0,1.5.0,1.5.0,1.5.0,1.5.0,1.4.0,1.4.0,1.4.0,1.4.0,1.3.0,1.3.0
|
||||
:doc:`Tensile <tensile:src/index>`,4.45.0,4.45.0,4.44.0,4.44.0,4.44.0,4.44.0,4.43.0,4.43.0,4.43.0,4.43.0,4.42.0,4.42.0,4.42.0,4.42.0,4.41.0,4.41.0,4.41.0,4.41.0,4.40.0,4.40.0,4.40.0,4.40.0,4.39.0,4.39.0
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
PRIMITIVES,.. _primitivelibs-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`hipCUB <hipcub:index>`,4.2.0,4.2.0,4.1.0,4.1.0,4.0.0,4.0.0,3.4.0,3.4.0,3.4.0,3.4.0,3.3.0,3.3.0,3.3.0,3.3.0,3.2.1,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0
|
||||
:doc:`hipTensor <hiptensor:index>`,2.2.0,2.2.0,2.0.0,2.0.0,2.0.0,2.0.0,1.5.0,1.5.0,1.5.0,1.5.0,1.4.0,1.4.0,1.4.0,1.4.0,1.3.0,1.3.0,1.3.0,1.3.0,1.2.0,1.2.0,1.2.0,1.2.0,1.1.0,1.1.0
|
||||
:doc:`rocPRIM <rocprim:index>`,4.2.0,4.2.0,4.1.0,4.1.0,4.0.1,4.0.0,3.4.1,3.4.1,3.4.0,3.4.0,3.3.0,3.3.0,3.3.0,3.3.0,3.2.2,3.2.0,3.2.0,3.2.0,3.1.0,3.1.0,3.1.0,3.1.0,3.0.0,3.0.0
|
||||
:doc:`rocThrust <rocthrust:index>`,4.2.0,4.2.0,4.1.0,4.1.0,4.0.0,4.0.0,3.3.0,3.3.0,3.3.0,3.3.0,3.3.0,3.3.0,3.3.0,3.3.0,3.1.1,3.1.0,3.1.0,3.0.1,3.0.1,3.0.1,3.0.1,3.0.1,3.0.0,3.0.0
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
SUPPORT LIBS,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
`hipother <https://github.com/ROCm/hipother>`_,7.2.53211,7.2.26015,7.1.52802,7.1.25424,7.0.51831,7.0.51830,6.4.43483,6.4.43483,6.4.43483,6.4.43482,6.3.42134,6.3.42134,6.3.42133,6.3.42131,6.2.41134,6.2.41134,6.2.41134,6.2.41133,6.1.40093,6.1.40093,6.1.40092,6.1.40091,6.1.32831,6.1.32830
|
||||
`rocm-core <https://github.com/ROCm/rocm-core>`_,7.2.1,7.2.0,7.1.1,7.1.0,7.0.2,7.0.1/7.0.0,6.4.3,6.4.2,6.4.1,6.4.0,6.3.3,6.3.2,6.3.1,6.3.0,6.2.4,6.2.2,6.2.1,6.2.0,6.1.5,6.1.2,6.1.1,6.1.0,6.0.2,6.0.0
|
||||
`ROCT-Thunk-Interface <https://github.com/ROCm/ROCT-Thunk-Interface>`_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,N/A [#ROCT-rocr-past-60]_,20240607.5.7,20240607.5.7,20240607.4.05,20240607.1.4246,20240125.5.08,20240125.5.08,20240125.5.08,20240125.3.30,20231016.2.245,20231016.2.245
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
SYSTEM MGMT TOOLS,.. _tools-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`AMD SMI <amdsmi:index>`,26.2.2,26.2.1,26.2.0,26.1.0,26.0.2,26.0.0,25.5.1,25.5.1,25.4.2,25.3.0,24.7.1,24.7.1,24.7.1,24.7.1,24.6.3,24.6.3,24.6.3,24.6.2,24.5.1,24.5.1,24.5.1,24.4.1,23.4.2,23.4.2
|
||||
:doc:`ROCm Data Center Tool <rdc:index>`,1.2.0,1.2.0,1.2.0,1.2.0,1.1.0,1.1.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0,0.3.0
|
||||
:doc:`rocminfo <rocminfo:index>`,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0
|
||||
:doc:`ROCm SMI <rocm_smi_lib:index>`,7.8.0,7.8.0,7.8.0,7.8.0,7.8.0,7.8.0,7.7.0,7.5.0,7.5.0,7.5.0,7.4.0,7.4.0,7.4.0,7.4.0,7.3.0,7.3.0,7.3.0,7.3.0,7.2.0,7.2.0,7.0.0,7.0.0,6.0.2,6.0.0
|
||||
:doc:`ROCm Validation Suite <rocmvalidationsuite:index>`,1.3.0,1.3.0,1.3.0,1.2.0,1.2.0,1.2.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.1.0,1.0.60204,1.0.60202,1.0.60201,1.0.60200,1.0.60105,1.0.60102,1.0.60101,1.0.60100,1.0.60002,1.0.60000
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
PERFORMANCE TOOLS,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`ROCm Bandwidth Test <rocm_bandwidth_test:index>`,2.6.0,2.6.0,2.6.0,2.6.0,2.6.0,2.6.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0,1.4.0
|
||||
:doc:`ROCm Compute Profiler <rocprofiler-compute:index>`,3.4.0,3.4.0,3.3.1,3.3.0,3.2.3,3.2.3,3.1.1,3.1.1,3.1.0,3.1.0,3.0.0,3.0.0,3.0.0,3.0.0,2.0.1,2.0.1,2.0.1,2.0.1,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
:doc:`ROCm Systems Profiler <rocprofiler-systems:index>`,1.3.0,1.3.0,1.2.1,1.2.0,1.1.1,1.1.0,1.0.2,1.0.2,1.0.1,1.0.0,0.1.2,0.1.1,0.1.0,0.1.0,1.11.2,1.11.2,1.11.2,1.11.2,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
:doc:`ROCProfiler <rocprofiler:index>`,2.0.70201,2.0.70200,2.0.70101,2.0.70100,2.0.70002,2.0.70000,2.0.60403,2.0.60402,2.0.60401,2.0.60400,2.0.60303,2.0.60302,2.0.60301,2.0.60300,2.0.60204,2.0.60202,2.0.60201,2.0.60200,2.0.60105,2.0.60102,2.0.60101,2.0.60100,2.0.60002,2.0.60000
|
||||
:doc:`ROCprofiler-SDK <rocprofiler-sdk:index>`,1.1.0,1.1.0,1.0.0,1.0.0,1.0.0,1.0.0,0.6.0,0.6.0,0.6.0,0.6.0,0.5.0,0.5.0,0.5.0,0.5.0,0.4.0,0.4.0,0.4.0,0.4.0,N/A,N/A,N/A,N/A,N/A,N/A
|
||||
:doc:`ROCTracer <roctracer:index>`,4.1.70201,4.1.70200,4.1.70101,4.1.70100,4.1.70002,4.1.70000,4.1.60403,4.1.60402,4.1.60401,4.1.60400,4.1.60303,4.1.60302,4.1.60301,4.1.60300,4.1.60204,4.1.60202,4.1.60201,4.1.60200,4.1.60105,4.1.60102,4.1.60101,4.1.60100,4.1.60002,4.1.60000
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
DEVELOPMENT TOOLS,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`HIPIFY <hipify:index>`,22.0.0,22.0.0,20.0.0,20.0.0,20.0.0,20.0.0,19.0.0,19.0.0,19.0.0,19.0.0,18.0.0.25012,18.0.0.25012,18.0.0.24491,18.0.0.24455,18.0.0.24392,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483
|
||||
:doc:`ROCm CMake <rocmcmakebuildtools:index>`,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.14.0,0.13.0,0.13.0,0.13.0,0.13.0,0.12.0,0.12.0,0.12.0,0.12.0,0.11.0,0.11.0
|
||||
:doc:`ROCdbgapi <rocdbgapi:index>`,0.77.4,0.77.4,0.77.4,0.77.4,0.77.4,0.77.3,0.77.2,0.77.2,0.77.2,0.77.2,0.77.0,0.77.0,0.77.0,0.77.0,0.76.0,0.76.0,0.76.0,0.76.0,0.71.0,0.71.0,0.71.0,0.71.0,0.71.0,0.71.0
|
||||
:doc:`ROCm Debugger (ROCgdb) <rocgdb:index>`,16.3.0,16.3.0,16.3.0,16.3.0,16.3.0,16.3.0,15.2.0,15.2.0,15.2.0,15.2.0,15.2.0,15.2.0,15.2.0,15.2.0,14.2.0,14.2.0,14.2.0,14.2.0,14.1.0,14.1.0,14.1.0,14.1.0,13.2.0,13.2.0
|
||||
`rocprofiler-register <https://github.com/ROCm/rocprofiler-register>`_,0.5.0,0.5.0,0.5.0,0.5.0,0.5.0,0.5.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.4.0,0.3.0,0.3.0,0.3.0,0.3.0,N/A,N/A
|
||||
:doc:`ROCr Debug Agent <rocr_debug_agent:index>`,2.1.0,2.1.0,2.1.0,2.1.0,2.1.0,2.1.0,2.0.4,2.0.4,2.0.4,2.0.4,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3,2.0.3
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
COMPILERS,.. _compilers-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,
|
||||
`clang-ocl <https://github.com/ROCm/clang-ocl>`_,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,0.5.0,0.5.0,0.5.0,0.5.0,0.5.0,0.5.0
|
||||
:doc:`hipCC <hipcc:index>`,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.1.1,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0,1.0.0
|
||||
`Flang <https://github.com/ROCm/flang>`_,22.0.0.26084,22.0.0.26014,20.0.025444,20.0.025425,20.0.0.25385,20.0.0.25314,19.0.0.25224,19.0.0.25224,19.0.0.25184,19.0.0.25133,18.0.0.25012,18.0.0.25012,18.0.0.24491,18.0.0.24455,18.0.0.24392,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483
|
||||
:doc:`llvm-project <llvm-project:index>`,22.0.0.26084,22.0.0.26014,20.0.025444,20.0.025425,20.0.0.25385,20.0.0.25314,19.0.0.25224,19.0.0.25224,19.0.0.25184,19.0.0.25133,18.0.0.25012,18.0.0.25012,18.0.0.24491,18.0.0.24491,18.0.0.24392,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483
|
||||
`OpenMP <https://github.com/ROCm/llvm-project/tree/amd-staging/openmp>`_,22.0.0.26084,22.0.0.26014,20.0.025444,20.0.025425,20.0.0.25385,20.0.0.25314,19.0.0.25224,19.0.0.25224,19.0.0.25184,19.0.0.25133,18.0.0.25012,18.0.0.25012,18.0.0.24491,18.0.0.24491,18.0.0.24392,18.0.0.24355,18.0.0.24355,18.0.0.24232,17.0.0.24193,17.0.0.24193,17.0.0.24154,17.0.0.24103,17.0.0.24012,17.0.0.23483
|
||||
,,,,,,,,,,,,,,,,,,,,,,,,
|
||||
RUNTIMES,.. _runtime-support-compatibility-matrix-past-60:,,,,,,,,,,,,,,,,,,,,,,,
|
||||
:doc:`AMD CLR <hip:understand/amd_clr>`,7.2.53211,7.2.26015,7.1.52802,7.1.25424,7.0.51831,7.0.51830,6.4.43484,6.4.43484,6.4.43483,6.4.43482,6.3.42134,6.3.42134,6.3.42133,6.3.42131,6.2.41134,6.2.41134,6.2.41134,6.2.41133,6.1.40093,6.1.40093,6.1.40092,6.1.40091,6.1.32831,6.1.32830
|
||||
:doc:`HIP <hip:index>`,7.2.53211,7.2.26015,7.1.52802,7.1.25424,7.0.51831,7.0.51830,6.4.43484,6.4.43484,6.4.43483,6.4.43482,6.3.42134,6.3.42134,6.3.42133,6.3.42131,6.2.41134,6.2.41134,6.2.41134,6.2.41133,6.1.40093,6.1.40093,6.1.40092,6.1.40091,6.1.32831,6.1.32830
|
||||
`OpenCL Runtime <https://github.com/ROCm/clr/tree/develop/opencl>`_,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0,2.0.0
|
||||
:doc:`ROCr Runtime <rocr-runtime:index>`,1.18.0,1.18.0,1.18.0,1.18.0,1.18.0,1.18.0,1.15.0,1.15.0,1.15.0,1.15.0,1.14.0,1.14.0,1.14.0,1.14.0,1.14.0,1.14.0,1.14.0,1.13.0,1.13.0,1.13.0,1.13.0,1.13.0,1.12.0,1.12.0
|
||||
|
||||
|
@@ -22,30 +22,31 @@ compatibility and system requirements.
|
||||
.. container:: format-big-table
|
||||
|
||||
.. csv-table::
|
||||
:header: "ROCm Version", "7.2.4/7.2.3", "7.2.2/7.2.1", "7.1.0"
|
||||
:header: "ROCm Version", "7.2.1", "7.2.0", "6.4.0"
|
||||
:stub-columns: 1
|
||||
|
||||
:ref:`Operating systems & kernels <OS-kernel-versions>` [#os-compatibility]_,Ubuntu 24.04.4,Ubuntu 24.04.4,Ubuntu 24.04.3
|
||||
:ref:`Operating systems & kernels <OS-kernel-versions>` [#os-compatibility]_,Ubuntu 24.04.4,Ubuntu 24.04.3,Ubuntu 24.04.2
|
||||
,Ubuntu 22.04.5,Ubuntu 22.04.5,Ubuntu 22.04.5
|
||||
,"RHEL 10.1, 10.0, |br| 9.7, 9.6, 9.4","RHEL 10.1, 10.0, |br| 9.7, 9.6, 9.4","RHEL 10.0, 9.6, 9.4"
|
||||
,"RHEL 10.1, 10.0, 9.7, 9.6, 9.4","RHEL 10.1, 10.0, 9.7, 9.6, 9.4","RHEL 9.5, 9.4"
|
||||
,RHEL 8.10,RHEL 8.10,RHEL 8.10
|
||||
,SLES 15 SP7,SLES 15 SP7,SLES 15 SP7
|
||||
,"Oracle Linux 10, 9, 8","Oracle Linux 10, 9, 8","Oracle Linux 10, 9, 8"
|
||||
,"Debian 13, 12","Debian 13, 12","Debian 13, 12"
|
||||
,Rocky Linux 9,Rocky Linux 9,Rocky Linux 9
|
||||
,SLES 15 SP7,SLES 15 SP7,SLES 15 SP6
|
||||
,"Oracle Linux 10, 9, 8","Oracle Linux 10, 9, 8","Oracle Linux 9, 8"
|
||||
,"Debian 13, 12","Debian 13, 12",Debian 12
|
||||
,,,Azure Linux 3.0
|
||||
,Rocky Linux 9,Rocky Linux 9,
|
||||
,.. _architecture-support-compatibility-matrix:,,
|
||||
:doc:`Architecture <rocm-install-on-linux:reference/system-requirements>`,CDNA4,CDNA4,CDNA4
|
||||
:doc:`Architecture <rocm-install-on-linux:reference/system-requirements>`,CDNA4,CDNA4,
|
||||
,CDNA3,CDNA3,CDNA3
|
||||
,CDNA2,CDNA2,CDNA2
|
||||
,CDNA,CDNA,CDNA
|
||||
,RDNA4,RDNA4,RDNA4
|
||||
,RDNA4,RDNA4,
|
||||
,RDNA3,RDNA3,RDNA3
|
||||
,RDNA2,RDNA2,RDNA2
|
||||
,.. _gpu-support-compatibility-matrix:,,
|
||||
:doc:`GPU / LLVM target <rocm-install-on-linux:reference/system-requirements>` [#gpu-compatibility]_,gfx950,gfx950,gfx950
|
||||
,gfx1201,gfx1201,gfx1201
|
||||
,gfx1200,gfx1200,gfx1200
|
||||
,gfx1101,gfx1101,gfx1101
|
||||
:doc:`GPU / LLVM target <rocm-install-on-linux:reference/system-requirements>` [#gpu-compatibility]_,gfx950,gfx950,
|
||||
,gfx1201,gfx1201,
|
||||
,gfx1200,gfx1200,
|
||||
,gfx1101,gfx1101,
|
||||
,gfx1100,gfx1100,gfx1100
|
||||
,gfx1030,gfx1030,gfx1030
|
||||
,gfx942,gfx942,gfx942
|
||||
@@ -53,107 +54,110 @@ compatibility and system requirements.
|
||||
,gfx908,gfx908,gfx908
|
||||
,,,
|
||||
FRAMEWORK SUPPORT,.. _framework-support-compatibility-matrix:,,
|
||||
:doc:`PyTorch <../compatibility/ml-compatibility/pytorch-compatibility>`,"2.9.1, 2.8.0, 2.7.1","2.9.1, 2.8.0, 2.7.1","2.8, 2.7, 2.6"
|
||||
:doc:`TensorFlow <../compatibility/ml-compatibility/tensorflow-compatibility>`,"2.20.0, 2.19.1, 2.18.1","2.20.0, 2.19.1, 2.18.1","2.20.0, 2.19.1, 2.18.1"
|
||||
:doc:`JAX <../compatibility/ml-compatibility/jax-compatibility>`,0.8.2,0.8.2,0.7.1
|
||||
`ONNX Runtime <https://onnxruntime.ai/docs/build/eps.html#amd-migraphx>`_,1.23.2,1.23.2,1.22.0
|
||||
:doc:`PyTorch <../compatibility/ml-compatibility/pytorch-compatibility>`,"2.9.1, 2.8.0, 2.7.1","2.9.1, 2.8.0, 2.7.1","2.6, 2.5, 2.4, 2.3"
|
||||
:doc:`TensorFlow <../compatibility/ml-compatibility/tensorflow-compatibility>`,"2.20.0, 2.19.1, 2.18.1","2.20.0, 2.19.1, 2.18.1","2.18.1, 2.17.1, 2.16.2"
|
||||
:doc:`JAX <../compatibility/ml-compatibility/jax-compatibility>`,0.8.2,0.8.0,0.4.35
|
||||
:doc:`DGL <../compatibility/ml-compatibility/dgl-compatibility>` [#dgl_compat]_,N/A,N/A,2.4.0
|
||||
`ONNX Runtime <https://onnxruntime.ai/docs/build/eps.html#amd-migraphx>`_,1.23.2,1.23.2,1.20.0
|
||||
,,,
|
||||
THIRD PARTY COMMS,.. _thirdpartycomms-support-compatibility-matrix:,,
|
||||
`UCC <https://github.com/ROCm/ucc>`_,>=1.6.0,>=1.6.0,>=1.4.0
|
||||
`UCX <https://github.com/ROCm/ucx>`_,>=1.17.0,>=1.17.0,>=1.17.0
|
||||
`UCC <https://github.com/ROCm/ucc>`_,>=1.6.0,>=1.4.0,>=1.3.0
|
||||
`UCX <https://github.com/ROCm/ucx>`_,>=1.17.0,>=1.17.0,>=1.15.0
|
||||
,,,
|
||||
THIRD PARTY ALGORITHM,.. _thirdpartyalgorithm-support-compatibility-matrix:,,
|
||||
Thrust,2.8.5,2.8.5,2.8.5
|
||||
CUB,2.8.5,2.8.5,2.8.5
|
||||
Thrust,2.8.5,2.8.5,2.5.0
|
||||
CUB,2.8.5,2.8.5,2.5.0
|
||||
,,,
|
||||
DRIVER & USER SPACE [#kfd_support]_,.. _kfd-userspace-support-compatibility-matrix:,,
|
||||
:doc:`AMD GPU Driver <rocm-install-on-linux:reference/user-kernel-space-compat-matrix>`,"30.30.x, 30.20.x [#mi325x_KVM]_, |br| 30.10.x [#driver_patch]_, 6.4.x","30.30.x, 30.20.x [#mi325x_KVM]_, |br| 30.10.x [#driver_patch]_, 6.4.x","30.20.0 [#mi325x_KVM]_, 30.10.x [#driver_patch]_, 6.4.x"
|
||||
:doc:`AMD GPU Driver <rocm-install-on-linux:reference/user-kernel-space-compat-matrix>`,"30.30.1, 30.30.0, 30.20.1, |br| 30.20.0 [#mi325x_KVM]_, 30.10.2, |br| 30.10.1 [#driver_patch]_, 30.10, 6.4.x","30.30.0, 30.20.1, 30.20.0 [#mi325x_KVM]_, |br| 30.10.2, 30.10.1 [#driver_patch]_, |br| 30.10, 6.4.x","6.4.x, 6.3.x, 6.2.x, 6.1.x"
|
||||
,,,
|
||||
ML & COMPUTER VISION,.. _mllibs-support-compatibility-matrix:,,
|
||||
:doc:`Composable Kernel <composable_kernel:index>`,1.2.0,1.2.0,1.1.0
|
||||
:doc:`MIGraphX <amdmigraphx:index>`,2.15.0,2.15.0,2.14.0
|
||||
:doc:`MIOpen <miopen:index>`,3.5.1,3.5.1,3.5.1
|
||||
:doc:`MIVisionX <mivisionx:index>`,3.5.0,3.5.0,3.4.0
|
||||
:doc:`rocAL <rocal:index>`,2.5.0,2.5.0,2.4.0
|
||||
:doc:`rocDecode <rocdecode:index>`,1.7.0,1.7.0,1.4.0
|
||||
:doc:`rocJPEG <rocjpeg:index>`,1.4.0,1.4.0,1.2.0
|
||||
:doc:`rocPyDecode <rocpydecode:index>`,0.8.0,0.8.0,0.7.0
|
||||
:doc:`RPP <rpp:index>`,2.2.1,2.2.1,2.1.0
|
||||
:doc:`MIGraphX <amdmigraphx:index>`,2.15.0,2.15.0,2.12.0
|
||||
:doc:`MIOpen <miopen:index>`,3.5.1,3.5.1,3.4.0
|
||||
:doc:`MIVisionX <mivisionx:index>`,3.5.0,3.5.0,3.2.0
|
||||
:doc:`rocAL <rocal:index>`,2.5.0,2.5.0,2.2.0
|
||||
:doc:`rocDecode <rocdecode:index>`,1.7.0,1.5.0,0.10.0
|
||||
:doc:`rocJPEG <rocjpeg:index>`,1.4.0,1.3.0,0.8.0
|
||||
:doc:`rocPyDecode <rocpydecode:index>`,0.8.0,0.8.0,0.3.1
|
||||
:doc:`RPP <rpp:index>`,2.2.1,2.2.0,1.9.10
|
||||
,,,
|
||||
COMMUNICATION,.. _commlibs-support-compatibility-matrix:,,
|
||||
:doc:`RCCL <rccl:index>`,2.27.7,2.27.7,2.27.7
|
||||
:doc:`rocSHMEM <rocshmem:index>`,3.2.0,3.2.0,3.0.0
|
||||
:doc:`RCCL <rccl:index>`,2.27.7,2.27.7,2.22.3
|
||||
:doc:`rocSHMEM <rocshmem:index>`,3.2.0,3.2.0,2.0.0
|
||||
,,,
|
||||
MATH LIBS,.. _mathlibs-support-compatibility-matrix:,,
|
||||
`half <https://github.com/ROCm/half>`_ ,1.12.0,1.12.0,1.12.0
|
||||
:doc:`hipBLAS <hipblas:index>`,3.2.0,3.2.0,3.1.0
|
||||
:doc:`hipBLASLt <hipblaslt:index>`,1.2.2,1.2.2,1.1.0
|
||||
:doc:`hipFFT <hipfft:index>`,1.0.22,1.0.22,1.0.21
|
||||
:doc:`hipfort <hipfort:index>`,0.7.1,0.7.1,0.7.1
|
||||
:doc:`hipRAND <hiprand:index>`,3.1.0,3.1.0,3.1.0
|
||||
:doc:`hipSOLVER <hipsolver:index>`,3.2.0,3.2.0,3.1.0
|
||||
:doc:`hipSPARSE <hipsparse:index>`,4.2.0,4.2.0,4.1.0
|
||||
:doc:`hipSPARSELt <hipsparselt:index>`,0.2.6,0.2.6,0.2.5
|
||||
:doc:`rocALUTION <rocalution:index>`,4.1.0,4.1.0,4.0.1
|
||||
:doc:`rocBLAS <rocblas:index>`,5.2.0,5.2.0,5.1.0
|
||||
:doc:`rocFFT <rocfft:index>`,1.0.36,1.0.36,1.0.35
|
||||
:doc:`rocRAND <rocrand:index>`,4.2.0,4.2.0,4.1.0
|
||||
:doc:`rocSOLVER <rocsolver:index>`,3.32.0,3.32.0,3.31.0
|
||||
:doc:`rocSPARSE <rocsparse:index>`,4.2.0,4.2.0,4.1.0
|
||||
:doc:`rocWMMA <rocwmma:index>`,2.2.0,2.2.0,2.0.0
|
||||
:doc:`Tensile <tensile:src/index>`,4.45.0,4.45.0,4.44.0
|
||||
:doc:`hipBLAS <hipblas:index>`,3.2.0,3.2.0,2.4.0
|
||||
:doc:`hipBLASLt <hipblaslt:index>`,1.2.2,1.2.1,0.12.0
|
||||
:doc:`hipFFT <hipfft:index>`,1.0.22,1.0.22,1.0.18
|
||||
:doc:`hipfort <hipfort:index>`,0.7.1,0.7.1,0.6.0
|
||||
:doc:`hipRAND <hiprand:index>`,3.1.0,3.1.0,2.12.0
|
||||
:doc:`hipSOLVER <hipsolver:index>`,3.2.0,3.2.0,2.4.0
|
||||
:doc:`hipSPARSE <hipsparse:index>`,4.2.0,4.2.0,3.2.0
|
||||
:doc:`hipSPARSELt <hipsparselt:index>`,0.2.6,0.2.6,0.2.3
|
||||
:doc:`rocALUTION <rocalution:index>`,4.1.0,4.1.0,3.2.2
|
||||
:doc:`rocBLAS <rocblas:index>`,5.2.0,5.2.0,4.4.0
|
||||
:doc:`rocFFT <rocfft:index>`,1.0.36,1.0.36,1.0.32
|
||||
:doc:`rocRAND <rocrand:index>`,4.2.0,4.2.0,3.3.0
|
||||
:doc:`rocSOLVER <rocsolver:index>`,3.32.0,3.32.0,3.28.0
|
||||
:doc:`rocSPARSE <rocsparse:index>`,4.2.0,4.2.0,3.4.0
|
||||
:doc:`rocWMMA <rocwmma:index>`,2.2.0,2.2.0,1.7.0
|
||||
:doc:`Tensile <tensile:src/index>`,4.45.0,4.45.0,4.43.0
|
||||
,,,
|
||||
PRIMITIVES,.. _primitivelibs-support-compatibility-matrix:,,
|
||||
:doc:`hipCUB <hipcub:index>`,4.2.0,4.2.0,4.1.0
|
||||
:doc:`hipTensor <hiptensor:index>`,2.2.0,2.2.0,2.0.0
|
||||
:doc:`rocPRIM <rocprim:index>`,4.2.0,4.2.0,4.1.0
|
||||
:doc:`rocThrust <rocthrust:index>`,4.2.0,4.2.0,4.1.0
|
||||
:doc:`hipCUB <hipcub:index>`,4.2.0,4.2.0,3.4.0
|
||||
:doc:`hipTensor <hiptensor:index>`,2.2.0,2.2.0,1.5.0
|
||||
:doc:`rocPRIM <rocprim:index>`,4.2.0,4.2.0,3.4.0
|
||||
:doc:`rocThrust <rocthrust:index>`,4.2.0,4.2.0,3.3.0
|
||||
,,,
|
||||
SUPPORT LIBS,,,
|
||||
`hipother <https://github.com/ROCm/hipother>`_,7.2.53211,7.2.53211,7.1.25424
|
||||
`rocm-core <https://github.com/ROCm/rocm-core>`_,7.2.4/7.2.3,7.2.2/7.2.1,7.1.0
|
||||
`hipother <https://github.com/ROCm/hipother>`_,7.2.53211,7.2.26015,6.4.43482
|
||||
`rocm-core <https://github.com/ROCm/rocm-core>`_,7.2.1,7.2.0,6.4.0
|
||||
`ROCT-Thunk-Interface <https://github.com/ROCm/ROCT-Thunk-Interface>`_,N/A [#ROCT-rocr]_,N/A [#ROCT-rocr]_,N/A [#ROCT-rocr]_
|
||||
,,,
|
||||
SYSTEM MGMT TOOLS,.. _tools-support-compatibility-matrix:,,
|
||||
:doc:`AMD SMI <amdsmi:index>`,26.2.2,26.2.2,26.1.0
|
||||
:doc:`ROCm Data Center Tool <rdc:index>`,1.2.0,1.2.0,1.2.0
|
||||
:doc:`AMD SMI <amdsmi:index>`,26.2.2,26.2.1,25.3.0
|
||||
:doc:`ROCm Data Center Tool <rdc:index>`,1.2.0,1.2.0,0.3.0
|
||||
:doc:`rocminfo <rocminfo:index>`,1.0.0,1.0.0,1.0.0
|
||||
:doc:`ROCm SMI <rocm_smi_lib:index>`,7.8.0,7.8.0,7.8.0
|
||||
:doc:`ROCm Validation Suite <rocmvalidationsuite:index>`,1.3.0,1.3.0,1.2.0
|
||||
:doc:`ROCm SMI <rocm_smi_lib:index>`,7.8.0,7.8.0,7.5.0
|
||||
:doc:`ROCm Validation Suite <rocmvalidationsuite:index>`,1.3.0,1.3.0,1.1.0
|
||||
,,,
|
||||
PERFORMANCE TOOLS,,,
|
||||
:doc:`ROCm Bandwidth Test <rocm_bandwidth_test:index>`,2.6.0,2.6.0,2.6.0
|
||||
:doc:`ROCm Compute Profiler <rocprofiler-compute:index>`,3.4.0,3.4.0,3.3.0
|
||||
:doc:`ROCm Systems Profiler <rocprofiler-systems:index>`,1.3.0,1.3.0,1.2.0
|
||||
:doc:`ROCProfiler <rocprofiler:index>`,2.0.70204/2.0.70203,2.0.70202/2.0.70201,2.0.70100
|
||||
:doc:`ROCprofiler-SDK <rocprofiler-sdk:index>`,1.1.0,1.1.0,1.0.0
|
||||
:doc:`ROCTracer <roctracer:index>`,4.1.70204/4.1.70203,4.1.70202/4.1.70201,4.1.70100
|
||||
:doc:`ROCm Bandwidth Test <rocm_bandwidth_test:index>`,2.6.0,2.6.0,1.4.0
|
||||
:doc:`ROCm Compute Profiler <rocprofiler-compute:index>`,3.4.0,3.4.0,3.1.0
|
||||
:doc:`ROCm Systems Profiler <rocprofiler-systems:index>`,1.3.0,1.3.0,1.0.0
|
||||
:doc:`ROCProfiler <rocprofiler:index>`,2.0.70201,2.0.70200,2.0.60400
|
||||
:doc:`ROCprofiler-SDK <rocprofiler-sdk:index>`,1.1.0,1.1.0,0.6.0
|
||||
:doc:`ROCTracer <roctracer:index>`,4.1.70201,4.1.70200,4.1.60400
|
||||
,,,
|
||||
DEVELOPMENT TOOLS,,,
|
||||
:doc:`HIPIFY <hipify:index>`,22.0.0,22.0.0,20.0.0
|
||||
:doc:`HIPIFY <hipify:index>`,22.0.0,22.0.0,19.0.0
|
||||
:doc:`ROCm CMake <rocmcmakebuildtools:index>`,0.14.0,0.14.0,0.14.0
|
||||
:doc:`ROCdbgapi <rocdbgapi:index>`,0.77.4,0.77.4,0.77.4
|
||||
:doc:`ROCm Debugger (ROCgdb) <rocgdb:index>`,16.3.0,16.3.0,16.3.0
|
||||
`rocprofiler-register <https://github.com/ROCm/rocprofiler-register>`_,0.5.0,0.5.0,0.5.0
|
||||
:doc:`ROCr Debug Agent <rocr_debug_agent:index>`,2.1.0,2.1.0,2.1.0
|
||||
:doc:`ROCdbgapi <rocdbgapi:index>`,0.77.4,0.77.4,0.77.2
|
||||
:doc:`ROCm Debugger (ROCgdb) <rocgdb:index>`,16.3.0,16.3.0,15.2.0
|
||||
`rocprofiler-register <https://github.com/ROCm/rocprofiler-register>`_,0.5.0,0.5.0,0.4.0
|
||||
:doc:`ROCr Debug Agent <rocr_debug_agent:index>`,2.1.0,2.1.0,2.0.4
|
||||
,,,
|
||||
COMPILERS,.. _compilers-support-compatibility-matrix:,,
|
||||
`clang-ocl <https://github.com/ROCm/clang-ocl>`_,N/A,N/A,N/A
|
||||
:doc:`hipCC <hipcc:index>`,1.1.1,1.1.1,1.1.1
|
||||
`Flang <https://github.com/ROCm/flang>`_,22.0.0.26084,22.0.0.26084,20.0.025425
|
||||
:doc:`llvm-project <llvm-project:index>`,22.0.0.26084,22.0.0.26084,20.0.025425
|
||||
`OpenMP <https://github.com/ROCm/llvm-project/tree/amd-staging/openmp>`_,22.0.0.26084,22.0.0.26084,20.0.025425
|
||||
`Flang <https://github.com/ROCm/flang>`_,22.0.0.26084,22.0.0.26014,19.0.0.25133
|
||||
:doc:`llvm-project <llvm-project:index>`,22.0.0.26084,22.0.0.26014,19.0.0.25133
|
||||
`OpenMP <https://github.com/ROCm/llvm-project/tree/amd-staging/openmp>`_,22.0.0.26084,22.0.0.26014,19.0.0.25133
|
||||
,,,
|
||||
RUNTIMES,.. _runtime-support-compatibility-matrix:,,
|
||||
:doc:`AMD CLR <hip:understand/amd_clr>`,7.2.53211,7.2.53211,7.1.25424
|
||||
:doc:`HIP <hip:index>`,7.2.53211,7.2.53211,7.1.25424
|
||||
:doc:`AMD CLR <hip:understand/amd_clr>`,7.2.53211,7.2.26015,6.4.43482
|
||||
:doc:`HIP <hip:index>`,7.2.53211,7.2.26015,6.4.43482
|
||||
`OpenCL Runtime <https://github.com/ROCm/clr/tree/develop/opencl>`_,2.0.0,2.0.0,2.0.0
|
||||
:doc:`ROCr Runtime <rocr-runtime:index>`,1.18.0,1.18.0,1.18.0
|
||||
:doc:`ROCr Runtime <rocr-runtime:index>`,1.18.0,1.18.0,1.15.0
|
||||
|
||||
|
||||
.. rubric:: Footnotes
|
||||
|
||||
.. [#os-compatibility] Some operating systems are supported on specific GPUs. For detailed information about operating systems supported on ROCm 7.2.4/7.2.3, see the latest :ref:`supported_distributions`. For version specific information, see `ROCm 7.2.2/7.2.1 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.2/reference/system-requirements.html#supported-operating-systems>`__, and `ROCm 7.1.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.1.0/reference/system-requirements.html#supported-operating-systems>`__.
|
||||
.. [#gpu-compatibility] Some GPUs have limited operating system support. For detailed information about GPUs supporting ROCm 7.2.4/7.2.3, see the latest :ref:`supported_GPUs`. For version specific information, see `ROCm 7.2.2/7.2.1 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.2/reference/system-requirements.html#supported-gpus>`__, and `ROCm 7.1.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.1.0/reference/system-requirements.html#supported-gpus>`__.
|
||||
.. [#os-compatibility] Some operating systems are supported on specific GPUs. For detailed information about operating systems supported on ROCm 7.2.1, see the latest :ref:`supported_distributions`. For version specific information, see `ROCm 7.2.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.0/reference/system-requirements.html#supported-operating-systems>`__, and `ROCm 6.4.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-6.4.0/reference/system-requirements.html#supported-operating-systems>`__.
|
||||
.. [#gpu-compatibility] Some GPUs have limited operating system support. For detailed information about GPUs supporting ROCm 7.2.1, see the latest :ref:`supported_GPUs`. For version specific information, see `ROCm 7.2.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.0/reference/system-requirements.html#supported-gpus>`__, and `ROCm 6.4.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-6.4.0/reference/system-requirements.html#supported-gpus>`__.
|
||||
.. [#dgl_compat] DGL is supported only on ROCm 7.0.0, ROCm 6.4.3, and ROCm 6.4.0.
|
||||
.. [#mi325x_KVM] For AMD Instinct MI325X KVM SR-IOV users, do not use AMD GPU Driver (amdgpu) 30.20.0.
|
||||
.. [#driver_patch] AMD GPU Driver (amdgpu) 30.10.1 is a quality release that resolves an issue identified in the 30.10 release. There are no other significant changes or feature additions in ROCm 7.0.1 from ROCm 7.0.0. AMD GPU Driver (amdgpu) 30.10.1 is compatible with ROCm 7.0.1 and ROCm 7.0.0.
|
||||
.. [#kfd_support] As of ROCm 6.4.0, forward and backward compatibility between the AMD GPU Driver (amdgpu) and its user space software is provided up to a year apart. For earlier ROCm releases, the compatibility is provided for +/- 2 releases. The supported user space versions on this page were accurate as of the time of initial ROCm release. For the most up-to-date information, see the latest version of this information at `User and AMD GPU Driver support matrix <https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/user-kernel-space-compat-matrix.html>`_.
|
||||
@@ -164,7 +168,7 @@ compatibility and system requirements.
|
||||
Operating systems, kernel and Glibc versions
|
||||
*********************************************
|
||||
|
||||
For detailed information on operating system supported on ROCm 7.2.4/7.2.3 and associated Kernel and Glibc version, see the latest :ref:`supported_distributions`. For version specific information, see `ROCm 7.2.2/7.2.1 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.2/reference/system-requirements.html#supported-operating-systems>`__, and `ROCm 7.1.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.1.0/reference/system-requirements.html#supported-operating-systems>`__.
|
||||
For detailed information on operating system supported on ROCm 7.2.1 and associated Kernel and Glibc version, see the latest :ref:`supported_distributions`. For version specific information, see `ROCm 7.2.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.0/reference/system-requirements.html#supported-operating-systems>`__, and `ROCm 6.4.0 <https://rocm.docs.amd.com/projects/install-on-linux/en/docs-6.4.0/reference/system-requirements.html#supported-operating-systems>`__.
|
||||
|
||||
.. note::
|
||||
|
||||
|
||||
+5
-7
@@ -81,7 +81,7 @@ latex_elements = {
|
||||
}
|
||||
|
||||
html_baseurl = os.environ.get("READTHEDOCS_CANONICAL_URL", "rocm.docs.amd.com")
|
||||
html_context = {"docs_header_version": "7.2.4"}
|
||||
html_context = {"docs_header_version": "7.2.1"}
|
||||
if os.environ.get("READTHEDOCS", "") == "True":
|
||||
html_context["READTHEDOCS"] = True
|
||||
|
||||
@@ -93,15 +93,15 @@ project = "ROCm Documentation"
|
||||
project_path = os.path.abspath(".").replace("\\", "/")
|
||||
author = "Advanced Micro Devices, Inc."
|
||||
copyright = "Copyright (c) 2026 Advanced Micro Devices, Inc. All rights reserved."
|
||||
version = "7.2.4"
|
||||
release = "7.2.4"
|
||||
version = "7.2.1"
|
||||
release = "7.2.1"
|
||||
setting_all_article_info = True
|
||||
all_article_info_os = ["linux", "windows"]
|
||||
all_article_info_author = ""
|
||||
|
||||
# pages with specific settings
|
||||
article_pages = [
|
||||
{"file": "about/release-notes", "os": ["linux"], "date": "2026-05-29"},
|
||||
{"file": "about/release-notes", "os": ["linux"], "date": "2026-03-25"},
|
||||
{"file": "release/changelog", "os": ["linux"],},
|
||||
{"file": "compatibility/compatibility-matrix", "os": ["linux"]},
|
||||
{"file": "compatibility/ml-compatibility/pytorch-compatibility", "os": ["linux"]},
|
||||
@@ -146,7 +146,6 @@ article_pages = [
|
||||
{"file": "how-to/rocm-for-ai/training/benchmark-docker/previous-versions/pytorch-training-v25.4", "os": ["linux"]},
|
||||
{"file": "how-to/rocm-for-ai/training/benchmark-docker/previous-versions/pytorch-training-v25.5", "os": ["linux"]},
|
||||
{"file": "how-to/rocm-for-ai/training/benchmark-docker/previous-versions/pytorch-training-v25.6", "os": ["linux"]},
|
||||
{"file": "how-to/rocm-for-ai/inference/xdit-diffusion-inference", "os": ["linux"]},
|
||||
{"file": "how-to/rocm-for-ai/training/benchmark-docker/previous-versions/pytorch-training-v25.7", "os": ["linux"]},
|
||||
{"file": "how-to/rocm-for-ai/training/benchmark-docker/previous-versions/pytorch-training-v25.8", "os": ["linux"]},
|
||||
{"file": "how-to/rocm-for-ai/training/benchmark-docker/previous-versions/pytorch-training-v25.9", "os": ["linux"]},
|
||||
@@ -205,7 +204,6 @@ article_pages = [
|
||||
{"file": "how-to/rocm-for-ai/inference/benchmark-docker/previous-versions/xdit-25.13", "os": ["linux"]},
|
||||
{"file": "how-to/rocm-for-ai/inference/benchmark-docker/previous-versions/xdit-26.1", "os": ["linux"]},
|
||||
{"file": "how-to/rocm-for-ai/inference/benchmark-docker/previous-versions/xdit-26.2", "os": ["linux"]},
|
||||
{"file": "how-to/rocm-for-ai/inference/benchmark-docker/previous-versions/xdit-26.3", "os": ["linux"]},
|
||||
|
||||
{"file": "how-to/rocm-for-ai/inference/deploy-your-model", "os": ["linux"]},
|
||||
|
||||
@@ -246,7 +244,7 @@ external_projects_current_project = "rocm"
|
||||
# external_projects_remote_repository = ""
|
||||
|
||||
html_baseurl = os.environ.get("READTHEDOCS_CANONICAL_URL", "https://rocm-stg.amd.com/")
|
||||
html_context = {"docs_header_version": "7.2.3"}
|
||||
html_context = {"docs_header_version": "7.2.1"}
|
||||
if os.environ.get("READTHEDOCS", "") == "True":
|
||||
html_context["READTHEDOCS"] = True
|
||||
|
||||
|
||||
@@ -4,15 +4,25 @@
|
||||
<meta name="keywords" content="ROCm, contributing, contribute, maintainer, contributor">
|
||||
</head>
|
||||
|
||||
# Contributing to ROCm documentation
|
||||
# Contributing to the ROCm documentation
|
||||
|
||||
ROCm documentation is open source and available on GitHub. You can contribute to ROCm documentation by cloning the appropriate repository, making your changes, and opening a pull request.
|
||||
The ROCm documentation, like all of ROCm, is open source and available on GitHub. You can contribute to the ROCm documentation by forking the appropriate repository, making your changes, and opening a pull request.
|
||||
|
||||
```{Note}
|
||||
To provide feedback on the ROCm documentation without contributing to it, see [Providing feedback about the ROCm documentation](./feedback.md).
|
||||
```
|
||||
To provide feedback on the ROCm documentation, including submitting an issue or suggesting a feature, see [Providing feedback about the ROCm documentation](./feedback.md).
|
||||
|
||||
The documentation for ROCm and for all ROCm components is under their respective `docs` folders. The `docs` folders for all components across ROCm have the same structure:
|
||||
## The ROCm repositories
|
||||
|
||||
The repositories for ROCm and all ROCm components are available on GitHub.
|
||||
|
||||
| Module | Documentation location |
|
||||
| --- | --- |
|
||||
| ROCm framework | [https://github.com/ROCm/ROCm/tree/develop/docs](https://github.com/ROCm/ROCm/tree/develop/docs) |
|
||||
| ROCm installation for Linux | [https://github.com/ROCm/rocm-install-on-linux/tree/develop/docs](https://github.com/ROCm/rocm-install-on-linux/tree/develop/docs) |
|
||||
| ROCm HIP SDK installation for Windows | [https://github.com/ROCm/rocm-install-on-windows/tree/develop/docs](https://github.com/ROCm/rocm-install-on-windows/tree/develop/docs) |
|
||||
|
||||
Individual components have their own repositories with their own documentation in their own `docs` folders.
|
||||
|
||||
The sub-folders within the `docs` folders across ROCm are typically structured as follows:
|
||||
|
||||
| Sub-folder name | Documentation type |
|
||||
|-------|----------|
|
||||
@@ -21,53 +31,45 @@ The documentation for ROCm and for all ROCm components is under their respective
|
||||
| `how-to` | How to implement specific use cases |
|
||||
| `tutorials` | Tutorials |
|
||||
| `reference` | API references and other reference resources |
|
||||
| `sphinx` | `_toc.yaml.in` file |
|
||||
|
||||
ROCm stack documentation differs from this structure.
|
||||
## Editing and adding to the documentation
|
||||
|
||||
Most documentation topics are written in [reStructuredText (rst)](https://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html), with some topics written in Markdown. Only use reStructuredText when adding new topics. Only use Markdown if the topic you are editing is already in Markdown.
|
||||
ROCm documentation follows the [Google developer documentation style guide](https://developers.google.com/style/highlights).
|
||||
|
||||
To edit or add to the documentation, first clone the appropriate repository, ensuring that you follow the repository recommendations as outlined in their respective README files:
|
||||
Most topics in the ROCm documentation are written in [reStructuredText (rst)](https://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html), with some topics written in Markdown. Only use reStructuredText when adding new topics. Only use Markdown if the topic you are editing is already in Markdown.
|
||||
|
||||
| Module | Repository |
|
||||
| --- | --- |
|
||||
| ROCm stack | [https://github.com/ROCm/ROCm](https://github.com/ROCm/ROCm) |
|
||||
| ROCm libraries | [https://github.com/ROCm/rocm-libraries](https://github.com/ROCm/rocm-libraries) |
|
||||
| ROCm systems projects | [https://github.com/ROCm/rocm-systems](https://github.com/ROCm/rocm-systems) |
|
||||
To edit or add to the documentation:
|
||||
|
||||
```{note}
|
||||
Individual components in the ROCm libraries and the ROCm systems projects repositories are located under the `projects` folder. Some components have their own individual repositories. Each component has a link to its GitHub location from its documentation.
|
||||
```
|
||||
1. Fork the repository you want to add to or edit.
|
||||
2. Clone your fork locally.
|
||||
3. Create a new local branch cut from the `develop` branch of the repository.
|
||||
4. Make your changes to the documentation.
|
||||
|
||||
Cut a local branch from the `develop` branch of the repository and make your changes. Your changes must adhere to the [Google developer documentation style guide](https://developers.google.com/style/highlights). If you're adding a topic, provide a link to it from the `index.rst` and `_toc.yaml.in` files.
|
||||
5. Optionally, build the documentation locally before creating a pull request by running the following commands from within the `docs` folder:
|
||||
|
||||
Build the documentation locally to verify your changes.
|
||||
```bash
|
||||
pip3 install -r sphinx/requirements.txt # You only need to run this command once
|
||||
python3 -m sphinx -T -E -b html -d _build/doctrees -D language=en . _build/html
|
||||
```
|
||||
|
||||
```{note}
|
||||
If you're making changes to the Doxygen comments within the source code, delete the `docs/doxygen/xml` and `docs/doxygen/html` folders between each build.
|
||||
```
|
||||
The output files will be located in the `docs/_build` folder. Open `docs/_build/html/index.html` to view the documentation.
|
||||
|
||||
From within the `docs` directory, run:
|
||||
For more information on ROCm build tools, see [Documentation toolchain](toolchain.md).
|
||||
6. Push your changes. A GitHub link will be returned in the output of the `git push` command. Open this link in a browser to create the pull request.
|
||||
|
||||
```bash
|
||||
python3 -m sphinx -T -E -b html -d _build/doctrees -D language=en . _build/html
|
||||
```
|
||||
The documentation is built as part of the checks on pull request, along with spell checking and linting. Scroll to the bottom of your pull request to view all the checks.
|
||||
|
||||
The output will be saved to the `docs/_build` folder. Open `docs/_build/html/index.html` to view the documentation.
|
||||
Verify that the linting and spell checking have passed, and that the documentation was built successfully. New words or acronyms can be added to the [wordlist file](https://github.com/ROCm/rocm-docs-core/blob/develop/.wordlist.txt). The wordlist is subject to approval by the ROCm documentation team.
|
||||
|
||||
```{note}
|
||||
If your build returns an error due to missing packages, run
|
||||
The Read The Docs build of your pull request can be accessed by clicking on the Details link next to the Read The Docs build check. Verify that your changes are in the build and look as expected.
|
||||
|
||||
`pip3 install -r sphinx/requirements.txt`
|
||||
|
||||
|
||||
|
||||
|
||||
This command only needs to be run once.
|
||||
```
|
||||
Your pull request will be reviewed by a member of the ROCm documentation team.
|
||||
|
||||
Once you've verified your changes, push your branch and create a pull request. Your pull request will be reviewed by a member of the ROCm documentation team.
|
||||
|
||||
For information about how to clone a repository, or how to create and push a local branch, see the [GitHub documentation](https://docs.github.com/en).
|
||||
|
||||
For information about ROCm build tools, see [Documentation toolchain](toolchain.md).
|
||||
See the [GitHub documentation](https://docs.github.com/en) for information on how to fork and clone a repository, and how to create and push a local branch.
|
||||
|
||||
```{important}
|
||||
By creating a pull request (PR), you agree to allow your contribution to be licensed under the terms of the
|
||||
|
||||
+1
-1
@@ -110,7 +110,7 @@ vllm_benchmark:
|
||||
- model: DBRX Instruct
|
||||
mad_tag: pyt_vllm_dbrx-instruct
|
||||
model_repo: databricks/dbrx-instruct
|
||||
url: https://huggingface.co/docs/transformers/model_doc/dbrx
|
||||
url: https://huggingface.co/databricks/dbrx-instruct
|
||||
precision: float16
|
||||
- model: DBRX Instruct FP8
|
||||
mad_tag: pyt_vllm_dbrx_fp8
|
||||
|
||||
-354
@@ -1,354 +0,0 @@
|
||||
docker:
|
||||
pull_tag: rocm/pytorch-xdit:v26.3
|
||||
docker_hub_url: https://hub.docker.com/layers/rocm/pytorch-xdit/v26.3/images/sha256-ac78a03d2911bf1b49c001d3be2e8bd745c1bc455cb49ae972825a7986880902
|
||||
ROCm: 7.12.0
|
||||
whats_new:
|
||||
- "Qwen-Image support"
|
||||
- "Qwen-Image-Edit support"
|
||||
- "Aiter update to support Sage attention v2"
|
||||
- "xDiT update to support MXFP4 GEMMs in Wan2.2, Wan2.1 and Flux.2"
|
||||
components:
|
||||
TheRock:
|
||||
version: e40a6da
|
||||
url: https://github.com/ROCm/TheRock
|
||||
rocm-libraries:
|
||||
version: 9e9e900
|
||||
url: https://github.com/ROCm/rocm-libraries
|
||||
rocm-systems:
|
||||
version: ca89a1a
|
||||
url: https://github.com/ROCm/rocm-systems
|
||||
torch:
|
||||
version: 91be249
|
||||
url: https://github.com/ROCm/pytorch
|
||||
torchaudio:
|
||||
version: e3c6ee2
|
||||
url: https://github.com/pytorch/audio
|
||||
torchvision:
|
||||
version: b919bd0
|
||||
url: https://github.com/pytorch/vision
|
||||
triton:
|
||||
version: a272dfa
|
||||
url: https://github.com/ROCm/triton
|
||||
accelerate:
|
||||
version: 46ba481
|
||||
url: https://github.com/huggingface/accelerate
|
||||
aiter:
|
||||
version: 82d733f
|
||||
url: https://github.com/ROCm/aiter
|
||||
diffusers:
|
||||
version: a80b192
|
||||
url: https://github.com/huggingface/diffusers
|
||||
xfuser:
|
||||
version: 2882027
|
||||
url: https://github.com/xdit-project/xDiT
|
||||
yunchang:
|
||||
version: 631bdfd
|
||||
url: https://github.com/feifeibear/long-context-attention
|
||||
supported_models:
|
||||
- group: Hunyuan Video
|
||||
js_tag: hunyuan
|
||||
models:
|
||||
- model: Hunyuan Video
|
||||
model_repo: tencent/HunyuanVideo
|
||||
revision: refs/pr/18
|
||||
url: https://huggingface.co/tencent/HunyuanVideo
|
||||
github: https://github.com/Tencent-Hunyuan/HunyuanVideo
|
||||
mad_tag: pyt_xdit_hunyuanvideo
|
||||
js_tag: hunyuan_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--prompt "In the large cage, two puppies were wagging their tails at each other." \'
|
||||
- '--batch_size 1 \'
|
||||
- '--height 720 --width 1280 \'
|
||||
- '--seed 1168860793 \'
|
||||
- '--num_frames 129 \'
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--warmup_calls 1 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--enable_tiling --enable_slicing \'
|
||||
- '--guidance_scale 6.0 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- model: Hunyuan Video 1.5
|
||||
model_repo: hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-720p_t2v
|
||||
url: https://huggingface.co/hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-720p_t2v
|
||||
github: https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5
|
||||
mad_tag: pyt_xdit_hunyuanvideo_1_5
|
||||
js_tag: hunyuan_1_5_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--prompt "In the large cage, two puppies were wagging their tails at each other." \'
|
||||
- '--task t2v \'
|
||||
- '--height 720 --width 1280 \'
|
||||
- '--seed 1168860793 \'
|
||||
- '--num_frames 129 \'
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--enable_tiling --enable_slicing \'
|
||||
- '--use_torch_compile \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- group: Wan-AI
|
||||
js_tag: wan
|
||||
models:
|
||||
- model: Wan2.1
|
||||
model_repo: Wan-AI/Wan2.1-I2V-14B-720P-Diffusers
|
||||
url: https://huggingface.co/Wan-AI/Wan2.1-I2V-14B-720P-Diffusers
|
||||
github: https://github.com/Wan-Video/Wan2.1
|
||||
mad_tag: pyt_xdit_wan_2_1
|
||||
js_tag: wan_21_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline''s intricate details and the refreshing atmosphere of the seaside." \'
|
||||
- '--height 720 \'
|
||||
- '--width 1280 \'
|
||||
- '--input_images /app/data/wan_input.jpg \'
|
||||
- '--num_frames 81 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--seed 42 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--num_inference_steps 40 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- model: Wan2.2
|
||||
model_repo: Wan-AI/Wan2.2-I2V-A14B-Diffusers
|
||||
url: https://huggingface.co/Wan-AI/Wan2.2-I2V-A14B-Diffusers
|
||||
github: https://github.com/Wan-Video/Wan2.2
|
||||
mad_tag: pyt_xdit_wan_2_2
|
||||
js_tag: wan_22_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline''s intricate details and the refreshing atmosphere of the seaside." \'
|
||||
- '--height 720 \'
|
||||
- '--width 1280 \'
|
||||
- '--input_images /app/data/wan_input.jpg \'
|
||||
- '--num_frames 81 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--seed 42 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--num_inference_steps 40 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- group: FLUX
|
||||
js_tag: flux
|
||||
models:
|
||||
- model: FLUX.1
|
||||
model_repo: black-forest-labs/FLUX.1-dev
|
||||
url: https://huggingface.co/black-forest-labs/FLUX.1-dev
|
||||
github: https://github.com/black-forest-labs/flux
|
||||
mad_tag: pyt_xdit_flux
|
||||
js_tag: flux_1_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "A small cat" \'
|
||||
- '--height 1024 \'
|
||||
- '--width 1024 \'
|
||||
- '--num_inference_steps 25 \'
|
||||
- '--max_sequence_length 256 \'
|
||||
- '--warmup_calls 5 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--guidance_scale 0.0 \'
|
||||
- '--num_iterations 50 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- model: FLUX.1 Kontext
|
||||
model_repo: black-forest-labs/FLUX.1-Kontext-dev
|
||||
url: https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev
|
||||
github: https://github.com/black-forest-labs/flux
|
||||
mad_tag: pyt_xdit_flux_kontext
|
||||
js_tag: flux_1_kontext_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "Add a cool hat to the cat" \'
|
||||
- '--height 1024 \'
|
||||
- '--width 1024 \'
|
||||
- '--num_inference_steps 30 \'
|
||||
- '--max_sequence_length 512 \'
|
||||
- '--warmup_calls 5 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--input_images /app/data/flux_cat.png \'
|
||||
- '--guidance_scale 2.5 \'
|
||||
- '--num_iterations 25 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- model: FLUX.2
|
||||
model_repo: black-forest-labs/FLUX.2-dev
|
||||
url: https://huggingface.co/black-forest-labs/FLUX.2-dev
|
||||
github: https://github.com/black-forest-labs/flux2
|
||||
mad_tag: pyt_xdit_flux_2
|
||||
js_tag: flux_2_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "Add a cool hat to the cat" \'
|
||||
- '--height 1024 \'
|
||||
- '--width 1024 \'
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--max_sequence_length 512 \'
|
||||
- '--warmup_calls 5 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--input_images /app/data/flux_cat.png \'
|
||||
- '--guidance_scale 4.0 \'
|
||||
- '--num_iterations 25 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- model: FLUX.2 Klein
|
||||
model_repo: black-forest-labs/FLUX.2-klein-9B
|
||||
url: https://huggingface.co/black-forest-labs/FLUX.2-klein-9B
|
||||
github: https://github.com/black-forest-labs/flux2
|
||||
mad_tag: pyt_xdit_flux_2_klein
|
||||
js_tag: flux_2_klein_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "A spectacular sunset over the ocean" \'
|
||||
- '--height 2048 \'
|
||||
- '--width 2048 \'
|
||||
- '--num_inference_steps 4 \'
|
||||
- '--warmup_calls 5 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--guidance_scale 1.0 \'
|
||||
- '--num_iterations 25 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- group: StableDiffusion
|
||||
js_tag: stablediffusion
|
||||
models:
|
||||
- model: stable-diffusion-3.5-large
|
||||
model_repo: stabilityai/stable-diffusion-3.5-large
|
||||
url: https://huggingface.co/stabilityai/stable-diffusion-3.5-large
|
||||
github: https://github.com/Stability-AI/sd3.5
|
||||
mad_tag: pyt_xdit_sd_3_5
|
||||
js_tag: stable_diffusion_3_5_large_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--prompt "A capybara holding a sign that reads Hello World" \'
|
||||
- '--num_iterations 50 \'
|
||||
- '--num_inference_steps 28 \'
|
||||
- '--pipefusion_parallel_degree 4 \'
|
||||
- '--use_cfg_parallel \'
|
||||
- '--use_torch_compile \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- group: Z-Image
|
||||
js_tag: z_image
|
||||
models:
|
||||
- model: Z-Image Turbo
|
||||
model_repo: Tongyi-MAI/Z-Image-Turbo
|
||||
url: https://huggingface.co/Tongyi-MAI/Z-Image-Turbo
|
||||
github: https://github.com/Tongyi-MAI/Z-Image
|
||||
mad_tag: pyt_xdit_z_image_turbo
|
||||
js_tag: z_image_turbo_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "A crowded beach" \'
|
||||
- '--height 1088 \'
|
||||
- '--width 1920 \'
|
||||
- '--num_inference_steps 9 \'
|
||||
- '--ulysses_degree 2 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--guidance_scale 0.0 \'
|
||||
- '--num_iterations 50 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- group: LTX
|
||||
js_tag: ltx
|
||||
models:
|
||||
- model: LTX-2
|
||||
model_repo: Lightricks/LTX-2
|
||||
url: https://huggingface.co/Lightricks/LTX-2
|
||||
github: https://github.com/Lightricks/LTX-2
|
||||
mad_tag: pyt_xdit_ltx2
|
||||
js_tag: ltx2_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "Cinematic action packed shot. The man says silently: \"We need to run.\". The camera zooms in on his mouth then immediately screams: \"NOW!\". The camera zooms back out, he turns around and bolts it." \'
|
||||
- '--height 1088 \'
|
||||
- '--width 1920 \'
|
||||
- '--num_inference_steps 40 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--guidance_scale 4.0 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- group: Qwen-Image
|
||||
js_tag: qwen_image
|
||||
models:
|
||||
- model: Qwen-Image
|
||||
model_repo: Qwen/Qwen-Image
|
||||
url: https://huggingface.co/Qwen/Qwen-Image
|
||||
github: https://github.com/QwenLM/Qwen-Image
|
||||
mad_tag: pyt_xdit_qwen_image
|
||||
js_tag: qwen_image_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "A cat holding a sign that says hello world" \'
|
||||
- '--height 2048 \'
|
||||
- '--width 2048 \'
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- model: Qwen-Image-Edit
|
||||
model_repo: Qwen/Qwen-Image-Edit
|
||||
url: https://huggingface.co/Qwen/Qwen-Image-Edit
|
||||
github: https://github.com/QwenLM/Qwen-Image
|
||||
mad_tag: pyt_xdit_qwen_image_edit
|
||||
js_tag: qwen_image_edit_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "Add a cool hat to the cat." \'
|
||||
- '--negative_prompt "" \'
|
||||
- '--input_images /app/data/flux_cat.png \'
|
||||
- '--height 2048 \'
|
||||
- '--width 2048 \'
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
-361
@@ -1,361 +0,0 @@
|
||||
docker:
|
||||
pull_tag: rocm/pytorch-xdit:v26.4
|
||||
docker_hub_url: https://hub.docker.com/layers/rocm/pytorch-xdit/v26.4/images/sha256-b4296a638eb8dc7ebcafc808e180b78a3c44177580c21986082ec9539496067c
|
||||
ROCm: 7.12.0
|
||||
whats_new:
|
||||
- "Qwen-Image-2512 support"
|
||||
- "Z-Image support"
|
||||
- "Parallel VAE decode support for Wan models"
|
||||
- "Batch inference and data parallel support"
|
||||
components:
|
||||
TheRock:
|
||||
version: 9b611c6
|
||||
url: https://github.com/ROCm/TheRock
|
||||
rocm-libraries:
|
||||
version: 7567d83
|
||||
url: https://github.com/ROCm/rocm-libraries
|
||||
rocm-systems:
|
||||
version: 93bc019
|
||||
url: https://github.com/ROCm/rocm-systems
|
||||
torch:
|
||||
version: ff65f5b
|
||||
url: https://github.com/ROCm/pytorch
|
||||
torchaudio:
|
||||
version: e3c6ee2
|
||||
url: https://github.com/pytorch/audio
|
||||
torchvision:
|
||||
version: b919bd0
|
||||
url: https://github.com/pytorch/vision
|
||||
triton:
|
||||
version: a272dfa
|
||||
url: https://github.com/ROCm/triton
|
||||
accelerate:
|
||||
version: 46ba481
|
||||
url: https://github.com/huggingface/accelerate
|
||||
aiter:
|
||||
version: a169e14
|
||||
url: https://github.com/ROCm/aiter
|
||||
diffusers:
|
||||
version: a80b192
|
||||
url: https://github.com/huggingface/diffusers
|
||||
distvae:
|
||||
version: bf7531e
|
||||
url: https://github.com/xdit-project/DistVAE
|
||||
xfuser:
|
||||
version: 45c44e7
|
||||
url: https://github.com/xdit-project/xDiT
|
||||
yunchang:
|
||||
version: 631bdfd
|
||||
url: https://github.com/feifeibear/long-context-attention
|
||||
supported_models:
|
||||
- group: Hunyuan Video
|
||||
js_tag: hunyuan
|
||||
models:
|
||||
- model: Hunyuan Video
|
||||
model_repo: tencent/HunyuanVideo
|
||||
revision: refs/pr/18
|
||||
url: https://huggingface.co/tencent/HunyuanVideo
|
||||
github: https://github.com/Tencent-Hunyuan/HunyuanVideo
|
||||
mad_tag: pyt_xdit_hunyuanvideo
|
||||
js_tag: hunyuan_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--prompt "In the large cage, two puppies were wagging their tails at each other." \'
|
||||
- '--batch_size 1 \'
|
||||
- '--height 720 --width 1280 \'
|
||||
- '--seed 1168860793 \'
|
||||
- '--num_frames 129 \'
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--warmup_calls 1 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--enable_tiling --enable_slicing \'
|
||||
- '--guidance_scale 6.0 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- model: Hunyuan Video 1.5
|
||||
model_repo: hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-720p_t2v
|
||||
url: https://huggingface.co/hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-720p_t2v
|
||||
github: https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5
|
||||
mad_tag: pyt_xdit_hunyuanvideo_1_5
|
||||
js_tag: hunyuan_1_5_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--prompt "In the large cage, two puppies were wagging their tails at each other." \'
|
||||
- '--task t2v \'
|
||||
- '--height 720 --width 1280 \'
|
||||
- '--seed 1168860793 \'
|
||||
- '--num_frames 129 \'
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--enable_tiling --enable_slicing \'
|
||||
- '--use_torch_compile \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- group: Wan-AI
|
||||
js_tag: wan
|
||||
models:
|
||||
- model: Wan2.1
|
||||
model_repo: Wan-AI/Wan2.1-I2V-14B-720P-Diffusers
|
||||
url: https://huggingface.co/Wan-AI/Wan2.1-I2V-14B-720P-Diffusers
|
||||
github: https://github.com/Wan-Video/Wan2.1
|
||||
mad_tag: pyt_xdit_wan_2_1
|
||||
js_tag: wan_21_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline''s intricate details and the refreshing atmosphere of the seaside." \'
|
||||
- '--height 720 \'
|
||||
- '--width 1280 \'
|
||||
- '--input_images /app/data/wan_input.jpg \'
|
||||
- '--num_frames 81 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_parallel_vae \'
|
||||
- '--seed 42 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--num_inference_steps 40 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- model: Wan2.2
|
||||
model_repo: Wan-AI/Wan2.2-I2V-A14B-Diffusers
|
||||
url: https://huggingface.co/Wan-AI/Wan2.2-I2V-A14B-Diffusers
|
||||
github: https://github.com/Wan-Video/Wan2.2
|
||||
mad_tag: pyt_xdit_wan_2_2
|
||||
js_tag: wan_22_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline''s intricate details and the refreshing atmosphere of the seaside." \'
|
||||
- '--height 720 \'
|
||||
- '--width 1280 \'
|
||||
- '--input_images /app/data/wan_input.jpg \'
|
||||
- '--num_frames 81 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_parallel_vae \'
|
||||
- '--seed 42 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--num_inference_steps 40 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- group: FLUX
|
||||
js_tag: flux
|
||||
models:
|
||||
- model: FLUX.1
|
||||
model_repo: black-forest-labs/FLUX.1-dev
|
||||
url: https://huggingface.co/black-forest-labs/FLUX.1-dev
|
||||
github: https://github.com/black-forest-labs/flux
|
||||
mad_tag: pyt_xdit_flux
|
||||
js_tag: flux_1_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "A small cat" \'
|
||||
- '--height 1024 \'
|
||||
- '--width 1024 \'
|
||||
- '--num_inference_steps 25 \'
|
||||
- '--max_sequence_length 256 \'
|
||||
- '--warmup_calls 5 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--guidance_scale 0.0 \'
|
||||
- '--num_iterations 50 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- model: FLUX.1 Kontext
|
||||
model_repo: black-forest-labs/FLUX.1-Kontext-dev
|
||||
url: https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev
|
||||
github: https://github.com/black-forest-labs/flux
|
||||
mad_tag: pyt_xdit_flux_kontext
|
||||
js_tag: flux_1_kontext_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "Add a cool hat to the cat" \'
|
||||
- '--height 1024 \'
|
||||
- '--width 1024 \'
|
||||
- '--num_inference_steps 30 \'
|
||||
- '--max_sequence_length 512 \'
|
||||
- '--warmup_calls 5 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--input_images /app/data/flux_cat.png \'
|
||||
- '--guidance_scale 2.5 \'
|
||||
- '--num_iterations 25 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- model: FLUX.2
|
||||
model_repo: black-forest-labs/FLUX.2-dev
|
||||
url: https://huggingface.co/black-forest-labs/FLUX.2-dev
|
||||
github: https://github.com/black-forest-labs/flux2
|
||||
mad_tag: pyt_xdit_flux_2
|
||||
js_tag: flux_2_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "Add a cool hat to the cat" \'
|
||||
- '--height 1024 \'
|
||||
- '--width 1024 \'
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--max_sequence_length 512 \'
|
||||
- '--warmup_calls 5 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--input_images /app/data/flux_cat.png \'
|
||||
- '--guidance_scale 4.0 \'
|
||||
- '--num_iterations 25 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- model: FLUX.2 Klein
|
||||
model_repo: black-forest-labs/FLUX.2-klein-9B
|
||||
url: https://huggingface.co/black-forest-labs/FLUX.2-klein-9B
|
||||
github: https://github.com/black-forest-labs/flux2
|
||||
mad_tag: pyt_xdit_flux_2_klein
|
||||
js_tag: flux_2_klein_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "A spectacular sunset over the ocean" \'
|
||||
- '--height 2048 \'
|
||||
- '--width 2048 \'
|
||||
- '--num_inference_steps 4 \'
|
||||
- '--warmup_calls 5 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--guidance_scale 1.0 \'
|
||||
- '--num_iterations 25 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- group: StableDiffusion
|
||||
js_tag: stablediffusion
|
||||
models:
|
||||
- model: stable-diffusion-3.5-large
|
||||
model_repo: stabilityai/stable-diffusion-3.5-large
|
||||
url: https://huggingface.co/stabilityai/stable-diffusion-3.5-large
|
||||
github: https://github.com/Stability-AI/sd3.5
|
||||
mad_tag: pyt_xdit_sd_3_5
|
||||
js_tag: stable_diffusion_3_5_large_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--prompt "A capybara holding a sign that reads Hello World" \'
|
||||
- '--num_iterations 50 \'
|
||||
- '--num_inference_steps 28 \'
|
||||
- '--pipefusion_parallel_degree 4 \'
|
||||
- '--use_cfg_parallel \'
|
||||
- '--use_torch_compile \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- group: Z-Image
|
||||
js_tag: z_image
|
||||
models:
|
||||
- model: Z-Image
|
||||
model_repo: Tongyi-MAI/Z-Image
|
||||
url: https://huggingface.co/Tongyi-MAI/Z-Image
|
||||
github: https://github.com/Tongyi-MAI/Z-Image
|
||||
mad_tag: pyt_xdit_z_image
|
||||
js_tag: z_image_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "A crowded beach" \'
|
||||
- '--height 1088 \'
|
||||
- '--width 1920 \'
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--ulysses_degree 2 \'
|
||||
- '--ring_degree 2 \'
|
||||
- '--use_cfg_parallel \'
|
||||
- '--use_torch_compile \'
|
||||
- '--guidance_scale 4.0 \'
|
||||
- '--num_iterations 25 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- group: LTX
|
||||
js_tag: ltx
|
||||
models:
|
||||
- model: LTX-2
|
||||
model_repo: Lightricks/LTX-2
|
||||
url: https://huggingface.co/Lightricks/LTX-2
|
||||
github: https://github.com/Lightricks/LTX-2
|
||||
mad_tag: pyt_xdit_ltx2
|
||||
js_tag: ltx2_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "Cinematic action packed shot. The man says silently: \"We need to run.\". The camera zooms in on his mouth then immediately screams: \"NOW!\". The camera zooms back out, he turns around and bolts it." \'
|
||||
- '--height 1088 \'
|
||||
- '--width 1920 \'
|
||||
- '--num_inference_steps 40 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--guidance_scale 4.0 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- group: Qwen-Image
|
||||
js_tag: qwen_image
|
||||
models:
|
||||
- model: Qwen-Image
|
||||
model_repo: Qwen/Qwen-Image-2512
|
||||
url: https://huggingface.co/Qwen/Qwen-Image-2512
|
||||
github: https://github.com/QwenLM/Qwen-Image
|
||||
mad_tag: pyt_xdit_qwen_image
|
||||
js_tag: qwen_image_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "A cat holding a sign that says hello world" \'
|
||||
- '--height 2048 \'
|
||||
- '--width 2048 \'
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- model: Qwen-Image-Edit
|
||||
model_repo: Qwen/Qwen-Image-Edit
|
||||
url: https://huggingface.co/Qwen/Qwen-Image-Edit
|
||||
github: https://github.com/QwenLM/Qwen-Image
|
||||
mad_tag: pyt_xdit_qwen_image_edit
|
||||
js_tag: qwen_image_edit_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
- '--model {model_repo} \'
|
||||
- '--seed 42 \'
|
||||
- '--prompt "Add a cool hat to the cat." \'
|
||||
- '--negative_prompt "" \'
|
||||
- '--input_images /app/data/flux_cat.png \'
|
||||
- '--height 2048 \'
|
||||
- '--width 2048 \'
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
@@ -1,23 +1,24 @@
|
||||
docker:
|
||||
pull_tag: rocm/pytorch-xdit:v26.5
|
||||
docker_hub_url: https://hub.docker.com/layers/rocm/pytorch-xdit/v26.5/images/sha256-b8ad9fd4b41bc116ac2aff07c1066bf369cf7fc110b1a323f6302191985a51fd
|
||||
ROCm: 7.13.0
|
||||
pull_tag: rocm/pytorch-xdit:v26.3
|
||||
docker_hub_url: https://hub.docker.com/layers/rocm/pytorch-xdit/v26.3/images/sha256-ac78a03d2911bf1b49c001d3be2e8bd745c1bc455cb49ae972825a7986880902
|
||||
ROCm: 7.12.0
|
||||
whats_new:
|
||||
- "Hunyuan Video 1.5 sparse attention (SSTA) support"
|
||||
- "Support fp8 MLA for MI355"
|
||||
- "Block wise sparsity support for AMD triton FAv3 Sage attention"
|
||||
- "Qwen-Image support"
|
||||
- "Qwen-Image-Edit support"
|
||||
- "Aiter update to support Sage attention v2"
|
||||
- "xDiT update to support MXFP4 GEMMs in Wan2.2, Wan2.1 and Flux.2"
|
||||
components:
|
||||
TheRock:
|
||||
version: cbff3d1
|
||||
version: e40a6da
|
||||
url: https://github.com/ROCm/TheRock
|
||||
rocm-libraries:
|
||||
version: a668483b
|
||||
version: 9e9e900
|
||||
url: https://github.com/ROCm/rocm-libraries
|
||||
rocm-systems:
|
||||
version: c76140fa
|
||||
version: ca89a1a
|
||||
url: https://github.com/ROCm/rocm-systems
|
||||
torch:
|
||||
version: ff65f5b
|
||||
version: 91be249
|
||||
url: https://github.com/ROCm/pytorch
|
||||
torchaudio:
|
||||
version: e3c6ee2
|
||||
@@ -32,16 +33,13 @@ docker:
|
||||
version: 46ba481
|
||||
url: https://github.com/huggingface/accelerate
|
||||
aiter:
|
||||
version: bc5ea32c
|
||||
version: 82d733f
|
||||
url: https://github.com/ROCm/aiter
|
||||
diffusers:
|
||||
version: 447e571a
|
||||
version: a80b192
|
||||
url: https://github.com/huggingface/diffusers
|
||||
distvae:
|
||||
version: 5a0fcbb
|
||||
url: https://github.com/xdit-project/DistVAE
|
||||
xfuser:
|
||||
version: 051db68f
|
||||
version: 2882027
|
||||
url: https://github.com/xdit-project/xDiT
|
||||
yunchang:
|
||||
version: 631bdfd
|
||||
@@ -116,9 +114,7 @@ docker:
|
||||
- '--input_images /app/data/wan_input.jpg \'
|
||||
- '--num_frames 81 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_parallel_vae \'
|
||||
- '--seed 42 \'
|
||||
- '--guidance_scale 3.0 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--num_inference_steps 40 \'
|
||||
- '--use_torch_compile \'
|
||||
@@ -140,7 +136,6 @@ docker:
|
||||
- '--input_images /app/data/wan_input.jpg \'
|
||||
- '--num_frames 81 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_parallel_vae \'
|
||||
- '--seed 42 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--num_inference_steps 40 \'
|
||||
@@ -267,12 +262,12 @@ docker:
|
||||
- group: Z-Image
|
||||
js_tag: z_image
|
||||
models:
|
||||
- model: Z-Image
|
||||
model_repo: Tongyi-MAI/Z-Image
|
||||
url: https://huggingface.co/Tongyi-MAI/Z-Image
|
||||
- model: Z-Image Turbo
|
||||
model_repo: Tongyi-MAI/Z-Image-Turbo
|
||||
url: https://huggingface.co/Tongyi-MAI/Z-Image-Turbo
|
||||
github: https://github.com/Tongyi-MAI/Z-Image
|
||||
mad_tag: pyt_xdit_z_image
|
||||
js_tag: z_image_tag
|
||||
mad_tag: pyt_xdit_z_image_turbo
|
||||
js_tag: z_image_turbo_tag
|
||||
benchmark_command:
|
||||
- mkdir results
|
||||
- 'xdit \'
|
||||
@@ -281,13 +276,11 @@ docker:
|
||||
- '--prompt "A crowded beach" \'
|
||||
- '--height 1088 \'
|
||||
- '--width 1920 \'
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--num_inference_steps 9 \'
|
||||
- '--ulysses_degree 2 \'
|
||||
- '--ring_degree 2 \'
|
||||
- '--use_cfg_parallel \'
|
||||
- '--use_torch_compile \'
|
||||
- '--guidance_scale 4.0 \'
|
||||
- '--num_iterations 25 \'
|
||||
- '--guidance_scale 0.0 \'
|
||||
- '--num_iterations 50 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- group: LTX
|
||||
@@ -318,8 +311,8 @@ docker:
|
||||
js_tag: qwen_image
|
||||
models:
|
||||
- model: Qwen-Image
|
||||
model_repo: Qwen/Qwen-Image-2512
|
||||
url: https://huggingface.co/Qwen/Qwen-Image-2512
|
||||
model_repo: Qwen/Qwen-Image
|
||||
url: https://huggingface.co/Qwen/Qwen-Image
|
||||
github: https://github.com/QwenLM/Qwen-Image
|
||||
mad_tag: pyt_xdit_qwen_image
|
||||
js_tag: qwen_image_tag
|
||||
@@ -334,8 +327,7 @@ docker:
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--guidance_scale 0.0 \'
|
||||
- '--num_iterations 25 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
- model: Qwen-Image-Edit
|
||||
@@ -357,7 +349,6 @@ docker:
|
||||
- '--num_inference_steps 50 \'
|
||||
- '--ulysses_degree 8 \'
|
||||
- '--use_torch_compile \'
|
||||
- '--guidance_scale 4.0 \'
|
||||
- '--num_iterations 25 \'
|
||||
- '--num_iterations 1 \'
|
||||
- '--attention_backend aiter \'
|
||||
- '--output_directory results'
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
dockers:
|
||||
- pull_tag: rocm/jax-training:maxtext-v26.3
|
||||
docker_hub_url: https://hub.docker.com/layers/rocm/jax-training/maxtext-v26.3/images/sha256-548381e496fa1367772964341130cc014ee1a9a5ca715ff4ca3aa92960d162c3
|
||||
- pull_tag: rocm/jax-training:maxtext-v26.2
|
||||
docker_hub_url: https://hub.docker.com/layers/rocm/jax-training/maxtext-v26.2/images/sha256-a89643388487b1e2fc6b6ef7bd3c44378c05d217309c977a1c18c72d05ebcaeb
|
||||
components:
|
||||
ROCm: 7.2.1
|
||||
ROCm: 7.1.1
|
||||
JAX: 0.8.2
|
||||
Python: 3.12
|
||||
Transformer Engine: 2.8.0.dev0+9b312832
|
||||
hipBLASLt: 1.3.0+bfcf25fa18
|
||||
Transformer Engine: 2.8.0.dev0+aec00a7f
|
||||
hipBLASLt: 1.2.x
|
||||
model_groups:
|
||||
- group: Meta Llama
|
||||
tag: llama
|
||||
@@ -93,24 +93,3 @@ model_groups:
|
||||
gfx950: env_scripts/gfx950_mixtral_8x7b.yml
|
||||
gfx942: env_scripts/llama3_8x7b.yml
|
||||
doc_options: ["single-node", "multi-node"]
|
||||
- group: Qwen
|
||||
tag: qwen
|
||||
models:
|
||||
- model: Qwen 14B
|
||||
mad_tag: jax_maxtext_train_qwen3-14b
|
||||
model_repo: Qwen3-14B
|
||||
precision: bf16
|
||||
primus_config_name: qwen3_14B-pretrain.yaml
|
||||
multinode_config:
|
||||
gfx950: env_scripts/gfx950_qwen3_14b.yml
|
||||
gfx942: env_scripts/qwen3_14b.yml
|
||||
doc_options: ["single-node", "multi-node"]
|
||||
- model: Qwen 30B A3B
|
||||
mad_tag: jax_maxtext_train_qwen3-30b-a3b
|
||||
model_repo: Qwen3-30B-A3B
|
||||
precision: bf16
|
||||
primus_config_name: qwen3_30B_a3b-pretrain.yaml
|
||||
multinode_config:
|
||||
gfx950: env_scripts/gfx950_qwen3_30b_a3b.yml
|
||||
gfx942: env_scripts/qwen3_30b_a3b.yml
|
||||
doc_options: ["single-node", "multi-node"]
|
||||
|
||||
-95
@@ -1,95 +0,0 @@
|
||||
dockers:
|
||||
- pull_tag: rocm/jax-training:maxtext-v26.2
|
||||
docker_hub_url: https://hub.docker.com/layers/rocm/jax-training/maxtext-v26.2/images/sha256-a89643388487b1e2fc6b6ef7bd3c44378c05d217309c977a1c18c72d05ebcaeb
|
||||
components:
|
||||
ROCm: 7.1.1
|
||||
JAX: 0.8.2
|
||||
Python: 3.12
|
||||
Transformer Engine: 2.8.0.dev0+aec00a7f
|
||||
hipBLASLt: 1.2.x
|
||||
model_groups:
|
||||
- group: Meta Llama
|
||||
tag: llama
|
||||
models:
|
||||
- model: Llama 2 7B
|
||||
mad_tag: jax_maxtext_train_llama-2-7b
|
||||
model_repo: Llama-2-7B
|
||||
precision: bf16
|
||||
primus_config_name: llama2_7B-pretrain.yaml
|
||||
multinode_config:
|
||||
gfx950: env_scripts/gfx950_llama2_7b.yml
|
||||
gfx942: env_scripts/llama2_7b.yml
|
||||
doc_options: ["single-node", "multi-node"]
|
||||
- model: Llama 2 70B
|
||||
mad_tag: jax_maxtext_train_llama-2-70b
|
||||
model_repo: Llama-2-70B
|
||||
precision: bf16
|
||||
primus_config_name: llama2_70B-pretrain.yaml
|
||||
multinode_config:
|
||||
gfx950: env_scripts/gfx950_llama2_70b.yml
|
||||
gfx942: env_scripts/llama2_70b.yml
|
||||
doc_options: ["single-node", "multi-node"]
|
||||
- model: Llama 3 8B
|
||||
mad_tag: jax_maxtext_train_llama-3-8b
|
||||
primus_config_name: llama3_8B-pretrain.yaml
|
||||
multinode_config:
|
||||
gfx950: env_scripts/gfx950_llama3_8b.yml
|
||||
gfx942: env_scripts/llama3_8b.yml
|
||||
doc_options: ["multi-node"]
|
||||
- model: Llama 3 70B
|
||||
mad_tag: jax_maxtext_train_llama-3-70b
|
||||
primus_config_name: llama3_70B-pretrain.yaml
|
||||
multinode_config:
|
||||
gfx950: env_scripts/gfx950_llama3_70b.yml
|
||||
gfx942: env_scripts/llama3_70b.yml
|
||||
doc_options: ["multi-node"]
|
||||
- model: Llama 3.1 8B
|
||||
mad_tag: jax_maxtext_train_llama-3.1-8b
|
||||
model_repo: Llama-3.1-8B
|
||||
precision: bf16
|
||||
doc_options: ["single-node"]
|
||||
- model: Llama 3.1 70B
|
||||
mad_tag: jax_maxtext_train_llama-3.1-70b
|
||||
model_repo: Llama-3.1-70B
|
||||
precision: bf16
|
||||
doc_options: ["single-node"]
|
||||
- model: Llama 3.1 405B (multi-node)
|
||||
mad_tag: jax_maxtext_train_llama-3.1-405b
|
||||
model_repo: Llama-3.1-405B
|
||||
precision: bf16
|
||||
multinode_config:
|
||||
gfx950: env_scripts/gfx950_llama3_405b.yml
|
||||
doc_options: ["multi-node"]
|
||||
- model: Llama 3.3 70B
|
||||
mad_tag: jax_maxtext_train_llama-3.3-70b
|
||||
model_repo: Llama-3.3-70B
|
||||
precision: bf16
|
||||
primus_config_name: llama3.3_70B-pretrain.yaml
|
||||
multinode_config:
|
||||
gfx950: env_scripts/gfx950_llama3.3_70b.yml
|
||||
gfx942: env_scripts/llama3.3_70b.yml
|
||||
doc_options: ["single-node", "multi-node"]
|
||||
- group: DeepSeek
|
||||
tag: deepseek
|
||||
models:
|
||||
- model: DeepSeek-V2-Lite (16B)
|
||||
mad_tag: jax_maxtext_train_deepseek-v2-lite-16b
|
||||
model_repo: DeepSeek-V2-lite
|
||||
precision: bf16
|
||||
primus_config_name: deepseek_v2_16B-pretrain.yaml
|
||||
multinode_config:
|
||||
gfx950: env_scripts/gfx950_deepseek2_16b.yml
|
||||
gfx942: env_scripts/deepseek2_16b.yml
|
||||
doc_options: ["single-node", "multi-node"]
|
||||
- group: Mistral AI
|
||||
tag: mistral
|
||||
models:
|
||||
- model: Mixtral 8x7B
|
||||
mad_tag: jax_maxtext_train_mixtral-8x7b
|
||||
model_repo: Mixtral-8x7B
|
||||
precision: bf16
|
||||
primus_config_name: mixtral_8x7B-pretrain.yaml
|
||||
multinode_config:
|
||||
gfx950: env_scripts/gfx950_mixtral_8x7b.yml
|
||||
gfx942: env_scripts/llama3_8x7b.yml
|
||||
doc_options: ["single-node", "multi-node"]
|
||||
-75
@@ -1,75 +0,0 @@
|
||||
docker:
|
||||
pull_tag: rocm/primus:v26.2
|
||||
docker_hub_url: https://hub.docker.com/layers/rocm/primus/v26.2/images/sha256-9148d1bfcd579bf92f44bd89090e0d8c958f149c134b4b34b9674ab559244585
|
||||
components:
|
||||
ROCm: 7.2.0
|
||||
PyTorch: 2.10.0a0+git449b176
|
||||
Python: "3.12.3"
|
||||
Transformer Engine: 2.8.0.dev0+51f74fa7
|
||||
Flash Attention: 2.8.3
|
||||
hipBLASLt: 1.2.0-de5c1aebb6
|
||||
Triton: 3.6.0
|
||||
RCCL: 2.27.7
|
||||
model_groups:
|
||||
- group: Meta Llama
|
||||
tag: llama
|
||||
models:
|
||||
- model: Llama 3.3 70B
|
||||
mad_tag: primus_pyt_megatron_lm_train_llama-3.3-70b
|
||||
config_name: llama3.3_70B-pretrain.yaml
|
||||
- model: Llama 3.1 8B
|
||||
mad_tag: primus_pyt_megatron_lm_train_llama-3.1-8b
|
||||
config_name: llama3.1_8B-pretrain.yaml
|
||||
- model: Llama 3.1 70B
|
||||
mad_tag: primus_pyt_megatron_lm_train_llama-3.1-70b
|
||||
config_name: llama3.1_70B-pretrain.yaml
|
||||
- model: Llama 2 7B
|
||||
mad_tag: primus_pyt_megatron_lm_train_llama-2-7b
|
||||
config_name: llama2_7B-pretrain.yaml
|
||||
- model: Llama 2 70B
|
||||
mad_tag: primus_pyt_megatron_lm_train_llama-2-70b
|
||||
config_name: llama2_70B-pretrain.yaml
|
||||
- group: AMD Zebra-Llama
|
||||
tag: zebra-llama
|
||||
models:
|
||||
- model: Zebra-Llama 1B
|
||||
mad_tag: primus_pyt_megatron_lm_train_zebra-llama-1b
|
||||
config_name: zebra_llama_1b-pretrain.yaml
|
||||
- model: Zebra-Llama 3B
|
||||
mad_tag: primus_pyt_megatron_lm_train_zebra-llama-3b
|
||||
config_name: zebra_llama_3b-pretrain.yaml
|
||||
- model: Zebra-Llama 8B
|
||||
mad_tag: primus_pyt_megatron_lm_train_zebra-llama-8b
|
||||
config_name: zebra_llama_8b-pretrain.yaml
|
||||
- group: DeepSeek
|
||||
tag: deepseek
|
||||
models:
|
||||
- model: DeepSeek-V3 (proxy)
|
||||
mad_tag: primus_pyt_megatron_lm_train_deepseek-v3-proxy
|
||||
config_name: deepseek_v3-pretrain.yaml
|
||||
- model: DeepSeek-V2-Lite
|
||||
mad_tag: primus_pyt_megatron_lm_train_deepseek-v2-lite-16b
|
||||
config_name: deepseek_v2_lite-pretrain.yaml
|
||||
- group: Mistral AI
|
||||
tag: mistral
|
||||
models:
|
||||
- model: Mixtral 8x7B
|
||||
mad_tag: primus_pyt_megatron_lm_train_mixtral-8x7b
|
||||
config_name: mixtral_8x7B_v0.1-pretrain.yaml
|
||||
- model: Mixtral 8x22B (proxy)
|
||||
mad_tag: primus_pyt_megatron_lm_train_mixtral-8x22b-proxy
|
||||
config_name: mixtral_8x22B_v0.1-pretrain.yaml
|
||||
- group: Qwen
|
||||
tag: qwen
|
||||
models:
|
||||
- model: Qwen 3 32B SFT
|
||||
mad_tag: primus_pyt_megatron_lm_train_qwen3-32b-sft
|
||||
- model: Qwen 3 32B LoRA
|
||||
mad_tag: primus_pyt_megatron_lm_train_qwen3-32b-lora
|
||||
config_name: primus_qwen2.5_7B-pretrain.yaml
|
||||
- model: Qwen 2.5 7B
|
||||
mad_tag: primus_pyt_megatron_lm_train_qwen2.5-7b
|
||||
config_name: primus_qwen2.5_7B-pretrain.yaml
|
||||
- model: Qwen 2.5 72B
|
||||
mad_tag: primus_pyt_megatron_lm_train_qwen2.5-72b
|
||||
config_name: qwen2.5_72B-pretrain.yaml
|
||||
-34
@@ -1,34 +0,0 @@
|
||||
docker:
|
||||
pull_tag: rocm/primus:v26.2
|
||||
docker_hub_url: https://hub.docker.com/layers/rocm/primus/v26.2/images/sha256-9148d1bfcd579bf92f44bd89090e0d8c958f149c134b4b34b9674ab559244585
|
||||
components:
|
||||
ROCm: 7.2.0
|
||||
PyTorch: 2.10.0a0+git449b176
|
||||
Python: "3.12.3"
|
||||
Transformer Engine: 2.8.0.dev0+51f74fa7
|
||||
Flash Attention: 2.8.3
|
||||
hipBLASLt: 1.2.0-de5c1aebb6
|
||||
Triton: 3.6.0
|
||||
RCCL: 2.27.7
|
||||
model_groups:
|
||||
- group: Meta Llama
|
||||
tag: llama
|
||||
models:
|
||||
- model: Llama 3.1 8B
|
||||
mad_tag: primus_pyt_train_llama-3.1-8b
|
||||
model_repo: Llama-3.1-8B
|
||||
url: https://huggingface.co/meta-llama/Llama-3.1-8B
|
||||
precision: BF16
|
||||
- model: Llama 3.1 70B
|
||||
mad_tag: primus_pyt_train_llama-3.1-70b
|
||||
model_repo: Llama-3.1-70B
|
||||
url: https://huggingface.co/meta-llama/Llama-3.1-70B
|
||||
precision: BF16
|
||||
- group: DeepSeek
|
||||
tag: deepseek
|
||||
models:
|
||||
- model: DeepSeek V3 16B
|
||||
mad_tag: primus_pyt_train_deepseek-v3-16b
|
||||
model_repo: DeepSeek-V3
|
||||
url: https://huggingface.co/deepseek-ai/DeepSeek-V3
|
||||
precision: BF16
|
||||
@@ -1,13 +1,13 @@
|
||||
docker:
|
||||
pull_tag: rocm/primus:v26.3
|
||||
docker_hub_url: https://hub.docker.com/layers/rocm/primus/v26.3/images/sha256-da50bfe9dc4bb70ad683d3ee4a176dccb66f596f48d30e8e52323b41892759b1
|
||||
pull_tag: rocm/primus:v26.2
|
||||
docker_hub_url: https://hub.docker.com/layers/rocm/primus/v26.2/images/sha256-9148d1bfcd579bf92f44bd89090e0d8c958f149c134b4b34b9674ab559244585
|
||||
components:
|
||||
ROCm: 7.2.1
|
||||
PyTorch: 2.10.0+git94c6e04
|
||||
ROCm: 7.2.0
|
||||
PyTorch: 2.10.0a0+git449b176
|
||||
Python: "3.12.3"
|
||||
Transformer Engine: 2.12.0.dev0+40434cf6
|
||||
Transformer Engine: 2.8.0.dev0+51f74fa7
|
||||
Flash Attention: 2.8.3
|
||||
hipBLASLt: 1.3.0-c4b2dc9869
|
||||
hipBLASLt: 1.2.0-de5c1aebb6
|
||||
Triton: 3.6.0
|
||||
RCCL: 2.27.7
|
||||
model_groups:
|
||||
@@ -41,15 +41,12 @@ model_groups:
|
||||
- model: Zebra-Llama 8B
|
||||
mad_tag: primus_pyt_megatron_lm_train_zebra-llama-8b
|
||||
config_name: zebra_llama_8b-pretrain.yaml
|
||||
- group: OpenAI GPT-OSS
|
||||
tag: gpt-oss
|
||||
models:
|
||||
- model: GPT-OSS-20B
|
||||
mad_tag: primus_pyt_megatron_lm_train_gpt-oss-20b
|
||||
config_name: gpt_oss_20B-BF16-pretrain.yaml
|
||||
- group: DeepSeek
|
||||
tag: deepseek
|
||||
models:
|
||||
- model: DeepSeek-V3 (proxy)
|
||||
mad_tag: primus_pyt_megatron_lm_train_deepseek-v3-proxy
|
||||
config_name: deepseek_v3-pretrain.yaml
|
||||
- model: DeepSeek-V2-Lite
|
||||
mad_tag: primus_pyt_megatron_lm_train_deepseek-v2-lite-16b
|
||||
config_name: deepseek_v2_lite-pretrain.yaml
|
||||
@@ -59,6 +56,9 @@ model_groups:
|
||||
- model: Mixtral 8x7B
|
||||
mad_tag: primus_pyt_megatron_lm_train_mixtral-8x7b
|
||||
config_name: mixtral_8x7B_v0.1-pretrain.yaml
|
||||
- model: Mixtral 8x22B (proxy)
|
||||
mad_tag: primus_pyt_megatron_lm_train_mixtral-8x22b-proxy
|
||||
config_name: mixtral_8x22B_v0.1-pretrain.yaml
|
||||
- group: Qwen
|
||||
tag: qwen
|
||||
models:
|
||||
@@ -67,9 +67,6 @@ model_groups:
|
||||
- model: Qwen 3 32B LoRA
|
||||
mad_tag: primus_pyt_megatron_lm_train_qwen3-32b-lora
|
||||
config_name: primus_qwen2.5_7B-pretrain.yaml
|
||||
- model: Qwen 3 30B A3B
|
||||
mad_tag: primus_pyt_megatron_lm_train_qwen3-30b-a3b
|
||||
config_name: qwen3_30B_A3B-pretrain.yaml
|
||||
- model: Qwen 2.5 7B
|
||||
mad_tag: primus_pyt_megatron_lm_train_qwen2.5-7b
|
||||
config_name: primus_qwen2.5_7B-pretrain.yaml
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
docker:
|
||||
pull_tag: rocm/primus:v26.3
|
||||
docker_hub_url: https://hub.docker.com/layers/rocm/primus/v26.3/images/sha256-da50bfe9dc4bb70ad683d3ee4a176dccb66f596f48d30e8e52323b41892759b1
|
||||
pull_tag: rocm/primus:v26.2
|
||||
docker_hub_url: https://hub.docker.com/layers/rocm/primus/v26.2/images/sha256-9148d1bfcd579bf92f44bd89090e0d8c958f149c134b4b34b9674ab559244585
|
||||
components:
|
||||
ROCm: 7.2.1
|
||||
PyTorch: 2.10.0+git94c6e04
|
||||
ROCm: 7.2.0
|
||||
PyTorch: 2.10.0a0+git449b176
|
||||
Python: "3.12.3"
|
||||
Transformer Engine: 2.12.0.dev0+40434cf6
|
||||
Transformer Engine: 2.8.0.dev0+51f74fa7
|
||||
Flash Attention: 2.8.3
|
||||
hipBLASLt: 1.3.0-c4b2dc9869
|
||||
hipBLASLt: 1.2.0-de5c1aebb6
|
||||
Triton: 3.6.0
|
||||
RCCL: 2.27.7
|
||||
model_groups:
|
||||
|
||||
@@ -127,7 +127,7 @@ Download the base model and fine-tuning dataset
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
hf auth login
|
||||
huggingface-cli login
|
||||
|
||||
.. note::
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
.. meta::
|
||||
:description: Learn about vLLM V1 inference tuning on AMD Instinct GPUs for optimal performance.
|
||||
:keywords: AMD, Instinct, MI300X, MI325X, MI350X, MI355X, HPC, tuning, BIOS settings, NBIO, ROCm,
|
||||
:keywords: AMD, Instinct, MI300X, HPC, tuning, BIOS settings, NBIO, ROCm,
|
||||
environment variable, performance, HIP, Triton, PyTorch TunableOp, vLLM, RCCL,
|
||||
MIOpen, GPU, resource utilization
|
||||
|
||||
@@ -25,7 +25,7 @@ Instinct MI300X, MI325X, MI350X, and MI355X GPUs. Learn how to:
|
||||
Performance environment variables
|
||||
=================================
|
||||
|
||||
The following variables are generally useful for Instinct MI300X/MI325X/MI350X/MI355X GPUs and vLLM:
|
||||
The following variables are generally useful for Instinct MI300X/MI355X GPUs and vLLM:
|
||||
|
||||
* **HIP and math libraries**
|
||||
|
||||
@@ -34,9 +34,6 @@ The following variables are generally useful for Instinct MI300X/MI325X/MI350X/M
|
||||
:doc:`vLLM ROCm Docker images
|
||||
</how-to/rocm-for-ai/inference/benchmark-docker/vllm>`. Bare-metal users
|
||||
should set this manually.
|
||||
* ``export SAFETENSORS_FAST_GPU=1`` — enables GPU-accelerated safetensors
|
||||
loading, significantly reducing model load time for large models. Already
|
||||
set in vLLM ROCm Docker images. Bare-metal users should set this manually.
|
||||
* ``export TORCH_BLAS_PREFER_HIPBLASLT=1`` — explicitly prefers hipBLASLt
|
||||
over hipBLAS for GEMM operations. By default, PyTorch uses heuristics to
|
||||
choose the best BLAS library. Setting this can improve linear layer
|
||||
@@ -45,7 +42,7 @@ The following variables are generally useful for Instinct MI300X/MI325X/MI350X/M
|
||||
* **RCCL (collectives for multi-GPU)**
|
||||
|
||||
* ``export NCCL_MIN_NCHANNELS=112`` — increases RCCL channels from default
|
||||
(typically 32-64) to 112 on the Instinct MI300X/MI325X. **Only beneficial for
|
||||
(typically 32-64) to 112 on the Instinct MI300X. **Only beneficial for
|
||||
multi-GPU distributed workloads** (tensor parallelism, pipeline
|
||||
parallelism). Single-GPU inference does not need this.
|
||||
|
||||
@@ -54,31 +51,31 @@ The following variables are generally useful for Instinct MI300X/MI325X/MI350X/M
|
||||
AITER (AI Tensor Engine for ROCm) switches
|
||||
==========================================
|
||||
|
||||
AITER (AI Tensor Engine for ROCm) provides ROCm-specific fused kernels optimized for Instinct MI350 Series and MI300X/MI325X GPUs in vLLM V1.
|
||||
AITER (AI Tensor Engine for ROCm) provides ROCm-specific fused kernels optimized for Instinct MI350 Series and MI300X GPUs in vLLM V1.
|
||||
|
||||
Enable all AITER optimizations with a single master switch:
|
||||
How AITER flags work:
|
||||
|
||||
* ``VLLM_ROCM_USE_AITER`` is the master switch (defaults to ``False``/``0``).
|
||||
* Individual feature flags (``VLLM_ROCM_USE_AITER_LINEAR``, ``VLLM_ROCM_USE_AITER_MOE``, and so on) default to ``True`` but only activate when the master switch is enabled.
|
||||
* To enable a specific AITER feature, you must set both ``VLLM_ROCM_USE_AITER=1`` and the specific feature flag to ``1``.
|
||||
|
||||
Quick start examples:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# Enable all AITER optimizations (recommended for most workloads)
|
||||
export VLLM_ROCM_USE_AITER=1
|
||||
vllm serve MODEL_NAME
|
||||
|
||||
Most individual AITER sub-flags default to ``1`` when the master switch is on,
|
||||
while specialized features retain the defaults listed below. You rarely need to
|
||||
change them. To select a specific attention backend, use ``--attention-backend``
|
||||
(see :ref:`backend selection <vllm-optimization-aiter-backend-selection>`).
|
||||
# Enable AITER Fused MoE and enable Triton Prefill-Decode (split) attention
|
||||
export VLLM_ROCM_USE_AITER=1
|
||||
export VLLM_V1_USE_PREFILL_DECODE_ATTENTION=1
|
||||
export VLLM_ROCM_USE_AITER_MHA=0
|
||||
vllm serve MODEL_NAME
|
||||
|
||||
**Flags you might adjust:**
|
||||
|
||||
* ``VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT=1`` — Set for high-concurrency MHA workloads (≥32 concurrent requests) with ``ROCM_AITER_FA``. Defaults to ``0``.
|
||||
* ``VLLM_ROCM_USE_AITER_MOE=0`` — Disable only if you hit ``RuntimeError: wrong! device_gemm ...``. Try ``AITER_ONLINE_TUNE=1`` first. See :ref:`AITER MoE requirements <vllm-optimization-aiter-moe-requirements>`.
|
||||
* ``VLLM_ROCM_USE_AITER=0`` — Disable AITER entirely to fall back to Triton kernels (for debugging).
|
||||
|
||||
Advanced: individual AITER flags
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
The following table lists AITER-related sub-flags for fine-grained control. Most
|
||||
users do not need to modify these; the default behavior for each flag is listed below.
|
||||
# Disable AITER entirely (i.e, use vLLM Triton Unified Attention Kernel)
|
||||
export VLLM_ROCM_USE_AITER=0
|
||||
vllm serve MODEL_NAME
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
@@ -88,135 +85,233 @@ users do not need to modify these; the default behavior for each flag is listed
|
||||
- Description (default behavior)
|
||||
|
||||
* - ``VLLM_ROCM_USE_AITER``
|
||||
- Master switch to enable AITER kernels (``0`` by default). All other ``VLLM_ROCM_USE_AITER_*`` flags require this to be set to ``1``.
|
||||
- Master switch to enable AITER kernels (``0``/``False`` by default). All other ``VLLM_ROCM_USE_AITER_*`` flags require this to be set to ``1``.
|
||||
|
||||
* - ``VLLM_ROCM_USE_AITER_LINEAR``
|
||||
- Use AITER quantization operators + GEMM for linear layers (defaults to ``1`` when AITER is on). Accelerates matrix multiplications in all transformer layers. **Recommended to keep enabled**.
|
||||
- Use AITER quantization operators + GEMM for linear layers (defaults to ``True`` when AITER is on). Accelerates matrix multiplications in all transformer layers. **Recommended to keep enabled**.
|
||||
|
||||
* - ``VLLM_ROCM_USE_AITER_MOE``
|
||||
- Use AITER fused-MoE kernels (defaults to ``1`` when AITER is on). Accelerates Mixture-of-Experts routing and computation. See the note on :ref:`AITER MoE requirements <vllm-optimization-aiter-moe-requirements>`.
|
||||
- Use AITER fused-MoE kernels (defaults to ``True`` when AITER is on). Accelerates Mixture-of-Experts routing and computation. See the note on :ref:`AITER MoE requirements <vllm-optimization-aiter-moe-requirements>`.
|
||||
|
||||
* - ``VLLM_ROCM_USE_AITER_RMSNORM``
|
||||
- Use AITER RMSNorm kernels (defaults to ``1`` when AITER is on). Accelerates normalization layers. **Recommended: keep enabled.**
|
||||
- Use AITER RMSNorm kernels (defaults to ``True`` when AITER is on). Accelerates normalization layers. **Recommended: keep enabled.**
|
||||
|
||||
* - ``VLLM_ROCM_USE_AITER_MLA``
|
||||
- Use AITER Multi-head Latent Attention for supported models, for example, DeepSeek-V3/R1 (defaults to ``1`` when AITER is on). See the section on :ref:`AITER MLA requirements <vllm-optimization-aiter-mla-requirements>`.
|
||||
- Use AITER Multi-head Latent Attention for supported models, for example, DeepSeek-V3/R1 (defaults to ``True`` when AITER is on). See the section on :ref:`AITER MLA requirements <vllm-optimization-aiter-mla-requirements>`.
|
||||
|
||||
* - ``VLLM_ROCM_USE_AITER_MHA``
|
||||
- Use AITER Multi-Head Attention kernels (defaults to ``1`` when AITER is on; set to ``0`` to use Triton attention backends or ``ROCM_ATTN`` backend instead). See :ref:`attention backend selection <vllm-optimization-aiter-backend-selection>`.
|
||||
- Use AITER Multi-Head Attention kernels (defaults to ``True`` when AITER is on; set to ``0`` to use Triton attention backends and Prefill-Decode attention backend instead). See :ref:`attention backend selection <vllm-optimization-aiter-backend-selection>`.
|
||||
|
||||
* - ``VLLM_ROCM_USE_AITER_UNIFIED_ATTENTION``
|
||||
- Enable AITER's optimized unified attention kernel (defaults to ``0``). Only takes effect when AITER is enabled and AITER MHA is disabled (``VLLM_ROCM_USE_AITER_MHA=0``). When set to ``0``, falls back to vLLM's Triton unified attention. Can also be enabled via ``--attention-backend ROCM_AITER_UNIFIED_ATTN``.
|
||||
- Enable AITER's optimized unified attention kernel (defaults to ``False``). Only takes effect when: AITER is enabled; unified attention mode is active (``VLLM_V1_USE_PREFILL_DECODE_ATTENTION=0``); and AITER MHA is disabled (``VLLM_ROCM_USE_AITER_MHA=0``). When disabled, falls back to vLLM's Triton unified attention.
|
||||
|
||||
* - ``VLLM_ROCM_USE_AITER_FP8BMM``
|
||||
- Use AITER ``FP8`` batched matmul (defaults to ``1`` when AITER is on). Fuses ``FP8`` per-token quantization with batched GEMM (used in MLA models like DeepSeek-V3).
|
||||
|
||||
* - ``VLLM_ROCM_USE_AITER_FP4BMM``
|
||||
- Use AITER ``FP4`` batched matmul (defaults to ``1`` when AITER is on). Fuses ``FP4`` per-token quantization with batched GEMM (used in MLA models like DeepSeek-V3). Requires an Instinct MI350X/MI355X GPU.
|
||||
|
||||
* - ``VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS``
|
||||
- Fuse shared expert computation into the AITER fused-MoE kernel (defaults to ``0``). Applies to MoE models with shared experts (for example, DeepSeek-V3/R1 with 1 shared expert). Requires SiLU/GELU activation (``is_act_and_mul``). Incompatible with `MoRI <https://github.com/ROCm/mori#mori>`__ scheduling — disable with ``VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=0`` if using MoRI.
|
||||
|
||||
* - ``VLLM_ROCM_USE_AITER_FP4_ASM_GEMM``
|
||||
- Enable AITER assembly (HIP) FP4 GEMM kernels for MXFP4-quantized models (defaults to ``0``). When set to ``1``, uses hand-tuned ASM kernels instead of Triton for ``FP4×FP4`` weight GEMMs — faster at small batch sizes (``M`` ≤ 64). Requires Instinct MI350X/MI355X (``supports_mx()``). Combine with ``--quantization quark`` for MXFP4 models.
|
||||
- Use AITER ``FP8`` batched matmul (defaults to ``True`` when AITER is on). Fuses ``FP8`` per-token quantization with batched GEMM (used in MLA models like DeepSeek-V3). Requires an Instinct MI300X/MI355X GPU.
|
||||
|
||||
* - ``VLLM_ROCM_USE_SKINNY_GEMM``
|
||||
- Prefer skinny-GEMM kernel variants for small batch sizes (defaults to ``1``). Improves performance when ``M`` dimension is small. **Recommended to keep enabled**.
|
||||
- Prefer skinny-GEMM kernel variants for small batch sizes (defaults to ``True``). Improves performance when ``M`` dimension is small. **Recommended to keep enabled**.
|
||||
|
||||
* - ``VLLM_ROCM_FP8_PADDING``
|
||||
- Pad ``FP8`` linear weight tensors to improve memory locality (defaults to ``1``). Minor memory overhead for better performance.
|
||||
- Pad ``FP8`` linear weight tensors to improve memory locality (defaults to ``True``). Minor memory overhead for better performance.
|
||||
|
||||
* - ``VLLM_ROCM_MOE_PADDING``
|
||||
- Pad MoE weight tensors for better memory access patterns (defaults to ``1``). Same memory/performance tradeoff as ``FP8`` padding.
|
||||
|
||||
* - ``VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT``
|
||||
- This only affects the ``ROCM_AITER_FA`` backend. When set to ``0``, it uses the HIP Paged Attention implementation ``torch.ops.aiter.paged_attention_v1`` from AITER. **Good for low concurrency (for example, concurrency ≤32).** When set to ``1``, it uses the ASM Paged Attention Kernel ``pa_fwd_asm`` from AITER. **Good for high concurrency (for example, concurrency ≥32).** (Defaults to ``0`` when AITER is on.)
|
||||
- Pad MoE weight tensors for better memory access patterns (defaults to ``True``). Same memory/performance tradeoff as ``FP8`` padding.
|
||||
|
||||
* - ``VLLM_ROCM_CUSTOM_PAGED_ATTN``
|
||||
- Use custom paged-attention decode kernel when ``ROCM_ATTN`` backend is selected (defaults to ``1``). See :ref:`Attention backend selection with AITER <vllm-optimization-aiter-backend-selection>`.
|
||||
- Use custom paged-attention decode kernel when Prefill-Decode attention backend is selected (defaults to ``True``). See :ref:`Attention backend selection with AITER <vllm-optimization-aiter-backend-selection>`.
|
||||
|
||||
.. note::
|
||||
|
||||
When ``VLLM_ROCM_USE_AITER=1``, most AITER component flags (``LINEAR``,
|
||||
``MOE``, ``RMSNORM``, ``MLA``, ``MHA``, ``FP8BMM``) automatically default to
|
||||
``True``. You typically only need to set the master switch
|
||||
``VLLM_ROCM_USE_AITER=1`` to enable all optimizations. ROCm provides a
|
||||
prebuilt optimized Docker image for validating the performance of LLM
|
||||
inference with vLLM on MI300X Series GPUs. The Docker image includes ROCm,
|
||||
vLLM, and PyTorch. For more information, see
|
||||
:doc:`/how-to/rocm-for-ai/inference/benchmark-docker/vllm`.
|
||||
|
||||
.. _vllm-optimization-aiter-moe-requirements:
|
||||
|
||||
AITER MoE requirements (Mixtral, DeepSeek-V2/V3, Qwen-MoE models)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
``VLLM_ROCM_USE_AITER_MOE`` enables AITER's optimized Mixture-of-Experts kernels, such as expert routing (topk selection) and expert computation for better performance.
|
||||
|
||||
Applicable models:
|
||||
|
||||
* Mixtral series: for example, Mixtral-8x7B / Mixtral-8x22B
|
||||
* Llama-4 family: for example, Llama-4-Scout-17B-16E / Llama-4-Maverick-17B-128E
|
||||
* DeepSeek family: DeepSeek-V2 / DeepSeek-V3 / DeepSeek-R1
|
||||
* Qwen family: Qwen1.5-MoE / Qwen2-MoE / Qwen2.5-MoE series
|
||||
* Other MoE architectures
|
||||
|
||||
When to enable:
|
||||
|
||||
* **Enable (default):** For all MoE models on the Instinct MI300X/MI355X for best throughput
|
||||
* **Disable:** Only for debugging or if you encounter numerical issues
|
||||
|
||||
Example usage:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# Standard MoE model (Mixtral)
|
||||
VLLM_ROCM_USE_AITER=1 vllm serve mistralai/Mixtral-8x7B-Instruct-v0.1
|
||||
|
||||
# Hybrid MoE+MLA model (DeepSeek-V3) - requires both MOE and MLA flags
|
||||
VLLM_ROCM_USE_AITER=1 vllm serve deepseek-ai/DeepSeek-V3 \
|
||||
--block-size 1 \
|
||||
--tensor-parallel-size 8
|
||||
|
||||
.. _vllm-optimization-aiter-mla-requirements:
|
||||
.. _vllm-optimization-aiter-mla-sparse-requirements:
|
||||
|
||||
AITER MLA requirements (DeepSeek-V3/R1 models)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
``VLLM_ROCM_USE_AITER_MLA`` enables AITER MLA (Multi-head Latent Attention) optimization for supported models. Defaults to **True** when AITER is on.
|
||||
|
||||
Critical requirement:
|
||||
|
||||
* **Must** explicitly set ``--block-size 1``
|
||||
|
||||
.. important::
|
||||
|
||||
If you omit ``--block-size 1``, vLLM will raise an error rather than defaulting to 1.
|
||||
|
||||
Applicable models:
|
||||
|
||||
* DeepSeek-V3 / DeepSeek-R1
|
||||
* DeepSeek-V2
|
||||
* Other models using multi-head latent attention (MLA) architecture
|
||||
|
||||
Example usage:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# DeepSeek-R1 with AITER MLA (requires 8 GPUs)
|
||||
VLLM_ROCM_USE_AITER=1 vllm serve deepseek-ai/DeepSeek-R1 \
|
||||
--block-size 1 \
|
||||
--tensor-parallel-size 8
|
||||
|
||||
.. _vllm-optimization-aiter-backend-selection:
|
||||
|
||||
Attention backend selection with AITER
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Most models work out of the box with ``VLLM_ROCM_USE_AITER=1`` — vLLM auto-selects
|
||||
the optimal backend. Use ``--attention-backend`` to override the auto-selected backend.
|
||||
Understanding which attention backend to use helps optimize your deployment.
|
||||
|
||||
.. code-block:: bash
|
||||
Quick reference: Which attention backend will I get?
|
||||
|
||||
export VLLM_ROCM_USE_AITER=1
|
||||
vllm serve <your-model> --tensor-parallel-size <tp>
|
||||
Default behavior (no configuration)
|
||||
|
||||
.. note::
|
||||
Always set ``VLLM_ROCM_USE_AITER=1`` even when using ``--attention-backend`` explicitly.
|
||||
``--attention-backend`` only overrides the attention kernel; ``VLLM_ROCM_USE_AITER=1``
|
||||
is still required to enable AITER for GEMM, RMSNorm, and MoE kernels.
|
||||
The Radeon/fallback backends (``ROCM_ATTN``, ``TRITON_MLA``) are the exception —
|
||||
they do not use AITER and do not require the env var.
|
||||
Without setting any environment variables, vLLM uses:
|
||||
|
||||
The table below shows which backend is selected per model type and how to tune it.
|
||||
* **vLLM Triton Unified Attention** — A single Triton kernel handling both prefill and decode phases
|
||||
* Works on all ROCm platforms
|
||||
* Good baseline performance
|
||||
|
||||
**Recommended**: Enable AITER (set ``VLLM_ROCM_USE_AITER=1``)
|
||||
|
||||
When you enable AITER, the backend is automatically selected based on your model:
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
Is your model using MLA architecture? (DeepSeek-V3/R1/V2)
|
||||
├─ YES → AITER MLA Backend
|
||||
│ • Requires --block-size 1
|
||||
│ • Best performance for MLA models
|
||||
│ • Automatically selected
|
||||
│
|
||||
└─ NO → AITER MHA Backend
|
||||
• For standard transformer models (Llama, Mistral, etc.)
|
||||
• Optimized for Instinct MI300X/MI355X
|
||||
• Automatically selected
|
||||
|
||||
**Advanced**: Manual backend selection
|
||||
|
||||
Most users won't need this, but you can override the defaults:
|
||||
|
||||
.. list-table::
|
||||
:widths: 40 60
|
||||
:header-rows: 1
|
||||
:widths: 15 18 35 32
|
||||
|
||||
* - Model type
|
||||
- Backend
|
||||
- How to enable
|
||||
- Tuning tips
|
||||
* - To use this backend
|
||||
- Set these flags
|
||||
|
||||
* - **MHA models** (Llama, Mistral, Qwen, Mixtral, MiniMax-M2.5)
|
||||
- **ROCM_AITER_FA** (recommended, auto-selected)
|
||||
- ``VLLM_ROCM_USE_AITER=1`` (auto-selected). To override: ``VLLM_ROCM_USE_AITER=1 --attention-backend ROCM_AITER_FA``. Add ``VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT=1`` for shuffled KV cache.
|
||||
- **2.7–4.4x TPS** over legacy ``ROCM_ATTN``. Set ``VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT=1`` for **15–20% decode improvement** at high concurrency. Low TTFT: ``--max-num-batched-tokens`` ≤ 8k–16k. High throughput: ≥ 32k with ``cudagraph_mode=FULL``.
|
||||
* - AITER MLA (MLA models only)
|
||||
- ``VLLM_ROCM_USE_AITER=1`` (auto-selects for DeepSeek-V3/R1)
|
||||
|
||||
* - **MLA models** (DeepSeek-V3/R1/V2, Kimi-K2.5, Mistral-Large-3-675B)
|
||||
- **ROCM_AITER_MLA** (recommended, auto-selected)
|
||||
- ``VLLM_ROCM_USE_AITER=1`` (auto-selected). To override: ``VLLM_ROCM_USE_AITER=1 --attention-backend ROCM_AITER_MLA``. ``--block-size 1`` is no longer mandatory (vLLM ≥0.14); still recommended for prefix-caching workloads.
|
||||
- **1.2–1.5x higher TPS** over ``TRITON_MLA``. Supports uniform-batch CUDA graphs and MTP. On MI300X/MI325X (gfx942), ``ROCM_AITER_TRITON_MLA`` may show 2–3% higher TPS. On MI355X (gfx950), ``ROCM_AITER_MLA`` is preferred (uses AITER assembly MHA for prefill).
|
||||
* - AITER MHA (standard models)
|
||||
- ``VLLM_ROCM_USE_AITER=1`` (auto-selects for non-MLA models)
|
||||
|
||||
* - **DSA models** (DeepSeek-V3.2, GLM-5)
|
||||
- **ROCM_AITER_MLA_SPARSE** (auto-selected)
|
||||
- ``VLLM_ROCM_USE_AITER=1`` — auto-detected from ``index_topk`` in model config. Requires ``--block-size 1``.
|
||||
- Instinct MI300X/MI325X/MI350X/MI355X only.
|
||||
* - vLLM Triton Unified (default)
|
||||
- ``VLLM_ROCM_USE_AITER=0`` (or unset)
|
||||
|
||||
* - **gpt-oss models** (gpt-oss-120b/20b)
|
||||
- **ROCM_AITER_UNIFIED_ATTN**
|
||||
- ``VLLM_ROCM_USE_AITER=1 --attention-backend ROCM_AITER_UNIFIED_ATTN``
|
||||
-
|
||||
* - Triton Prefill-Decode (split) without AITER
|
||||
- | ``VLLM_V1_USE_PREFILL_DECODE_ATTENTION=1``
|
||||
|
||||
* - **Radeon / fallback**
|
||||
- **ROCM_ATTN** (MHA) or **TRITON_MLA** (MLA)
|
||||
- ``--attention-backend ROCM_ATTN`` or ``TRITON_MLA``
|
||||
- ``ROCM_ATTN`` is preferred over ``TRITON_ATTN`` — it uses a custom HIP paged-attention kernel for decode when the model's KV head size is supported, and falls back to Triton only when not. If ``ROCM_ATTN`` is slow for your model (unsupported head size triggers Triton decode), try ``TRITON_ATTN``. Both work on Radeon GPUs.
|
||||
* - Triton Prefill-Decode (split) along with AITER Fused-MoE
|
||||
- | ``VLLM_ROCM_USE_AITER=1``
|
||||
| ``VLLM_ROCM_USE_AITER_MHA=0``
|
||||
| ``VLLM_V1_USE_PREFILL_DECODE_ATTENTION=1``
|
||||
|
||||
.. note::
|
||||
**MoE models** (Mixtral, Llama-4-Scout/Maverick, DeepSeek-V2/V3/R1, Kimi-K2.5, MiniMax-M2.5, GLM-5, Qwen-MoE): AITER MoE kernels activate automatically with ``VLLM_ROCM_USE_AITER=1`` — no extra attention backend flags needed. If you hit ``RuntimeError: wrong! device_gemm ...``, set ``AITER_ONLINE_TUNE=1`` and retry. Only disable MoE kernels (``VLLM_ROCM_USE_AITER_MOE=0``) if that also fails.
|
||||
|
||||
Once AITER is configured, see `Parallelism strategies (run vLLM on multiple GPUs)`_ for TP/DP/EP choices — especially for MLA and MoE models where the wrong strategy wastes memory or throughput.
|
||||
* - AITER Unified Attention
|
||||
- | ``VLLM_ROCM_USE_AITER=1``
|
||||
| ``VLLM_ROCM_USE_AITER_MHA=0``
|
||||
| ``VLLM_ROCM_USE_AITER_UNIFIED_ATTENTION=1``
|
||||
|
||||
**Quick start examples**:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# DSA model (DeepSeek-V3.2) — backend auto-selected from model config
|
||||
VLLM_ROCM_USE_AITER=1 vllm serve deepseek-ai/DeepSeek-V3.2 \
|
||||
# Recommended: Standard model with AITER (Llama, Mistral, Qwen, etc.)
|
||||
VLLM_ROCM_USE_AITER=1 vllm serve meta-llama/Llama-3.3-70B-Instruct
|
||||
|
||||
# MLA model with AITER (DeepSeek-V3/R1)
|
||||
VLLM_ROCM_USE_AITER=1 vllm serve deepseek-ai/DeepSeek-R1 \
|
||||
--block-size 1 \
|
||||
--tensor-parallel-size 8
|
||||
|
||||
# Explicitly select a backend for MLA models
|
||||
VLLM_ROCM_USE_AITER=1 vllm serve deepseek-ai/DeepSeek-R1-0528 \
|
||||
--tensor-parallel-size 8 \
|
||||
--attention-backend ROCM_AITER_MLA
|
||||
# Advanced: Use Prefill-Decode split (for short input cases) with AITER Fused-MoE
|
||||
VLLM_ROCM_USE_AITER=1 \
|
||||
VLLM_ROCM_USE_AITER_MHA=0 \
|
||||
VLLM_V1_USE_PREFILL_DECODE_ATTENTION=1 \
|
||||
vllm serve meta-llama/Llama-4-Scout-17B-16E
|
||||
|
||||
# MHA model with shuffled KV cache layout for high concurrency
|
||||
VLLM_ROCM_USE_AITER=1 VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT=1 \
|
||||
vllm serve meta-llama/Llama-3.3-70B-Instruct \
|
||||
--attention-backend ROCM_AITER_FA
|
||||
**Which backend should I choose?**
|
||||
|
||||
.. list-table::
|
||||
:widths: 30 70
|
||||
:header-rows: 1
|
||||
|
||||
* - Your use case
|
||||
- Recommended backend
|
||||
|
||||
* - **Standard transformer models** (Llama, Mistral, Qwen, Mixtral)
|
||||
- **AITER MHA** (``VLLM_ROCM_USE_AITER=1``) — **Recommended for most workloads** on Instinct MI300X/MI355X. Provides optimized attention kernels for both prefill and decode phases.
|
||||
|
||||
* - **MLA models** (DeepSeek-V3/R1/V2)
|
||||
- **AITER MLA** (auto-selected with ``VLLM_ROCM_USE_AITER=1``) — Required for optimal performance, must use ``--block-size 1``
|
||||
|
||||
* - **gpt-oss models** (gpt-oss-120b/20b)
|
||||
- **AITER Unified Attention** (``VLLM_ROCM_USE_AITER=1``, ``VLLM_ROCM_USE_AITER_MHA=0``, ``VLLM_ROCM_USE_AITER_UNIFIED_ATTENTION=1``) — Required for optimal performance
|
||||
|
||||
* - **Debugging or compatibility**
|
||||
- **vLLM Triton Unified** (default with ``VLLM_ROCM_USE_AITER=0``) — Generic fallback, works everywhere
|
||||
|
||||
**Important notes:**
|
||||
|
||||
* **AITER MHA and AITER MLA are mutually exclusive** — vLLM automatically detects MLA models and selects the appropriate backend
|
||||
* **For 95% of users:** Simply set ``VLLM_ROCM_USE_AITER=1`` and let vLLM choose the right backend
|
||||
* When in doubt, start with AITER enabled (the recommended configuration) and profile your specific workload
|
||||
|
||||
Backend choice quick recipes
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
* **Standard transformers (any prompt length):** Start with ``VLLM_ROCM_USE_AITER=1`` → AITER MHA. For CUDA graph modes, see architecture-specific guidance below (Dense vs MoE models have different optimal modes).
|
||||
* **Latency-sensitive chat (low TTFT):** keep ``--max-num-batched-tokens`` ≤ **8k–16k** with AITER.
|
||||
* **Streaming decode (low ITL):** raise ``--max-num-batched-tokens`` to **32k–64k**.
|
||||
* **Offline max throughput:** ``--max-num-batched-tokens`` ≥ **32k** with ``cudagraph_mode=FULL``.
|
||||
|
||||
**How to verify which backend is active**
|
||||
|
||||
@@ -225,13 +320,68 @@ Check vLLM's startup logs to confirm which attention backend is being used:
|
||||
.. code-block:: bash
|
||||
|
||||
# Start vLLM and check logs
|
||||
VLLM_ROCM_USE_AITER=1 vllm serve meta-llama/Llama-3.3-70B-Instruct 2>&1 | grep -i "using.*backend"
|
||||
VLLM_ROCM_USE_AITER=1 vllm serve meta-llama/Llama-3.3-70B-Instruct 2>&1 | grep -i attention
|
||||
|
||||
Look for ``Using <backend_name> backend.`` in the startup output — for example,
|
||||
``Using ROCM_AITER_FA backend.``
|
||||
**Expected log messages:**
|
||||
|
||||
For in-depth architecture and benchmarks of all 7 ROCm attention backends, see the
|
||||
`ROCm Attention Backend blog post <https://vllm.ai/blog/rocm-attention-backend>`_.
|
||||
* AITER MHA: ``Using Aiter Flash Attention backend on V1 engine.``
|
||||
* AITER MLA: ``Using AITER MLA backend on V1 engine.``
|
||||
* vLLM Triton MLA: ``Using Triton MLA backend on V1 engine.``
|
||||
* vLLM Triton Unified: ``Using Triton Attention backend on V1 engine.``
|
||||
* AITER Triton Unified: ``Using Aiter Unified Attention backend on V1 engine.``
|
||||
* AITER Triton Prefill-Decode: ``Using Rocm Attention backend on V1 engine.``
|
||||
|
||||
Attention backend technical details
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
This section provides technical details about vLLM's attention backends on ROCm.
|
||||
|
||||
vLLM V1 on ROCm provides these attention implementations:
|
||||
|
||||
1. **vLLM Triton Unified Attention** (default when AITER is **off**)
|
||||
|
||||
* Single unified Triton kernel handling both chunked prefill and decode phases
|
||||
* Generic implementation that works across all ROCm platforms
|
||||
* Good baseline performance
|
||||
* Automatically selected when ``VLLM_ROCM_USE_AITER=0`` (or unset)
|
||||
* Supports GPT-OSS
|
||||
|
||||
2. **AITER Triton Unified Attention** (advanced, requires manual configuration)
|
||||
|
||||
* The AMD optimized unified Triton kernel
|
||||
* Enable with ``VLLM_ROCM_USE_AITER=1``, ``VLLM_ROCM_USE_AITER_MHA=0``, and ``VLLM_ROCM_USE_AITER_UNIFIED_ATTENTION=1``.
|
||||
* Only useful for specific workloads. Most users should use AITER MHA instead.
|
||||
* Recommended this backend when running GPT-OSS.
|
||||
|
||||
3. **AITER Triton Prefill–Decode Attention** (hybrid, Instinct MI300X-optimized)
|
||||
|
||||
* Enable with ``VLLM_V1_USE_PREFILL_DECODE_ATTENTION=1``
|
||||
* Uses separate kernels for prefill and decode phases:
|
||||
|
||||
* **Prefill**: ``context_attention_fwd`` Triton kernel
|
||||
* **Primary decode**: ``torch.ops._rocm_C.paged_attention`` (custom ROCm kernel optimized for head sizes 64/128, block sizes 16/32, GQA 1–16, context ≤131k; sliding window not supported)
|
||||
* **Fallback decode**: ``kernel_paged_attention_2d`` Triton kernel when shapes don't meet primary decode requirements
|
||||
|
||||
* Usually better compared to unified Triton kernels
|
||||
* Performance vs AITER MHA varies: AITER MHA is typically faster overall, but Prefill-Decode split may win in short input scenarios
|
||||
* The custom paged attention decode kernel is controlled by ``VLLM_ROCM_CUSTOM_PAGED_ATTN`` (default **True**)
|
||||
|
||||
4. **AITER Multi-Head Attention (MHA)** (default when AITER is **on**)
|
||||
|
||||
* Controlled by ``VLLM_ROCM_USE_AITER_MHA`` (**1** = enabled)
|
||||
* Best all-around performance for standard transformer models
|
||||
* Automatically selected when ``VLLM_ROCM_USE_AITER=1`` and model is not MLA
|
||||
|
||||
5. **vLLM Triton Multi-head Latent Attention (MLA)** (for DeepSeek-V3/R1/V2)
|
||||
|
||||
* Automatically selected when ``VLLM_ROCM_USE_AITER=0`` (or unset)
|
||||
|
||||
6. **AITER Multi-head Latent Attention (MLA)** (for DeepSeek-V3/R1/V2)
|
||||
|
||||
* Controlled by ``VLLM_ROCM_USE_AITER_MLA`` (``1`` = enabled)
|
||||
* Required for optimal performance on MLA architecture models
|
||||
* Automatically selected when ``VLLM_ROCM_USE_AITER=1`` and model uses MLA
|
||||
* Requires ``--block-size 1``
|
||||
|
||||
Quick Reduce (large all-reduces on ROCm)
|
||||
========================================
|
||||
@@ -246,7 +396,7 @@ It supports FP16/BF16 as well as symmetric INT8/INT6/INT4 quantized all-reduce (
|
||||
Control via:
|
||||
|
||||
* ``VLLM_ROCM_QUICK_REDUCE_QUANTIZATION`` ∈ ``["NONE","FP","INT8","INT6","INT4"]`` (default ``NONE``).
|
||||
* ``VLLM_ROCM_QUICK_REDUCE_CAST_BF16_TO_FP16``: cast BF16 input to FP16 (``1`` by default for performance).
|
||||
* ``VLLM_ROCM_QUICK_REDUCE_CAST_BF16_TO_FP16``: cast BF16 input to FP16 (``1/True`` by default for performance).
|
||||
* ``VLLM_ROCM_QUICK_REDUCE_MAX_SIZE_BYTES_MB``: cap the preset buffer (default ``NONE`` ≈ ``2048`` MB).
|
||||
|
||||
Quick Reduce tends to help **throughput** at higher TP counts (for example, 4–8) with many concurrent requests.
|
||||
@@ -263,34 +413,12 @@ vLLM supports the following parallelism strategies:
|
||||
|
||||
For more details, see `Parallelism and scaling <https://docs.vllm.ai/en/stable/serving/parallelism_scaling.html>`_.
|
||||
|
||||
**Quick-reference decision table:**
|
||||
**Choosing the right strategy:**
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 30 35 35
|
||||
|
||||
* - Model type
|
||||
- Low concurrency (≤128 requests)
|
||||
- High concurrency (≥512 requests)
|
||||
|
||||
* - **Dense** (for example, Llama, Qwen-dense, Mistral-dense)
|
||||
- TP only
|
||||
- TP + independent DP replicas (your own load balancer)
|
||||
|
||||
* - **MoE, standard density ≥3%** (for example, Qwen3-235B-A22B, DeepSeek-V3/R1)
|
||||
- TP + EP
|
||||
- DP + EP
|
||||
|
||||
* - **MoE, ultra-sparse <1%** (for example, Llama-4-Maverick at 0.78%)
|
||||
- TP only — **no EP** (AllToAll overhead exceeds benefit)
|
||||
- DP only — **no EP**
|
||||
|
||||
* - **MLA models** (for example, DeepSeek-V2/V3/R1, Kimi-K2.5, Mistral-Large-3-675B)
|
||||
- TP + EP
|
||||
- **DP + EP** — TP alone duplicates the full KV cache on every GPU; use DP Attention to partition it
|
||||
|
||||
EP = ``--enable-expert-parallel``. DP = ``--data-parallel-size N``.
|
||||
See `Data Parallel Attention (advanced)`_ for the MLA memory explanation and `Expert parallelism`_ for EP details.
|
||||
* **Tensor Parallelism (TP)**: Use when model doesn't fit on one GPU. Prefer staying within a single XGMI island (≤8 GPUs on the Instinct MI300X).
|
||||
* **Pipeline Parallelism (PP)**: Use for very large models across nodes. Set TP to GPUs per node, scale with PP across nodes.
|
||||
* **Data Parallelism (DP)**: Use when model fits on single GPU or TP group, and you need higher throughput. Combine with TP/PP for large models.
|
||||
* **Expert Parallelism (EP)**: Use for MoE models with ``--enable-expert-parallel``. More efficient than TP for MoE layers.
|
||||
|
||||
Tensor parallelism
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
@@ -319,9 +447,6 @@ Tensor parallelism splits each layer of the model weights across multiple GPUs w
|
||||
.. tip::
|
||||
For structured data parallelism deployments with load balancing, see :ref:`data-parallelism-section`.
|
||||
|
||||
.. note::
|
||||
**MLA models (DeepSeek, Kimi-K2.5, Mistral-Large-3-675B):** TP alone replicates the full KV cache on every GPU, which wastes memory at high concurrency. See `Data Parallel Attention (advanced)`_ for the DP+EP configuration that partitions the KV cache instead.
|
||||
|
||||
Pipeline parallelism
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
@@ -344,7 +469,7 @@ Pipeline parallelism splits the model's layers across multiple GPUs or nodes, wi
|
||||
--pipeline-parallel-size 2
|
||||
|
||||
.. note::
|
||||
**ROCm best practice**: On Instinct MI300X/MI325X/MI350X/MI355X, prefer staying within a single XGMI island (≤8 GPUs) using TP only. Use PP when scaling beyond eight GPUs or across nodes.
|
||||
**ROCm best practice**: On the Instinct MI300X, prefer staying within a single XGMI island (≤8 GPUs) using TP only. Use PP when scaling beyond eight GPUs or across nodes.
|
||||
|
||||
.. _data-parallelism-section:
|
||||
|
||||
@@ -413,21 +538,29 @@ For more technical details, see `vLLM Data Parallel Deployment <https://docs.vll
|
||||
Data Parallel Attention (advanced)
|
||||
""""""""""""""""""""""""""""""""""
|
||||
|
||||
For MLA models (DeepSeek V2/V3/R1, Kimi-K2.5), **DP+EP is the recommended configuration at high concurrency** (≥512 concurrent requests). Unlike traditional DP which replicates model weights, Data Parallel Attention uses inter-GPU AllToAll communication to partition KV cache across GPUs, avoiding the KV cache duplication that occurs with tensor parallelism.
|
||||
For models with Multi-head Latent Attention (MLA) architecture like DeepSeek V2, V3, and R1, vLLM supports **Data Parallel Attention**,
|
||||
which provides request-level parallelism instead of model replication. This avoids KV cache duplication across tensor parallel ranks,
|
||||
significantly reducing memory usage and enabling larger batch sizes.
|
||||
|
||||
* At **≤128 concurrent requests**, TP=8 provides 40–86% higher throughput
|
||||
* At **≥512 concurrent requests**, DP=8+EP provides 16–47% higher throughput
|
||||
* Crossover typically occurs around **256–512 concurrent requests**
|
||||
**Key benefits for MLA models:**
|
||||
|
||||
* Eliminates KV cache duplication when using tensor parallelism
|
||||
* Enables higher throughput for high-QPS serving scenarios
|
||||
* Better memory efficiency for large context windows
|
||||
|
||||
**Usage with Expert Parallelism:**
|
||||
|
||||
Data parallel attention works seamlessly with Expert Parallelism for MoE models:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# DeepSeek-R1 with DP attention and expert parallelism (high concurrency)
|
||||
# DeepSeek-R1 with DP attention and expert parallelism
|
||||
VLLM_ALL2ALL_BACKEND="allgather_reducescatter" vllm serve deepseek-ai/DeepSeek-R1 \
|
||||
--data-parallel-size 8 \
|
||||
--enable-expert-parallel \
|
||||
--disable-nccl-for-dp-synchronization
|
||||
|
||||
For more technical details, see `vLLM RFC #16037 <https://github.com/vllm-project/vllm/issues/16037>`_ and the `vLLM MoE Playbook <https://rocm.blogs.amd.com/software-tools-optimization/vllm-moe-guide/README.html>`_.
|
||||
For more technical details, see `vLLM RFC #16037 <https://github.com/vllm-project/vllm/issues/16037>`_.
|
||||
|
||||
Expert parallelism
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
@@ -435,47 +568,23 @@ Expert parallelism
|
||||
Expert parallelism (EP) distributes expert layers of Mixture-of-Experts (MoE) models across multiple GPUs,
|
||||
where tokens are routed to the GPUs holding the experts they need.
|
||||
|
||||
**Performance considerations:**
|
||||
|
||||
Expert parallelism is designed primarily for cross-node MoE deployments where high-bandwidth interconnects (like InfiniBand) between nodes make EP communication efficient. For single-node Instinct MI300X/MI355X deployments with XGMI connectivity, tensor parallelism typically provides better performance due to optimized all-to-all collectives on XGMI.
|
||||
|
||||
**When to use EP:**
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: 30 35 35
|
||||
* Multi-node MoE deployments with fast inter-node networking
|
||||
* Models with very large numbers of experts that benefit from expert distribution
|
||||
* Workloads where EP's reduced data movement outweighs communication overhead
|
||||
|
||||
* - Scenario
|
||||
- Recommended config
|
||||
- Rationale
|
||||
|
||||
* - **Low concurrency** (≤128 requests)
|
||||
- TP=8 (EP optional)
|
||||
- 40–86% higher throughput than DP at low concurrency.
|
||||
|
||||
* - **High concurrency** (≥512 requests)
|
||||
- DP=8 + EP
|
||||
- 16–47% higher throughput at scale (for example, 7,114 TPS for DeepSeek-R1 at 1024 concurrent requests).
|
||||
|
||||
* - **MLA/MQA models** (DeepSeek-V2/V3/R1, Kimi-K2.5)
|
||||
- DP + EP
|
||||
- Avoids KV cache duplication across TP ranks. Mandatory for optimal memory at high concurrency.
|
||||
|
||||
* - **Ultra-sparse MoE** (<1% activation density, for example, Llama-4-Maverick)
|
||||
- DP or TP **without** EP
|
||||
- EP adds AllToAll overhead that exceeds the benefit — EP is 7–12% *slower* for these models.
|
||||
|
||||
* - **Standard MoE** (≥3% activation density, for example, DeepSeek-R1, Qwen3-235B)
|
||||
- EP flag
|
||||
- Improves expert routing efficiency.
|
||||
**Single-node recommendation:** For Instinct MI300X/MI355X within a single node (≤8 GPUs), prefer tensor parallelism over expert parallelism for MoE models to leverage XGMI's high bandwidth and low latency.
|
||||
|
||||
**Basic usage:**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# DP + EP for MLA+MoE models (DeepSeek-R1, high concurrency)
|
||||
VLLM_ALL2ALL_BACKEND="allgather_reducescatter" vllm serve deepseek-ai/DeepSeek-R1 \
|
||||
--data-parallel-size 8 \
|
||||
--enable-expert-parallel \
|
||||
--disable-nccl-for-dp-synchronization
|
||||
|
||||
# TP + EP (low concurrency, non-MLA models)
|
||||
# Enable expert parallelism for MoE models (DeepSeek example with 8 GPUs)
|
||||
vllm serve deepseek-ai/DeepSeek-R1 \
|
||||
--tensor-parallel-size 8 \
|
||||
--enable-expert-parallel
|
||||
@@ -487,37 +596,17 @@ When EP is enabled alongside tensor parallelism:
|
||||
* Fused MoE layers use expert parallelism
|
||||
* Non-fused MoE layers use tensor parallelism
|
||||
|
||||
Multimodal model optimization (vision-language)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
**Combining with Data Parallelism:**
|
||||
|
||||
For multimodal models (Qwen3-VL, InternVL, step3), use batch-level data parallelism
|
||||
for the vision encoder instead of the default tensor parallelism:
|
||||
EP works seamlessly with Data Parallel Attention for optimal memory efficiency in MLA+MoE models (for example, DeepSeek V3):
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
vllm serve Qwen/Qwen3-VL-235B-A22B-Instruct \
|
||||
--tensor-parallel-size 8 \
|
||||
--mm-encoder-tp-mode data \
|
||||
# DP attention + EP for DeepSeek-R1
|
||||
VLLM_ALL2ALL_BACKEND="allgather_reducescatter" vllm serve deepseek-ai/DeepSeek-R1 \
|
||||
--data-parallel-size 8 \
|
||||
--enable-expert-parallel \
|
||||
--max-model-len 32768
|
||||
|
||||
``--mm-encoder-tp-mode data`` replaces per-layer all-reduce synchronization (58–126 ops
|
||||
in TP mode) with a single all-gather after encoding, yielding **10–45% throughput
|
||||
improvement** with negligible memory overhead (0.2–2.3% model size increase).
|
||||
|
||||
**When it helps most:**
|
||||
|
||||
* High-resolution images (1024×1024 px): **+16% average** throughput
|
||||
* 1–3 images per request: **+13–16%** throughput
|
||||
* Deep vision encoders (for example, InternVL 45 blocks, step3 63 blocks)
|
||||
|
||||
**When to skip it:**
|
||||
|
||||
* Very small vision encoders (<1% of total model parameters)
|
||||
* 10+ small images per request (diminishing returns)
|
||||
* Memory-constrained deployments (encoder weights are replicated per GPU)
|
||||
|
||||
For more details, see the `vLLM Multimodal DP blog post <https://rocm.blogs.amd.com/software-tools-optimization/vllm-dp-vision/README.html>`_.
|
||||
--disable-nccl-for-dp-synchronization
|
||||
|
||||
Throughput benchmarking
|
||||
=======================
|
||||
@@ -556,7 +645,7 @@ Maximizing instances per node
|
||||
To maximize **per-node throughput**, run as many vLLM instances as model memory allows,
|
||||
balancing KV-cache capacity.
|
||||
|
||||
* **HBM capacities**: MI300X = 192 GB HBM3; MI325X = 256 GB HBM3E; MI350X/MI355X = 288 GB HBM3E.
|
||||
* **HBM capacities**: MI300X = 192 GB HBM3; MI355X = 288 GB HBM3E.
|
||||
|
||||
* Up to **eight** single-GPU vLLM instances can run in parallel on an 8×GPU node (one per GPU):
|
||||
|
||||
@@ -571,7 +660,7 @@ balancing KV-cache capacity.
|
||||
Total throughput from **N** single-GPU instances usually exceeds one instance stretched across **N** GPUs (``-tp N``).
|
||||
|
||||
**Model coverage**: Llama 2 (7B/13B/70B), Llama 3 (8B/70B), Qwen2 (7B/72B), Mixtral-8x7B/8x22B, and others Llama2‑70B
|
||||
and Llama3‑70B can fit a single MI300X/MI325X/MI350X/MI355X; Llama3.1‑405B fits on a single 8×MI300X/MI325X/MI350X/MI355X node.
|
||||
and Llama3‑70B can fit a single MI300X/MI355X; Llama3.1‑405B fits on a single 8×MI300X/MI355X node.
|
||||
|
||||
Configure the gpu-memory-utilization parameter
|
||||
==================================================
|
||||
@@ -681,15 +770,11 @@ CUDA graphs reduce kernel launch overhead by capturing and replaying GPU operati
|
||||
|
||||
* - Attention backend
|
||||
- CUDA graph support
|
||||
* - ``TRITON_ATTN``
|
||||
* - vLLM/AITER Triton Unified Attention, vLLM Prefill-Decode Attention
|
||||
- Full support (prefill + decode)
|
||||
* - ``ROCM_ATTN``, ``ROCM_AITER_UNIFIED_ATTN``
|
||||
- Full support (prefill + decode)
|
||||
* - ``ROCM_AITER_FA``, ``ROCM_AITER_MLA``, ``ROCM_AITER_TRITON_MLA``
|
||||
* - AITER MHA, AITER MLA
|
||||
- Uniform batches only
|
||||
* - ``ROCM_AITER_MLA_SPARSE``
|
||||
- Uniform single-token decode only
|
||||
* - ``TRITON_MLA``
|
||||
* - vLLM Triton MLA
|
||||
- Must exclude attention from graph — ``PIECEWISE`` required
|
||||
|
||||
**Usage examples:**
|
||||
@@ -716,7 +801,7 @@ CUDA graphs reduce kernel launch overhead by capturing and replaying GPU operati
|
||||
Quantization support
|
||||
====================
|
||||
|
||||
vLLM supports FP4/FP8 (4-bit/8-bit floating point) weight and activation quantization using hardware acceleration on the Instinct MI300X, MI325X, MI350X, and MI355X.
|
||||
vLLM supports FP4/FP8 (4-bit/8-bit floating point) weight and activation quantization using hardware acceleration on the Instinct MI300X and MI355X.
|
||||
Quantization of models with FP4/FP8 allows for a **2x-4x** reduction in model memory requirements and up to a **1.6x**
|
||||
improvement in throughput with minimal impact on accuracy.
|
||||
|
||||
@@ -729,7 +814,7 @@ vLLM ROCm supports a variety of quantization demands:
|
||||
Supported quantization methods
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
vLLM on ROCm supports the following quantization methods for the AMD Instinct MI300 series and Instinct MI350 series GPUs:
|
||||
vLLM on ROCm supports the following quantization methods for the AMD Instinct MI300 series and Instinct MI355X GPUs:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
@@ -851,23 +936,40 @@ For models without pre-quantization, vLLM can quantize ``FP16``/``BF16`` models
|
||||
GPTQ
|
||||
^^^^
|
||||
|
||||
GPTQ (4-bit/8-bit weight quantization) is fully supported on ROCm via HIP-compiled kernels.
|
||||
Pre-quantized GPTQ models from Hugging Face work out of the box. For better throughput on AMD Instinct GPUs,
|
||||
consider **AWQ with Triton kernels** or **FP8 quantization** instead.
|
||||
GPTQ is a 4-bit/8-bit weight quantization method that compresses models with minimal accuracy loss. GPTQ
|
||||
is fully supported on ROCm via HIP-compiled kernels in vLLM.
|
||||
|
||||
**ROCm support status**:
|
||||
|
||||
- **Fully supported** - GPTQ kernels compile and run on ROCm via HIP
|
||||
- **Pre-quantized models work** with standard GPTQ kernels
|
||||
|
||||
**Recommendation**: For the AMD Instinct MI300X, **AWQ with Triton kernels** or **FP8 quantization** might provide better
|
||||
performance due to ROCm-specific optimizations, but GPTQ is a viable alternative.
|
||||
|
||||
**Using pre-quantized GPTQ models**:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# Using pre-quantized GPTQ model on ROCm
|
||||
vllm serve RedHatAI/Meta-Llama-3.1-70B-Instruct-quantized.w4a16 \
|
||||
--quantization gptq \
|
||||
--dtype auto \
|
||||
--tensor-parallel-size 1
|
||||
|
||||
**Important notes**:
|
||||
|
||||
- **Kernel support:** GPTQ uses standard HIP-compiled kernels on ROCm
|
||||
- **Performance:** AWQ with Triton kernels might offer better throughput on AMD GPUs due to ROCm optimizations
|
||||
- **Compatibility:** GPTQ models from Hugging Face work on ROCm with standard performance
|
||||
- **Use case:** GPTQ is suitable when pre-quantized GPTQ models are readily available
|
||||
|
||||
AWQ (Activation-aware Weight Quantization)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
AWQ (Activation-aware Weight Quantization) is a 4-bit weight quantization technique that provides excellent
|
||||
model compression with minimal accuracy loss (<1%). ROCm supports AWQ quantization on the AMD Instinct MI300 series and
|
||||
MI350 series GPUs with vLLM.
|
||||
MI355X GPUs with vLLM.
|
||||
|
||||
**Using pre-quantized AWQ models:**
|
||||
|
||||
@@ -1037,16 +1139,8 @@ Multi-node checklist and troubleshooting
|
||||
3. For GPUDirect RDMA, set ``RCCL_NET_GDR_LEVEL=2`` and verify links (``ibstat``). Requires supported NICs (for example, ConnectX‑6+).
|
||||
4. Collect RCCL logs: ``RCCL_DEBUG=INFO`` and optionally ``RCCL_DEBUG_SUBSYS=INIT,GRAPH`` for init/graph stalls.
|
||||
|
||||
Deprecated terms
|
||||
================
|
||||
|
||||
* **Prefill-Decode attention** has been renamed to **ROCM_ATTN** (ROCm attention). Use ``--attention-backend ROCM_ATTN`` to select this backend.
|
||||
|
||||
Further reading
|
||||
===============
|
||||
|
||||
* :doc:`workload`
|
||||
* :doc:`/how-to/rocm-for-ai/inference/benchmark-docker/vllm`
|
||||
* `ROCm Attention Backend deep-dive <https://vllm.ai/blog/rocm-attention-backend>`_ — architecture and benchmarks for all 7 backends
|
||||
* `vLLM MoE Playbook - A Practical Guide to TP, DP, PP and Expert Parallelism <https://rocm.blogs.amd.com/software-tools-optimization/vllm-moe-guide/README.html>`_ — DP+EP tuning for MoE models
|
||||
* `Multimodal DP optimization <https://rocm.blogs.amd.com/software-tools-optimization/vllm-dp-vision/README.html>`_ — batch-level DP for vision encoders
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X GPUs using AMD MAD and the
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X accelerators using AMD MAD and the ROCm vLLM Docker image.
|
||||
|
||||
+1
-2
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X GPUs using AMD MAD and the ROCm vLLM Docker image.
|
||||
@@ -480,4 +479,4 @@ Previous versions
|
||||
=================
|
||||
|
||||
See :doc:`vllm-history` to find documentation for previous releases
|
||||
of the ``ROCm/vllm`` Docker image.
|
||||
of the ``ROCm/vllm`` Docker image.
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X GPUs using AMD MAD and the ROCm vLLM Docker image.
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X GPUs using AMD MAD and the unified
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X GPUs using AMD MAD and the unified
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X GPUs using AMD MAD and the
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X GPUs using AMD MAD and the
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X GPUs using AMD MAD and the
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X GPUs using AMD MAD and the
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X GPUs using AMD MAD and the
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X GPUs using AMD MAD and the
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X GPUs using AMD MAD and the
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to validate LLM inference performance on MI300X GPUs using AMD MAD and the
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn to validate diffusion model video generation on MI300X, MI350X and MI355X accelerators using
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn to validate diffusion model video generation on MI300X, MI350X and MI355X accelerators using
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn to validate diffusion model video generation on MI300X, MI350X and MI355X accelerators using
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn to validate diffusion model video generation on MI300X, MI350X and MI355X accelerators using
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn to validate diffusion model video generation on MI300X, MI350X and MI355X accelerators using
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn to validate diffusion model video generation on MI300X, MI350X and MI355X accelerators using
|
||||
|
||||
@@ -1,321 +0,0 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn to validate diffusion model video generation on MI300X, MI350X and MI355X accelerators using
|
||||
prebuilt and optimized docker images.
|
||||
:keywords: xDiT, diffusion, video, video generation, image, image generation, validate, benchmark
|
||||
|
||||
************************
|
||||
xDiT diffusion inference
|
||||
************************
|
||||
|
||||
.. caution::
|
||||
|
||||
This documentation does not reflect the latest version of the xDiT diffusion
|
||||
inference performance documentation. See
|
||||
:doc:`/how-to/rocm-for-ai/inference/xdit-diffusion-inference` for the latest
|
||||
version.
|
||||
|
||||
.. _xdit-video-diffusion-263:
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.3-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
The `rocm/pytorch-xdit <{{ docker.docker_hub_url }}>`_ Docker image offers a prebuilt, optimized environment based on `xDiT <https://github.com/xdit-project/xDiT>`_ for
|
||||
benchmarking diffusion model video and image generation on gfx942 and gfx950 series (AMD Instinct™ MI300X, MI325X, MI350X, and MI355X) GPUs.
|
||||
The image runs ROCm **{{docker.ROCm}}** (preview) based on `TheRock <https://github.com/ROCm/TheRock>`_
|
||||
and includes the following components:
|
||||
|
||||
.. dropdown:: Software components - {{ docker.pull_tag.split('-')|last }}
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Software component
|
||||
- Version
|
||||
|
||||
{% for component_name, component_data in docker.components.items() %}
|
||||
* - `{{ component_name }} <{{ component_data.url }}>`_
|
||||
- {{ component_data.version }}
|
||||
{% endfor %}
|
||||
|
||||
Follow this guide to pull the required image, spin up a container, download the model, and run a benchmark.
|
||||
For preview and development releases, see `amdsiloai/pytorch-xdit <https://hub.docker.com/r/amdsiloai/pytorch-xdit>`_.
|
||||
|
||||
What's new
|
||||
==========
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.3-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
{% for item in docker.whats_new %}
|
||||
* {{ item }}
|
||||
{% endfor %}
|
||||
|
||||
.. _xdit-video-diffusion-supported-models-263:
|
||||
|
||||
Supported models
|
||||
================
|
||||
|
||||
The following models are supported for inference performance benchmarking.
|
||||
Some instructions, commands, and recommendations in this documentation might
|
||||
vary by model -- select one to get started.
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.3-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<div id="vllm-benchmark-ud-params-picker" class="container-fluid">
|
||||
<div class="row gx-0">
|
||||
<div class="col-2 me-1 px-2 model-param-head">Model</div>
|
||||
<div class="row col-10 pe-0">
|
||||
{% for model_group in docker.supported_models %}
|
||||
<div class="col-6 px-2 model-param" data-param-k="model-group" data-param-v="{{ model_group.js_tag }}" tabindex="0">{{ model_group.group }}</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row gx-0 pt-1">
|
||||
<div class="col-2 me-1 px-2 model-param-head">Variant</div>
|
||||
<div class="row col-10 pe-0">
|
||||
{% for model_group in docker.supported_models %}
|
||||
{% set models = model_group.models %}
|
||||
{% for model in models %}
|
||||
{% if models|length % 3 == 0 %}
|
||||
<div class="col-4 px-2 model-param" data-param-k="model" data-param-v="{{ model.js_tag }}" data-param-group="{{ model_group.js_tag }}" tabindex="0">{{ model.model }}</div>
|
||||
{% else %}
|
||||
<div class="col-6 px-2 model-param" data-param-k="model" data-param-v="{{ model.js_tag }}" data-param-group="{{ model_group.js_tag }}" tabindex="0">{{ model.model }}</div>
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{% for model_group in docker.supported_models %}
|
||||
{% for model in model_group.models %}
|
||||
|
||||
.. container:: model-doc {{ model.js_tag }}
|
||||
|
||||
.. note::
|
||||
|
||||
To learn more about your specific model see the `{{ model.model }} model card on Hugging Face <{{ model.url }}>`_
|
||||
or visit the `GitHub page <{{ model.github }}>`__. Note that some models require access authorization before use via an
|
||||
external license agreement through a third party.
|
||||
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
System validation
|
||||
=================
|
||||
|
||||
Before running AI workloads, it's important to validate that your AMD hardware is configured
|
||||
correctly and performing optimally.
|
||||
|
||||
If you have already validated your system settings, including aspects like NUMA auto-balancing, you
|
||||
can skip this step. Otherwise, complete the procedures in the :ref:`System validation and
|
||||
optimization <rocm-for-ai-system-optimization>` guide to properly configure your system settings
|
||||
before starting.
|
||||
|
||||
To test for optimal performance, consult the recommended :ref:`System health benchmarks
|
||||
<rocm-for-ai-system-health-bench>`. This suite of tests will help you verify and fine-tune your
|
||||
system's configuration.
|
||||
|
||||
Pull the Docker image
|
||||
=====================
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.3-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
For this tutorial, it's recommended to use the latest ``{{ docker.pull_tag }}`` Docker image.
|
||||
Pull the image using the following command:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker pull {{ docker.pull_tag }}
|
||||
|
||||
Validate and benchmark
|
||||
======================
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.3-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
Once the image has been downloaded you can follow these steps to
|
||||
run benchmarks and generate outputs.
|
||||
|
||||
{% for model_group in docker.supported_models %}
|
||||
{% for model in model_group.models %}
|
||||
|
||||
.. container:: model-doc {{model.js_tag}}
|
||||
|
||||
The following commands are written for {{ model.model }}.
|
||||
See :ref:`xdit-video-diffusion-supported-models-263` to switch to another available model.
|
||||
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
Choose your setup method
|
||||
------------------------
|
||||
|
||||
You can either use an existing Hugging Face cache or download the model fresh inside the container.
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.3-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
{% for model_group in docker.supported_models %}
|
||||
{% for model in model_group.models %}
|
||||
.. container:: model-doc {{model.js_tag}}
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Option 1: Use existing Hugging Face cache
|
||||
|
||||
If you already have models downloaded on your host system, you can mount your existing cache.
|
||||
|
||||
1. Set your Hugging Face cache location.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
export HF_HOME=/your/hf_cache/location
|
||||
|
||||
2. Download the model (if not already cached).
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
huggingface-cli download {{ model.model_repo }} {% if model.revision %} --revision {{ model.revision }} {% endif %}
|
||||
|
||||
3. Launch the container with mounted cache.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker run \
|
||||
-it --rm \
|
||||
--cap-add=SYS_PTRACE \
|
||||
--security-opt seccomp=unconfined \
|
||||
--user root \
|
||||
--device=/dev/kfd \
|
||||
--device=/dev/dri \
|
||||
--group-add video \
|
||||
--ipc=host \
|
||||
--network host \
|
||||
--privileged \
|
||||
--shm-size 128G \
|
||||
--name pytorch-xdit \
|
||||
-e HSA_NO_SCRATCH_RECLAIM=1 \
|
||||
-e OMP_NUM_THREADS=16 \
|
||||
-e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
|
||||
-e HF_HOME=/app/huggingface_models \
|
||||
-v $HF_HOME:/app/huggingface_models \
|
||||
{{ docker.pull_tag }}
|
||||
|
||||
.. tab-item:: Option 2: Download inside container
|
||||
|
||||
If you prefer to keep the container self-contained or don't have an existing cache.
|
||||
|
||||
1. Launch the container
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker run \
|
||||
-it --rm \
|
||||
--cap-add=SYS_PTRACE \
|
||||
--security-opt seccomp=unconfined \
|
||||
--user root \
|
||||
--device=/dev/kfd \
|
||||
--device=/dev/dri \
|
||||
--group-add video \
|
||||
--ipc=host \
|
||||
--network host \
|
||||
--privileged \
|
||||
--shm-size 128G \
|
||||
--name pytorch-xdit \
|
||||
-e HSA_NO_SCRATCH_RECLAIM=1 \
|
||||
-e OMP_NUM_THREADS=16 \
|
||||
-e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
|
||||
{{ docker.pull_tag }}
|
||||
|
||||
2. Inside the container, set the Hugging Face cache location and download the model.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
export HF_HOME=/app/huggingface_models
|
||||
huggingface-cli download {{ model.model_repo }} {% if model.revision %} --revision {{ model.revision }} {% endif %}
|
||||
|
||||
.. warning::
|
||||
|
||||
Models will be downloaded to the container's filesystem and will be lost when the container is removed unless you persist the data with a volume.
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
Run inference
|
||||
=============
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.3-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
{% for model_group in docker.supported_models %}
|
||||
{% for model in model_group.models %}
|
||||
|
||||
.. container:: model-doc {{ model.js_tag }}
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MAD-integrated benchmarking
|
||||
|
||||
1. Clone the ROCm Model Automation and Dashboarding (`<https://github.com/ROCm/MAD>`__) repository to a local
|
||||
directory and install the required packages on the host machine.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git clone https://github.com/ROCm/MAD
|
||||
cd MAD
|
||||
pip install -r requirements.txt
|
||||
|
||||
2. On the host machine, use this command to run the performance benchmark test on
|
||||
the `{{model.model}} <{{ model.url }}>`_ model using one node.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
|
||||
madengine run \
|
||||
--tags {{model.mad_tag}} \
|
||||
--keep-model-dir \
|
||||
--live-output
|
||||
|
||||
MAD launches a Docker container with the name
|
||||
``container_ci-{{model.mad_tag}}``. The throughput and serving reports of the
|
||||
model are collected in the following paths: ``{{ model.mad_tag }}_throughput.csv``
|
||||
and ``{{ model.mad_tag }}_serving.csv``.
|
||||
|
||||
.. tab-item:: Standalone benchmarking
|
||||
|
||||
To run the benchmarks for {{ model.model }}, use the following command:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
{{ model.benchmark_command
|
||||
| map('replace', '{model_repo}', model.model_repo)
|
||||
| map('trim')
|
||||
| join('\n ') }}
|
||||
|
||||
The generated content and timing information will be stored under the results directory.
|
||||
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
Previous versions
|
||||
=================
|
||||
|
||||
See
|
||||
:doc:`/how-to/rocm-for-ai/inference/benchmark-docker/previous-versions/xdit-history`
|
||||
to find documentation for previous releases of xDiT diffusion inference
|
||||
performance testing.
|
||||
@@ -1,321 +0,0 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: Learn to validate diffusion model video generation on MI300X, MI350X and MI355X accelerators using
|
||||
prebuilt and optimized docker images.
|
||||
:keywords: xDiT, diffusion, video, video generation, image, image generation, validate, benchmark
|
||||
|
||||
************************
|
||||
xDiT diffusion inference
|
||||
************************
|
||||
|
||||
.. caution::
|
||||
|
||||
This documentation does not reflect the latest version of the xDiT diffusion
|
||||
inference performance documentation. See
|
||||
:doc:`/how-to/rocm-for-ai/inference/xdit-diffusion-inference` for the latest
|
||||
version.
|
||||
|
||||
.. _xdit-video-diffusion-264:
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
The `rocm/pytorch-xdit <{{ docker.docker_hub_url }}>`_ Docker image offers a prebuilt, optimized environment based on `xDiT <https://github.com/xdit-project/xDiT>`_ for
|
||||
benchmarking diffusion model video and image generation on gfx942 and gfx950 series (AMD Instinct™ MI300X, MI325X, MI350X, and MI355X) GPUs.
|
||||
The image runs `ROCm {{docker.ROCm}} (preview) <https://rocm.docs.amd.com/en/7.12.0-preview/about/release-notes.html>`__ based on `TheRock <https://github.com/ROCm/TheRock>`_
|
||||
and includes the following components:
|
||||
|
||||
.. dropdown:: Software components - {{ docker.pull_tag.split('-')|last }}
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Software component
|
||||
- Version
|
||||
|
||||
{% for component_name, component_data in docker.components.items() %}
|
||||
* - `{{ component_name }} <{{ component_data.url }}>`_
|
||||
- {{ component_data.version }}
|
||||
{% endfor %}
|
||||
|
||||
Follow this guide to pull the required image, spin up a container, download the model, and run a benchmark.
|
||||
For preview and development releases, see `amdsiloai/pytorch-xdit <https://hub.docker.com/r/amdsiloai/pytorch-xdit>`_.
|
||||
|
||||
What's new
|
||||
==========
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
{% for item in docker.whats_new %}
|
||||
* {{ item }}
|
||||
{% endfor %}
|
||||
|
||||
.. _xdit-video-diffusion-supported-models-264:
|
||||
|
||||
Supported models
|
||||
================
|
||||
|
||||
The following models are supported for inference performance benchmarking.
|
||||
Some instructions, commands, and recommendations in this documentation might
|
||||
vary by model -- select one to get started.
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<div id="vllm-benchmark-ud-params-picker" class="container-fluid">
|
||||
<div class="row gx-0">
|
||||
<div class="col-2 me-1 px-2 model-param-head">Model</div>
|
||||
<div class="row col-10 pe-0">
|
||||
{% for model_group in docker.supported_models %}
|
||||
<div class="col-6 px-2 model-param" data-param-k="model-group" data-param-v="{{ model_group.js_tag }}" tabindex="0">{{ model_group.group }}</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row gx-0 pt-1">
|
||||
<div class="col-2 me-1 px-2 model-param-head">Variant</div>
|
||||
<div class="row col-10 pe-0">
|
||||
{% for model_group in docker.supported_models %}
|
||||
{% set models = model_group.models %}
|
||||
{% for model in models %}
|
||||
{% if models|length % 3 == 0 %}
|
||||
<div class="col-4 px-2 model-param" data-param-k="model" data-param-v="{{ model.js_tag }}" data-param-group="{{ model_group.js_tag }}" tabindex="0">{{ model.model }}</div>
|
||||
{% else %}
|
||||
<div class="col-6 px-2 model-param" data-param-k="model" data-param-v="{{ model.js_tag }}" data-param-group="{{ model_group.js_tag }}" tabindex="0">{{ model.model }}</div>
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{% for model_group in docker.supported_models %}
|
||||
{% for model in model_group.models %}
|
||||
|
||||
.. container:: model-doc {{ model.js_tag }}
|
||||
|
||||
.. note::
|
||||
|
||||
To learn more about your specific model see the `{{ model.model }} model card on Hugging Face <{{ model.url }}>`_
|
||||
or visit the `GitHub page <{{ model.github }}>`__. Note that some models require access authorization before use via an
|
||||
external license agreement through a third party.
|
||||
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
System validation
|
||||
=================
|
||||
|
||||
Before running AI workloads, it's important to validate that your AMD hardware is configured
|
||||
correctly and performing optimally.
|
||||
|
||||
If you have already validated your system settings, including aspects like NUMA auto-balancing, you
|
||||
can skip this step. Otherwise, complete the procedures in the :ref:`System validation and
|
||||
optimization <rocm-for-ai-system-optimization>` guide to properly configure your system settings
|
||||
before starting.
|
||||
|
||||
To test for optimal performance, consult the recommended :ref:`System health benchmarks
|
||||
<rocm-for-ai-system-health-bench>`. This suite of tests will help you verify and fine-tune your
|
||||
system's configuration.
|
||||
|
||||
Pull the Docker image
|
||||
=====================
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
For this tutorial, it's recommended to use the latest ``{{ docker.pull_tag }}`` Docker image.
|
||||
Pull the image using the following command:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker pull {{ docker.pull_tag }}
|
||||
|
||||
Validate and benchmark
|
||||
======================
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
Once the image has been downloaded you can follow these steps to
|
||||
run benchmarks and generate outputs.
|
||||
|
||||
{% for model_group in docker.supported_models %}
|
||||
{% for model in model_group.models %}
|
||||
|
||||
.. container:: model-doc {{model.js_tag}}
|
||||
|
||||
The following commands are written for {{ model.model }}.
|
||||
See :ref:`xdit-video-diffusion-supported-models-264` to switch to another available model.
|
||||
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
Choose your setup method
|
||||
------------------------
|
||||
|
||||
You can either use an existing Hugging Face cache or download the model fresh inside the container.
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
{% for model_group in docker.supported_models %}
|
||||
{% for model in model_group.models %}
|
||||
.. container:: model-doc {{model.js_tag}}
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Option 1: Use existing Hugging Face cache
|
||||
|
||||
If you already have models downloaded on your host system, you can mount your existing cache.
|
||||
|
||||
1. Set your Hugging Face cache location.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
export HF_HOME=/your/hf_cache/location
|
||||
|
||||
2. Download the model (if not already cached).
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
hf download {{ model.model_repo }} {% if model.revision %} --revision {{ model.revision }} {% endif %}
|
||||
|
||||
3. Launch the container with mounted cache.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker run \
|
||||
-it --rm \
|
||||
--cap-add=SYS_PTRACE \
|
||||
--security-opt seccomp=unconfined \
|
||||
--user root \
|
||||
--device=/dev/kfd \
|
||||
--device=/dev/dri \
|
||||
--group-add video \
|
||||
--ipc=host \
|
||||
--network host \
|
||||
--privileged \
|
||||
--shm-size 128G \
|
||||
--name pytorch-xdit \
|
||||
-e HSA_NO_SCRATCH_RECLAIM=1 \
|
||||
-e OMP_NUM_THREADS=16 \
|
||||
-e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
|
||||
-e HF_HOME=/app/huggingface_models \
|
||||
-v $HF_HOME:/app/huggingface_models \
|
||||
{{ docker.pull_tag }}
|
||||
|
||||
.. tab-item:: Option 2: Download inside container
|
||||
|
||||
If you prefer to keep the container self-contained or don't have an existing cache.
|
||||
|
||||
1. Launch the container
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker run \
|
||||
-it --rm \
|
||||
--cap-add=SYS_PTRACE \
|
||||
--security-opt seccomp=unconfined \
|
||||
--user root \
|
||||
--device=/dev/kfd \
|
||||
--device=/dev/dri \
|
||||
--group-add video \
|
||||
--ipc=host \
|
||||
--network host \
|
||||
--privileged \
|
||||
--shm-size 128G \
|
||||
--name pytorch-xdit \
|
||||
-e HSA_NO_SCRATCH_RECLAIM=1 \
|
||||
-e OMP_NUM_THREADS=16 \
|
||||
-e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
|
||||
{{ docker.pull_tag }}
|
||||
|
||||
2. Inside the container, set the Hugging Face cache location and download the model.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
export HF_HOME=/app/huggingface_models
|
||||
hf download {{ model.model_repo }} {% if model.revision %} --revision {{ model.revision }} {% endif %}
|
||||
|
||||
.. warning::
|
||||
|
||||
Models will be downloaded to the container's filesystem and will be lost when the container is removed unless you persist the data with a volume.
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
Run inference
|
||||
=============
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/previous-versions/xdit_26.4-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
|
||||
{% for model_group in docker.supported_models %}
|
||||
{% for model in model_group.models %}
|
||||
|
||||
.. container:: model-doc {{ model.js_tag }}
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MAD-integrated benchmarking
|
||||
|
||||
1. Clone the ROCm Model Automation and Dashboarding (`<https://github.com/ROCm/MAD>`__) repository to a local
|
||||
directory and install the required packages on the host machine.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git clone https://github.com/ROCm/MAD
|
||||
cd MAD
|
||||
pip install -r requirements.txt
|
||||
|
||||
2. On the host machine, use this command to run the performance benchmark test on
|
||||
the `{{model.model}} <{{ model.url }}>`_ model using one node.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
|
||||
madengine run \
|
||||
--tags {{model.mad_tag}} \
|
||||
--keep-model-dir \
|
||||
--live-output
|
||||
|
||||
MAD launches a Docker container with the name
|
||||
``container_ci-{{model.mad_tag}}``. The throughput and serving reports of the
|
||||
model are collected in the following paths: ``{{ model.mad_tag }}_throughput.csv``
|
||||
and ``{{ model.mad_tag }}_serving.csv``.
|
||||
|
||||
.. tab-item:: Standalone benchmarking
|
||||
|
||||
To run the benchmarks for {{ model.model }}, use the following command:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
{{ model.benchmark_command
|
||||
| map('replace', '{model_repo}', model.model_repo)
|
||||
| map('trim')
|
||||
| join('\n ') }}
|
||||
|
||||
The generated content and timing information will be stored under the results directory.
|
||||
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
Previous versions
|
||||
=================
|
||||
|
||||
See
|
||||
:doc:`/how-to/rocm-for-ai/inference/benchmark-docker/previous-versions/xdit-history`
|
||||
to find documentation for previous releases of xDiT diffusion inference
|
||||
performance testing.
|
||||
+2
-19
@@ -15,28 +15,11 @@ benchmarking, see the version-specific documentation.
|
||||
- Components
|
||||
- Resources
|
||||
|
||||
* - ``rocm/pytorch-xdit:v26.5`` (latest)
|
||||
* - ``rocm/pytorch-xdit:v26.3`` (latest)
|
||||
-
|
||||
* ROCm 7.13.0
|
||||
* TheRock cbff3d1
|
||||
-
|
||||
* :doc:`Documentation </how-to/rocm-for-ai/inference/xdit-diffusion-inference>`
|
||||
* `Docker Hub <https://hub.docker.com/layers/rocm/pytorch-xdit/v26.5/images/sha256-b8ad9fd4b41bc116ac2aff07c1066bf369cf7fc110b1a323f6302191985a51fd>`__
|
||||
|
||||
* - ``rocm/pytorch-xdit:v26.4``
|
||||
-
|
||||
* `ROCm 7.12.0 preview <https://rocm.docs.amd.com/en/7.12.0-preview/about/release-notes.html>`__
|
||||
* TheRock 9b611c6
|
||||
-
|
||||
* :doc:`Documentation <xdit-26.4>`
|
||||
* `Docker Hub <https://hub.docker.com/layers/rocm/pytorch-xdit/v26.4/images/sha256-b4296a638eb8dc7ebcafc808e180b78a3c44177580c21986082ec9539496067c>`__
|
||||
|
||||
* - ``rocm/pytorch-xdit:v26.3``
|
||||
-
|
||||
* `ROCm 7.12.0 preview <https://rocm.docs.amd.com/en/7.12.0-preview/about/release-notes.html>`__
|
||||
* TheRock e40a6da
|
||||
-
|
||||
* :doc:`Documentation <xdit-26.3>`
|
||||
* :doc:`Documentation </how-to/rocm-for-ai/inference/xdit-diffusion-inference>`
|
||||
* `Docker Hub <https://hub.docker.com/layers/rocm/pytorch-xdit/v26.3/images/sha256-ac78a03d2911bf1b49c001d3be2e8bd745c1bc455cb49ae972825a7986880902>`__
|
||||
|
||||
* - ``rocm/pytorch-xdit:v26.2``
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
.. meta::
|
||||
:description: SGLang multi-node disaggregated distributed inference using Mooncake
|
||||
:keywords: model, sglang, mooncake, disagg, disaggregated, distributed, multi-node, docker
|
||||
|
||||
******************************************
|
||||
SGLang distributed inference with Mooncake
|
||||
******************************************
|
||||
|
||||
As LLM inference increasingly demands handling massive models and dynamic workloads, efficient
|
||||
distributed inference becomes essential. Traditional co-located architectures face bottlenecks due
|
||||
to tightly coupled memory and compute resources, which limits scalability and flexibility.
|
||||
Disaggregated inference refers to the process of splitting the inference of LLMs into distinct
|
||||
phases. This architecture, facilitated by libraries like Mooncake, uses high-bandwidth
|
||||
RDMA to transfer the Key-Value (KV) cache between prefill and decode nodes.
|
||||
This allows for independent resource scaling and optimization, resulting in
|
||||
improved efficiency and throughput.
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/sglang-distributed-benchmark-models.yaml
|
||||
|
||||
{% set docker = data.dockers[0] %}
|
||||
|
||||
`SGLang <https://docs.sglang.ai>`__ is a high-performance inference and
|
||||
serving engine for large language models (LLMs) and vision models. The
|
||||
ROCm-enabled `SGLang base Docker image <{{ docker.docker_hub_url }}>`__
|
||||
bundles SGLang with PyTorch, which is optimized for AMD Instinct MI300X Series
|
||||
GPUs. It includes the following software components:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Software component
|
||||
- Version
|
||||
|
||||
{% for component_name, component_version in docker.components.items() %}
|
||||
* - {{ component_name }}
|
||||
- {{ component_version }}
|
||||
{% endfor %}
|
||||
|
||||
The following guides on setting up and running SGLang and Mooncake for disaggregated
|
||||
distributed inference on a Slurm cluster using AMD Instinct MI300X Series GPUs backed by
|
||||
Mellanox CX-7 NICs.
|
||||
|
||||
Prerequisites
|
||||
=============
|
||||
|
||||
Before starting, ensure you have:
|
||||
|
||||
* A Slurm cluster with at least three nodes: one for the proxy, one for prefill (``xP``), and one for decode (``yD``).
|
||||
|
||||
``Nodes -> xP + yD + 1``
|
||||
|
||||
* A Dockerized environment with SGLang, Mooncake, etcd, and NIC drivers built in. See :ref:`sglang-disagg-inf-build-docker-image` for instructions.
|
||||
|
||||
* A shared filesystem for storing models, scripts, and logs (cluster-specific).
|
||||
|
||||
Supported models
|
||||
================
|
||||
|
||||
The following models are supported for SGLang disaggregated prefill/decode
|
||||
inference. Some instructions, commands, and recommendations in this
|
||||
documentation might vary by selected model.
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/sglang-distributed-benchmark-models.yaml
|
||||
|
||||
{% set model_groups = data.model_groups %}
|
||||
.. raw:: html
|
||||
|
||||
<div id="vllm-benchmark-ud-params-picker" class="container-fluid">
|
||||
<div class="row gx-0">
|
||||
<div class="col-2 me-1 px-2 model-param-head">Model type</div>
|
||||
<div class="row col-10 pe-0">
|
||||
{% for model_group in model_groups %}
|
||||
<div class="col-6 px-2 model-param" data-param-k="model-group" data-param-v="{{ model_group.tag }}" tabindex="0">{{ model_group.group }}</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row gx-0 pt-1">
|
||||
<div class="col-2 me-1 px-2 model-param-head">Model</div>
|
||||
<div class="row col-10 pe-0">
|
||||
{% for model_group in model_groups %}
|
||||
{% set models = model_group.models %}
|
||||
{% for model in models %}
|
||||
{% if models|length % 3 == 0 %}
|
||||
<div class="col-4 px-2 model-param" data-param-k="model" data-param-v="{{ model.model_repo | lower }}" data-param-group="{{ model_group.tag }}" tabindex="0">{{ model.model }}</div>
|
||||
{% else %}
|
||||
<div class="col-6 px-2 model-param" data-param-k="model" data-param-v="{{ model.model_repo | lower }}" data-param-group="{{ model_group.tag }}" tabindex="0">{{ model.model }}</div>
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{% for model_group in model_groups %}
|
||||
{% for model in model_group.models %}
|
||||
|
||||
.. container:: model-doc {{ model.model_repo }}
|
||||
|
||||
.. note::
|
||||
|
||||
See the `{{ model.model }} model card on Hugging Face <{{ model.url }}>`__ to learn more about this model.
|
||||
Some models require access authorization prior to use through an external license agreement with a third party.
|
||||
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
.. _sglang-disagg-inf-build-docker-image:
|
||||
|
||||
Build the Docker image
|
||||
----------------------
|
||||
|
||||
Get the Dockerfile located in
|
||||
`<https://github.com/ROCm/MAD/blob/develop/docker/sglang_disagg_inference.ubuntu.amd.Dockerfile>`__.
|
||||
It uses `lmsysorg/sglang:v0.5.2rc1-rocm700-mi30x
|
||||
<https://hub.docker.com/layers/lmsysorg/sglang/v0.4.9.post1-rocm630/images/sha256-2f6b1748e4bcc70717875a7da76c87795fd8aa46a9646e08d38aa7232fc78538>`__
|
||||
as the base Docker image and installs the necessary components for Mooncake, etcd, and Mellanox network
|
||||
drivers.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git clone https://github.com/ROCm/MAD.git
|
||||
cd MAD/docker
|
||||
docker build \
|
||||
-t sglang_disagg_pd_image \
|
||||
-f sglang_disagg_inference.ubuntu.amd.Dockerfile .
|
||||
|
||||
Benchmarking
|
||||
============
|
||||
|
||||
The `<https://github.com/ROCm/MAD/tree/develop/scripts/sglang_disagg>`__
|
||||
repository contains scripts to launch SGLang inference with prefill/decode
|
||||
disaggregation via Mooncake for supported models.
|
||||
|
||||
* `scripts/sglang_dissag/run_xPyD_models.slurm <https://github.com/ROCm/MAD/blob/develop/scripts/sglang_disagg/run_xPyD_models.slurm>`__
|
||||
-- the main Slurm batch script to launch Docker containers on all nodes using ``sbatch`` or ``salloc``.
|
||||
|
||||
* `scripts/sglang_dissag/sglang_disagg_server.sh <https://github.com/ROCm/MAD/blob/develop/scripts/sglang_disagg/sglang_disagg_server.sh>`__
|
||||
-- the entrypoint script that runs inside each container to start the correct service -- proxy, prefill, or decode.
|
||||
|
||||
* `scripts/sglang_dissag/benchmark_xPyD.sh <https://github.com/ROCm/MAD/blob/develop/scripts/sglang_disagg/benchmark_xPyD.sh>`__
|
||||
-- the benchmark script to run the GSM8K accuracy benchmark and the SGLang benchmarking tool for performance measurement.
|
||||
|
||||
* `scripts/sglang_dissag/benchmark_parser.py <https://github.com/ROCm/MAD/blob/develop/scripts/sglang_disagg/benchmark_parser.py>`__
|
||||
-- the log parser script to be run on the concurrency benchmark log file to generate tabulated data.
|
||||
|
||||
Launch the service
|
||||
------------------
|
||||
|
||||
The service is deployed using a Slurm batch script that orchestrates the containers across the
|
||||
allocated nodes.
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/sglang-distributed-benchmark-models.yaml
|
||||
|
||||
{% set model_groups = data.model_groups %}
|
||||
{% for model_group in model_groups %}
|
||||
{% for model in model_group.models %}
|
||||
|
||||
.. container:: model-doc {{ model.model_repo }}
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
# Clone the MAD repo if you haven't already and
|
||||
# navigate to the scripts directory
|
||||
git clone https://github.com/ROCm/MAD.git
|
||||
cd MAD/scripts/sglang_disagg/
|
||||
|
||||
# Slurm sbatch run command
|
||||
export DOCKER_IMAGE_NAME=sglang_disagg_pd_image
|
||||
export xP=<num_prefill_nodes>
|
||||
export yD=<num_decode_nodes>
|
||||
export MODEL_NAME={{ model.model_repo }}
|
||||
# num_nodes = xP + yD + 1
|
||||
sbatch -N <num_nodes> -n <num_nodes> --nodelist=<Nodes> run_xPyD_models.slurm
|
||||
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
Post-run logs and testing
|
||||
-------------------------
|
||||
|
||||
Logs are stored in your shared filesystem in the directory specified by the ``LOG_PATH`` variable in the Slurm script.
|
||||
A new directory named after the Slurm job ID is created for each run.
|
||||
|
||||
Inside that directory, you can access various logs:
|
||||
|
||||
* ``pd_sglang_bench_serving.sh_NODE<...>.log`` -- the main log for each server node.
|
||||
|
||||
* ``etcd_NODE<...>.log`` -- logs for etcd services.
|
||||
|
||||
* ``prefill_NODE<...>.log`` -- logs for the prefill services.
|
||||
|
||||
* ``decode_NODE<...>.log`` -- logs for the decode services.
|
||||
|
||||
Use the benchmark parser script for concurrency logs to tabulate different data.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
python3 benchmark_parser.py <log_path/benchmark_XXX_CONCURRENCY.log>
|
||||
|
||||
To verify the service is responsive, you can try sending a ``curl`` request to test the launched
|
||||
server from the Docker container on the proxy node. For example:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
curl -X POST http://127.0.0.1:30000/generate \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{ "text": "Let me tell you a story ", "sampling_params": { "temperature": 0.3 } }'
|
||||
|
||||
Known issues
|
||||
============
|
||||
|
||||
When running larger models, such as DeepSeek-V3 and Llama-3.1-405B-Instruct-FP8-KV, at
|
||||
higher concurrency levels (512+), the following error might occur:
|
||||
|
||||
.. code-block:: shell-session
|
||||
|
||||
<TransferEncodingError: 400, message:
|
||||
Not enough data to satisfy transfer length header.
|
||||
|
||||
The above exception was the direct cause of the following exception:
|
||||
|
||||
Traceback (most recent call last):
|
||||
...
|
||||
|
||||
This leads to dropping requests and lower throughput.
|
||||
|
||||
Further reading
|
||||
===============
|
||||
|
||||
- To learn about Mooncake, see `Welcome to Mooncake <https://kvcache-ai.github.io/Mooncake/>`__.
|
||||
|
||||
- To learn more about the options for latency and throughput benchmark scripts,
|
||||
see `<https://github.com/sgl-project/sglang/tree/main/benchmark/blog_v0_2>`__.
|
||||
|
||||
- See the base upstream Docker image on `Docker Hub <https://hub.docker.com/layers/lmsysorg/sglang/v0.5.2rc1-rocm700-mi30x/images/sha256-10c4ee502ddba44dd8c13325e6e03868bfe7f43d23d0a44780a8ee8b393f4729>`__.
|
||||
|
||||
- To learn more about system settings and management practices to configure your system for
|
||||
MI300X Series GPUs, see `AMD Instinct MI300X system optimization <https://instinct.docs.amd.com/projects/amdgpu-docs/en/latest/system-optimization/mi300x.html>`__.
|
||||
|
||||
- For application performance optimization strategies for HPC and AI workloads,
|
||||
including inference with vLLM, see :doc:`/how-to/rocm-for-ai/inference-optimization/workload`.
|
||||
|
||||
- To learn how to run community models from Hugging Face on AMD GPUs, see
|
||||
:doc:`Running models from Hugging Face </how-to/rocm-for-ai/inference/hugging-face-models>`.
|
||||
|
||||
- To learn how to fine-tune LLMs and optimize inference, see
|
||||
:doc:`Fine-tuning LLMs and inference optimization </how-to/rocm-for-ai/fine-tuning/fine-tuning-and-inference>`.
|
||||
|
||||
- For a list of other ready-made Docker images for AI with ROCm, see
|
||||
`AMD Infinity Hub <https://www.amd.com/en/developer/resources/infinity-hub.html#f-amd_hub_category=AI%20%26%20ML%20Models>`_.
|
||||
|
||||
Previous versions
|
||||
=================
|
||||
|
||||
See :doc:`previous-versions/sglang-history` to find documentation for previous releases
|
||||
of SGLang inference performance testing.
|
||||
@@ -12,15 +12,14 @@ scripts.
|
||||
|
||||
The following configuration is required to implement this setup:
|
||||
|
||||
* **Nodes**: A minimum of three GPU nodes (virtual machines or physical
|
||||
machines) for wide expert parallelism (EP) evaluation.
|
||||
* **GPUs**: 8x AMD Instinct MI355X GPU cards per node.
|
||||
* **Networking**: 8x RDMA-capable NICs per node (AMD Pensando Pollara 400,
|
||||
NVIDIA Mellanox ConnectX-7, or Broadcom Thor 2), providing a dedicated 1:1
|
||||
mapping between GPUs and network interfaces for optimal inter-node
|
||||
communication.
|
||||
* **Orchestration**: A Slurm cluster with at least three nodes — one for
|
||||
prefill service and two for decode services (EP16).
|
||||
* **Nodes:** A minimum of three GPU nodes (Virtual machines or Physical
|
||||
machines) for wide expert parallelism (EP) evaluation.
|
||||
* **GPUs** 8x AMD Instinct MI355X GPU cards per node.
|
||||
* **Networking:** 8x AMD Pensando™ Pollara 400 AI NICs per node, providing
|
||||
a dedicated 1:1 mapping between GPUs and network interfaces for optimal
|
||||
inter-node communication.
|
||||
* **Orchestration:** A Slurm cluster with at least three nodes -- one for
|
||||
prefill service and two for decode services (EP16)
|
||||
|
||||
## System configuration
|
||||
|
||||
@@ -30,6 +29,8 @@ baselines and firmware versions, configuring the AMD Pensando Pollara 400 AI
|
||||
NICs for high-bandwidth networking, and applying thermal and Quality of Service
|
||||
(QoS) tunings to ensure a stable, lossless RDMA fabric.
|
||||
|
||||
(sglang-mori-verify-baseline)=
|
||||
|
||||
### Verify baseline software
|
||||
|
||||
The following table outlines the validated software stack. Use the provided
|
||||
@@ -80,19 +81,18 @@ Redfish API:
|
||||
|
||||
Before proceeding with software deployment, verify that all cluster nodes
|
||||
comply with the [MI355X Basic Health
|
||||
Checks](https://instinct.docs.amd.com/projects/system-acceptance/en/latest/gpus/mi355x.html#basic-health-checks).
|
||||
Checks](https://instinct.docs.amd.com/projects/system-acceptance/en/latest/gpus/mi355x.html#basic-health-checks)
|
||||
Key requirements include specific kernel boot arguments, minimum system memory
|
||||
thresholds, PCIe Gen5 link stability, and so on.
|
||||
|
||||
### NIC installation
|
||||
|
||||
#### AMD Pensando Pollara 400 AI NIC installation
|
||||
### Install AMD Pensando Pollara 400 AI NIC drivers
|
||||
|
||||
For detailed instructions on upgrading the firmware and installing drivers for
|
||||
the AMD Pensando Pollara 400 AI NIC, refer to the [AMD Instinct System
|
||||
Acceptance Guide](https://instinct.docs.amd.com/projects/system-acceptance/en/latest/network/nic-installation.html#amd-pensando-pollara-400-ai-nic).
|
||||
Acceptance
|
||||
Guide](https://instinct.docs.amd.com/projects/system-acceptance/en/latest/network/nic-installation.html#amd-pensando-pollara-400-ai-nic).
|
||||
After installation, verify the active firmware version on all NICs to ensure it
|
||||
matches the software baseline. See [Verify baseline software](#verify-baseline-software).
|
||||
matches the software baseline. See [Verify baseline software](#verify-best-known-configuration-bkc).
|
||||
|
||||
To display the current firmware version for all AI NICs, use the following command.
|
||||
|
||||
@@ -100,26 +100,6 @@ To display the current firmware version for all AI NICs, use the following comma
|
||||
sudo nicctl show version firmware
|
||||
```
|
||||
|
||||
#### CX7 driver and firmware installation
|
||||
|
||||
1. Download and install the `DOCA 2.9.3` driver following the instructions in
|
||||
[NVIDIA DOCA 2.9.3 Downloads](https://developer.nvidia.com/doca-downloads).
|
||||
2. Download the appropriate firmware for your hardware PSID from the
|
||||
[ConnectX-7 Firmware Download
|
||||
Center](https://network.nvidia.com/support/firmware/connectx7/) and flash
|
||||
the device.
|
||||
3. To verify driver and firmware versions, use the following command. Replace
|
||||
`IB Device` with your specific backend interface.
|
||||
|
||||
```bash
|
||||
ethtool -i <IB Device>
|
||||
```
|
||||
|
||||
#### Broadcom BNXT driver and firmware installation
|
||||
|
||||
Refer to your Broadcom representative for driver and firmware installation
|
||||
instructions specific to your NIC model.
|
||||
|
||||
### Configure thermal management (fan speed)
|
||||
|
||||
For systems equipped with 400G optics, standard fan profiles are often
|
||||
@@ -160,10 +140,8 @@ the addresses `192.168.1.36`, `192.168.2.36`, and so on. Another node would
|
||||
have `192.168.1.37`, `192.168.2.37`, and so on. Ensure MTU is set to `9000`.
|
||||
|
||||
```{note}
|
||||
Ensure you identify the correct interface names for your system using `ip link`
|
||||
before applying this configuration. The `macaddress:` values in the example
|
||||
below are illustrative only and must be replaced with the actual MAC addresses
|
||||
of your NICs, which you can find using `ip link show <interface>`.
|
||||
Ensure you identify the correct interface names for your system using ip link
|
||||
before applying this configuration.
|
||||
```
|
||||
|
||||
For example, your `/etc/netplan/70-backend.yaml` should look like the
|
||||
@@ -176,7 +154,7 @@ network:
|
||||
addresses:
|
||||
- 192.168.8.38/31
|
||||
match:
|
||||
macaddress: 04:90:81:00:00:08
|
||||
macaddress: 04:90:81:2a:34:08
|
||||
mtu: 9000
|
||||
routes:
|
||||
- table: 108
|
||||
@@ -190,7 +168,7 @@ network:
|
||||
addresses:
|
||||
- 192.168.7.38/31
|
||||
match:
|
||||
macaddress: 04:90:81:00:00:07
|
||||
macaddress: 04:90:81:2b:82:40
|
||||
mtu: 9000
|
||||
routes:
|
||||
- table: 107
|
||||
@@ -204,7 +182,7 @@ network:
|
||||
addresses:
|
||||
- 192.168.6.38/31
|
||||
match:
|
||||
macaddress: 04:90:81:00:00:06
|
||||
macaddress: 04:90:81:30:c9:30
|
||||
mtu: 9000
|
||||
routes:
|
||||
- table: 106
|
||||
@@ -218,7 +196,7 @@ network:
|
||||
addresses:
|
||||
- 192.168.5.38/31
|
||||
match:
|
||||
macaddress: 04:90:81:00:00:05
|
||||
macaddress: 04:90:81:2a:23:40
|
||||
mtu: 9000
|
||||
routes:
|
||||
- table: 105
|
||||
@@ -232,7 +210,7 @@ network:
|
||||
addresses:
|
||||
- 192.168.4.38/31
|
||||
match:
|
||||
macaddress: 04:90:81:00:00:04
|
||||
macaddress: 04:90:81:2d:69:60
|
||||
mtu: 9000
|
||||
routes:
|
||||
- table: 104
|
||||
@@ -246,7 +224,7 @@ network:
|
||||
addresses:
|
||||
- 192.168.3.38/31
|
||||
match:
|
||||
macaddress: 04:90:81:00:00:03
|
||||
macaddress: 04:90:81:2a:2c:40
|
||||
mtu: 9000
|
||||
routes:
|
||||
- table: 103
|
||||
@@ -260,7 +238,7 @@ network:
|
||||
addresses:
|
||||
- 192.168.2.38/31
|
||||
match:
|
||||
macaddress: 04:90:81:00:00:02
|
||||
macaddress: 04:90:81:30:d5:30
|
||||
mtu: 9000
|
||||
routes:
|
||||
- table: 102
|
||||
@@ -274,7 +252,7 @@ network:
|
||||
addresses:
|
||||
- 192.168.1.38/31
|
||||
match:
|
||||
macaddress: 04:90:81:00:00:01
|
||||
macaddress: 04:90:81:30:e4:00
|
||||
mtu: 9000
|
||||
routes:
|
||||
- table: 101
|
||||
@@ -298,18 +276,17 @@ To verify your configuration, use the following command.
|
||||
sudo apt install -y net-tools && ip -br a
|
||||
```
|
||||
|
||||
### Configure quality of service (QoS) and congestion control (DCQCN)
|
||||
### Configure Quality of Service (QoS) and Congestion Control (DCQCN)
|
||||
|
||||
To ensure lossless communication and optimal performance for RDMA traffic, the
|
||||
network must be configured with specific QoS and Data Center Quantized
|
||||
Congestion Notification (DCQCN) settings.
|
||||
|
||||
The following configuration:
|
||||
|
||||
* Enables RX and TX pause frames on the ports.
|
||||
* Maps DSCP 24 (Data) to Q3 and DSCP 46 (CNP) to Q6, with all other DSCP to Q0.
|
||||
* Enables PFC for Q3.
|
||||
* Scheduling: 99% to Q3, 1% to Q0, and strict priority for Q6.
|
||||
The following configuration achieves:
|
||||
• It enables RX and TX Pause frames on the ports
|
||||
• Maps DSCP 24 (Data) to Q3 and DSCP 46 (CNP) to Q6, all other DSCP to Q0
|
||||
• Enables PFC for Q3
|
||||
• Scheduling : 99% to Q3, 1% to Q0 and strict priority for Q6
|
||||
|
||||
#### Configure DCQCN
|
||||
|
||||
@@ -317,7 +294,7 @@ Create and run a `/nfsdata/enable_dcqcn.sh` script to initialize congestion
|
||||
control parameters.
|
||||
|
||||
``` bash
|
||||
#!/bin/bash
|
||||
# !/bin/bash
|
||||
|
||||
TOKEN_BUCKET_SIZE=800000
|
||||
AI_RATE=160
|
||||
@@ -384,7 +361,7 @@ sudo nicctl update qos pfc --priority $data_prio --no-drop enable
|
||||
sudo nicctl update qos scheduling --priority $data_prio,$default_prio,$cts_prio --dwrr 99,1,0 --rate-limit 0,0,10
|
||||
```
|
||||
|
||||
#### Verify your configuration
|
||||
#### Verification your configuration
|
||||
|
||||
Verify the configuration using `nicctl`.
|
||||
|
||||
@@ -397,9 +374,9 @@ Verify the configuration using `nicctl`.
|
||||
Expected QoS output:
|
||||
|
||||
``` bash
|
||||
NIC : 00000000-0000-0000-0000-000000000001 (0000:f6:00.0)
|
||||
NIC : 42424650-4c32-3531-3230-303443000000 (0000:f6:00.0)
|
||||
|
||||
Port : 00000000-0001-4242-4242-000000000000
|
||||
Port : 04908130-a7a0-4242-4242-000011010000
|
||||
|
||||
Classification type : DSCP
|
||||
|
||||
@@ -421,10 +398,10 @@ Verify the configuration using `nicctl`.
|
||||
Expected DCQCN and scheduling output:
|
||||
|
||||
``` bash
|
||||
NIC : 00000000-0000-0000-0000-000000000001 (0000:f6:00.0)
|
||||
NIC : 42424650-4c32-3531-3230-303443000000 (0000:f6:00.0)
|
||||
------------------------------------------------------------------------------------------
|
||||
|
||||
Lif id : 00000000-0100-0000-4242-000000000000
|
||||
Lif id : 43000070-0100-0000-4242-04908130a7a0
|
||||
ROCE device : ionic_7
|
||||
DCQCN profile id : 1
|
||||
Status : Enabled
|
||||
@@ -520,10 +497,9 @@ the cluster interconnects.
|
||||
### Verify network connectivity
|
||||
|
||||
Verify that all network interfaces are reachable across the cluster nodes.
|
||||
Assuming `benic1p1` through `benic8p1` are the dedicated RoCE backend
|
||||
interfaces, use the following ping loop to verify reachability across the
|
||||
backend subnets (for instance, a target node at host IP suffix
|
||||
`.38`).
|
||||
Assuming `eth0` is the management interface, and `benic1p1` through `benic8p1` are the
|
||||
dedicated RoCE backend interfaces, use the following loop to test reachability
|
||||
to a remote node (for instance, a target node with host IP suffix `.38`).
|
||||
|
||||
```bash
|
||||
# Test connectivity for RoCE subnets 192.168.x.38 (node B) through 192.168.x.37 (node A)
|
||||
@@ -546,9 +522,9 @@ The output should look something like this:
|
||||
```bash
|
||||
-------------------------------------------------------------------------------------
|
||||
|
||||
NIC : 00000000-0000-0000-0000-000000000002 (0000:f6:00.0)
|
||||
NIC : 42424650-4c32-3531-3530-314343000000 (0000:f6:00.0)
|
||||
|
||||
Port : 00000000-0002-4242-4242-000000000000 (eth1/1)
|
||||
Port : 04908132-5d88-4242-4242-000011010000 (eth1/1)
|
||||
Spec:
|
||||
Ifindex : 0x11010000
|
||||
Type : ETH
|
||||
@@ -572,7 +548,7 @@ Port : 00000000-0002-4242-4242-000000000000 (eth1/1)
|
||||
Auto negotiation : disabled
|
||||
MAC ID : 0
|
||||
MAC channel : 0
|
||||
MAC address : 04:90:81:00:00:00
|
||||
MAC address : 04:90:81:32:5d:88
|
||||
Transceiver type : QSFP_CMIS
|
||||
Transceiver state : SPROM-READ
|
||||
Transceiver PID : QSFP-400G-DR4
|
||||
@@ -593,7 +569,7 @@ ibv_devinfo -v | grep GID
|
||||
The output should look something like this:
|
||||
|
||||
```bash
|
||||
GID[ 0]: fe80::6a00:00ff:fe00:0001, RoCE v2
|
||||
GID[ 0]: fe80::690:81ff:fe30:a7a0, RoCE v2
|
||||
GID[ 1]: ::ffff:192.168.7.36, RoCE v2
|
||||
```
|
||||
|
||||
@@ -625,17 +601,17 @@ appropriate IP.
|
||||
|
||||
```bash
|
||||
# On Server Node
|
||||
./ib_write_bw --use_rocm=0 -d ionic_0 --report_gbits -a
|
||||
./ib_write_bw --use_rocm=0 -d mlx5_0 --report_gbits -a
|
||||
|
||||
# On Client Node
|
||||
./ib_write_bw --use_rocm=0 -d ionic_0 --report_gbits -a <SERVER_IP>
|
||||
./ib_write_bw --use_rocm=0 -d mlx5_0 --report_gbits -a <SERVER_IP>
|
||||
```
|
||||
|
||||
## SGLang serving and MoRI unit tests
|
||||
|
||||
### Install Docker Engine
|
||||
|
||||
Install the Docker engine to manage the containerized SGLang and MoRI serving
|
||||
Install the Docker engine to manage the containerized vLLM and MoRI serving
|
||||
environments.
|
||||
|
||||
```bash
|
||||
@@ -653,7 +629,7 @@ IMAGE_NAME=rocm/sgl-dev:sglang-0.5.6.post1-rocm700-mi35x-mori-0113
|
||||
|
||||
docker run -it \
|
||||
--rm \
|
||||
--device /dev/dri --device /dev/kfd -v /dev/infiniband:/dev/infiniband \
|
||||
--device /dev/dri --device /dev/kfd --device=/dev/infiniBand \
|
||||
--network host --ipc host \
|
||||
--group-add video \
|
||||
--cap-add SYS_PTRACE \
|
||||
@@ -666,7 +642,7 @@ docker run -it \
|
||||
|
||||
### Run MoRI inter-node unit tests
|
||||
|
||||
Before starting the SGLang service, run the MoRI unit test to verify that the
|
||||
Before starting the vLLM service, run the MoRI unit test to verify that the
|
||||
inter-node communication backend is correctly configured.
|
||||
|
||||
MoRI unit test uses 2 nodes as a minimal validation before running the full
|
||||
@@ -674,34 +650,18 @@ MoRI unit test uses 2 nodes as a minimal validation before running the full
|
||||
|
||||
The key configuration variables are:
|
||||
|
||||
* `GLOO_SOCKET_IFNAME`: The network interface used for backend initialization (for example, `benic1p1`).
|
||||
* `MORI_SOCKET_IFNAME`: The network interface used by MoRI's own bootstrap. Set it to the same backend interface as `GLOO_SOCKET_IFNAME`.
|
||||
* `MORI_GPU_ARCHS`: The target GPU architecture. Set to `gfx950` for MI355X; otherwise the test may auto-select the wrong arch (for example, `gfx942`).
|
||||
* `GLOO_SOCKET_IFNAME`: The network interface used for backend initialization such as `eth2`.
|
||||
* `<MASTER_IP>`: The IP address of the primary node's backend interface.
|
||||
|
||||
Performance reference data can be found in the [ROCm/MoRI
|
||||
repository](https://github.com/ROCm/mori?tab=readme-ov-file#mori-ep).
|
||||
|
||||
```{note}
|
||||
The `rocm/sgl-dev:sglang-0.5.6.post1-rocm700-mi35x-mori-0113` image ships MoRI
|
||||
under `/sgl-workspace/mori`. If `/sgl-workspace/mori` or the example test
|
||||
scripts (for example, `test_dispatch_combine_internode.py`) are missing, clone
|
||||
the repository:
|
||||
|
||||
`git clone https://github.com/ROCm/mori.git /sgl-workspace/mori`
|
||||
You can find reference performance data in the [ROCm/MoRI
|
||||
repository](https://github.com/ROCm/mori?tab=readme-ov-file#mori-ep).
|
||||
```
|
||||
|
||||
```bash
|
||||
# Set up environment inside the container
|
||||
cd /sgl-workspace/mori
|
||||
|
||||
# prettytable is required to render the benchmark result table:
|
||||
pip install prettytable
|
||||
|
||||
export PYTHONPATH=/sgl-workspace/mori:$PYTHONPATH
|
||||
export MORI_GPU_ARCHS=gfx950 # MI355X arch; avoids auto-selecting gfx942
|
||||
export GLOO_SOCKET_IFNAME=<BACKEND_INTERFACE> # e.g. benic1p1
|
||||
export MORI_SOCKET_IFNAME=<BACKEND_INTERFACE> # MoRI bootstrap interface; same as GLOO_SOCKET_IFNAME
|
||||
export PYTHONPATH=/app/mori:$PYTHONPATH
|
||||
export GLOO_SOCKET_IFNAME=<BACKEND_INTERFACE>
|
||||
|
||||
# Node 0 (Primary)
|
||||
torchrun --nnodes=2 --node_rank=0 --nproc_per_node=1 \
|
||||
@@ -719,7 +679,9 @@ torchrun --nnodes=2 --node_rank=1 --nproc_per_node=1 \
|
||||
## End-to-end 1P2D performance testing
|
||||
|
||||
This section guides you through running distributed inference benchmarks using
|
||||
the SGLang disagg recipe.
|
||||
the SGLang disagg recipe. For detailed implementation details, refer to the
|
||||
[SGLang Disaggregation
|
||||
Recipe](https://github.com/billishyahao/sglang_disagg/blob/9n_cluster/README.md).
|
||||
|
||||
### Download the model and setup your run environment
|
||||
|
||||
@@ -730,7 +692,7 @@ This performance test supports the following models:
|
||||
* [DeepSeek-R1-0528](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528)
|
||||
|
||||
To set up your environment and download the models using the Hugging Face CLI,
|
||||
use the following commands. Modify the `hf download` command
|
||||
use the following commands. Modify the `huggingface-cli download` command
|
||||
to download the desired model.
|
||||
|
||||
```bash
|
||||
@@ -742,19 +704,20 @@ pip install huggingface_hub
|
||||
|
||||
# Download the model to the shared NFS mount point
|
||||
# Replace 'deepseek-ai/DeepSeek-R1-0528' with your desired model
|
||||
hf download --token <your_hf_token> \
|
||||
huggingface-cli download --token <your_hf_token> \
|
||||
deepseek-ai/DeepSeek-R1-0528 \
|
||||
--local-dir /mount/point/models/DeepSeek-R1
|
||||
```
|
||||
|
||||
### Clone the SGLang disaggregation recipe
|
||||
|
||||
Clone the [ROCm/distributed_inference](https://github.com/ROCm/distributed_inference)
|
||||
repository to the shared file system:
|
||||
Clone the SGLang disaggregation repository to the shared file system and switch
|
||||
to the appropriate branch:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/ROCm/distributed_inference.git
|
||||
cd distributed_inference
|
||||
git clone https://github.com/billishyahao/sglang_disagg.git
|
||||
git checkout 9n_cluster
|
||||
cd sglang_disagg
|
||||
```
|
||||
|
||||
```{note}
|
||||
@@ -786,54 +749,28 @@ Identify and configure the available InfiniBand devices.
|
||||
ionic_7
|
||||
```
|
||||
|
||||
2. Update environment variables. Edit `set_env_vars.sh` and set the
|
||||
following variables:
|
||||
2. Update environment variables. Edit `set_env_vars.sh` and add the
|
||||
comma-separated list of your system's IB devices. For example:
|
||||
|
||||
```bash
|
||||
export IBDEVICES=ionic_0,ionic_1,ionic_2,ionic_3,ionic_4,ionic_5,ionic_6,ionic_7
|
||||
|
||||
# Must be >= chunked_prefill_size / dp_size.
|
||||
# Default recipe: 262144 / 8 = 32768. set_env_vars.sh's value (16384) is
|
||||
# too small and causes an AssertionError on prefill startup.
|
||||
export SGLANG_MORI_NUM_MAX_DISPATCH_TOKENS_PER_RANK=32768
|
||||
|
||||
# Must be large enough for the dispatch buffer. The default 4 GB heap
|
||||
# causes an out-of-memory error at first inference with a 32768-token budget.
|
||||
export MORI_SHMEM_HEAP_SIZE=16G
|
||||
```
|
||||
|
||||
### Configure the script and submit the job
|
||||
|
||||
```{important}
|
||||
Two `set_env_vars.sh` variables must be set before submitting the job, or
|
||||
the prefill service crashes at startup:
|
||||
|
||||
* `SGLANG_MORI_NUM_MAX_DISPATCH_TOKENS_PER_RANK` — SGLang asserts this value is
|
||||
≥ `chunked_prefill_size / dp_size`. For the default DeepSeek-R1 recipe
|
||||
(262144 / 8 = 32768), `set_env_vars.sh`'s shipped value of 16384 triggers an
|
||||
`AssertionError` on the prefill node only (decode is exempt). Raising this to
|
||||
32768 fixes the crash.
|
||||
* `MORI_SHMEM_HEAP_SIZE` — Raising the dispatch budget to 32768 tokens exceeds
|
||||
MoRI's default 4 GB static heap and causes an out-of-memory error at first
|
||||
inference. Set this to `16G` (MoRI's own inter-node test default).
|
||||
|
||||
These values are set in the preceding [Configure InfiniBand
|
||||
devices](#configure-infiniBand-devices) step.
|
||||
```
|
||||
|
||||
1. To set the required configuration parameters, update the following
|
||||
environment variables in `run_submit_disagg.sh` to match your cluster setup:
|
||||
|
||||
```bash
|
||||
# SLURM Job Configuration
|
||||
export SLURM_ACCOUNT="<your_slurm_account>" # The account name for SLURM job accounting and resource allocation
|
||||
export SLURM_ACCOUNT="amd" # The account name for SLURM job accounting and resource allocation
|
||||
export SLURM_PARTITION="compute" # The specific cluster partition (queue) to submit the job to
|
||||
export TIME_LIMIT="24:00:00" # Maximum wall time for the job (Hours:Minutes:Seconds)
|
||||
|
||||
# Model Configuration
|
||||
export MODEL_PATH="/nfsdata" # Base directory where the model weights are stored
|
||||
export MODEL_NAME="DeepSeek-R1" # Specific model directory name (joined with MODEL_PATH)
|
||||
export CONTAINER_IMAGE="lmsysorg/sglang-rocm:v0.5.12.post1-rocm720-mi35x-20260529" # Docker image to use for the environment
|
||||
export CONTAINER_IMAGE="rocm/sgl-dev:sglang-0.5.6.post1-rocm700-mi35x-mori-1224" # Docker image to use for the environment
|
||||
|
||||
# Cluster Topology (Disaggregation Setup)
|
||||
export PREFILL_NODES=1 # Number of prefill nodes
|
||||
@@ -890,7 +827,7 @@ devices](#configure-infiniBand-devices) step.
|
||||
```{note}
|
||||
The following benchmark utility output is provided for reference only and
|
||||
should not be used to compare performance. See the
|
||||
[InferenceX](https://inferencex.semianalysis.com/) website for validated
|
||||
[InferenceMAX](https://inferencemax.semianalysis.com/) website for validated
|
||||
performance results.
|
||||
```
|
||||
|
||||
@@ -926,7 +863,7 @@ devices](#configure-infiniBand-devices) step.
|
||||
|
||||
The following section outlines common issues and their solutions.
|
||||
|
||||
### Bandwidth test failures
|
||||
### Bandwidth test fails with error
|
||||
|
||||
1. Use ROCm-optimized `rdma-perftest`, not the generic `perftest`
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -30,8 +30,8 @@ training, fine-tuning, and inference. It leverages popular machine learning fram
|
||||
|
||||
- :doc:`SGLang distributed inference with MoRI <benchmark-docker/sglang-mori-distributed>`
|
||||
|
||||
- :doc:`SGLang distributed inference with Mooncake <benchmark-docker/sglang-distributed>`
|
||||
|
||||
- :doc:`xDiT diffusion inference <xdit-diffusion-inference>`
|
||||
|
||||
- :doc:`Deploying your model <deploy-your-model>`
|
||||
|
||||
- :doc:`xDiT diffusion inference <xdit-diffusion-inference>`
|
||||
|
||||
@@ -15,7 +15,7 @@ xDiT diffusion inference
|
||||
|
||||
The `rocm/pytorch-xdit <{{ docker.docker_hub_url }}>`_ Docker image offers a prebuilt, optimized environment based on `xDiT <https://github.com/xdit-project/xDiT>`_ for
|
||||
benchmarking diffusion model video and image generation on gfx942 and gfx950 series (AMD Instinct™ MI300X, MI325X, MI350X, and MI355X) GPUs.
|
||||
The image runs `ROCm {{docker.ROCm}}` based on `TheRock <https://github.com/ROCm/TheRock>`_
|
||||
The image runs ROCm **{{docker.ROCm}}** (preview) based on `TheRock <https://github.com/ROCm/TheRock>`_
|
||||
and includes the following components:
|
||||
|
||||
.. dropdown:: Software components - {{ docker.pull_tag.split('-')|last }}
|
||||
@@ -36,7 +36,6 @@ For preview and development releases, see `amdsiloai/pytorch-xdit <https://hub.d
|
||||
|
||||
What's new
|
||||
==========
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/inference/xdit-inference-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
@@ -180,7 +179,7 @@ You can either use an existing Hugging Face cache or download the model fresh in
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
hf download {{ model.model_repo }} {% if model.revision %} --revision {{ model.revision }} {% endif %}
|
||||
huggingface-cli download {{ model.model_repo }} {% if model.revision %} --revision {{ model.revision }} {% endif %}
|
||||
|
||||
3. Launch the container with mounted cache.
|
||||
|
||||
@@ -237,7 +236,7 @@ You can either use an existing Hugging Face cache or download the model fresh in
|
||||
.. code-block:: shell
|
||||
|
||||
export HF_HOME=/app/huggingface_models
|
||||
hf download {{ model.model_repo }} {% if model.revision %} --revision {{ model.revision }} {% endif %}
|
||||
huggingface-cli download {{ model.model_repo }} {% if model.revision %} --revision {{ model.revision }} {% endif %}
|
||||
|
||||
.. warning::
|
||||
|
||||
|
||||
@@ -52,7 +52,7 @@ MaxText with on ROCm provides the following key features to train large language
|
||||
|
||||
- NANOO FP8 (for MI300X series GPUs) and FP8 (for MI355X and MI350X) quantization support
|
||||
|
||||
.. _amd-maxtext-model-support-v26.3:
|
||||
.. _amd-maxtext-model-support-v26.2:
|
||||
|
||||
Supported models
|
||||
================
|
||||
@@ -72,7 +72,7 @@ started.
|
||||
<div class="col-2 me-1 px-2 model-param-head">Model</div>
|
||||
<div class="row col-10 pe-0">
|
||||
{% for model_group in model_groups %}
|
||||
<div class="col-3 px-2 model-param" data-param-k="model-group" data-param-v="{{ model_group.tag }}" tabindex="0">{{ model_group.group }}</div>
|
||||
<div class="col-4 px-2 model-param" data-param-k="model-group" data-param-v="{{ model_group.tag }}" tabindex="0">{{ model_group.group }}</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
@@ -134,7 +134,7 @@ Use the following command to pull the Docker image from Docker Hub.
|
||||
|
||||
docker pull {{ docker.pull_tag }}
|
||||
|
||||
.. _amd-maxtext-multi-node-setup-v26.3:
|
||||
.. _amd-maxtext-multi-node-setup-v26.2:
|
||||
|
||||
Multi-node configuration
|
||||
------------------------
|
||||
@@ -142,7 +142,7 @@ Multi-node configuration
|
||||
See :doc:`/how-to/rocm-for-ai/system-setup/multi-node-setup` to configure your
|
||||
environment for multi-node training.
|
||||
|
||||
.. _amd-maxtext-get-started-v26.3:
|
||||
.. _amd-maxtext-get-started-v26.2:
|
||||
|
||||
Benchmarking
|
||||
============
|
||||
@@ -169,7 +169,7 @@ benchmark results:
|
||||
.. container:: model-doc {{ model.mad_tag }}
|
||||
|
||||
The following run commands are tailored to {{ model.model }}.
|
||||
See :ref:`amd-maxtext-model-support-v26.3` to switch to another available model.
|
||||
See :ref:`amd-maxtext-model-support-v26.2` to switch to another available model.
|
||||
|
||||
.. rubric:: Download the Docker image and required packages
|
||||
|
||||
@@ -214,7 +214,7 @@ benchmark results:
|
||||
|
||||
git clone https://github.com/AMD-AIG-AIMA/Primus.git
|
||||
cd Primus
|
||||
git checkout main
|
||||
git checkout dev/fuyuajin/maxtext-backend-test
|
||||
git submodule update --init third_party/maxtext/
|
||||
|
||||
.. rubric:: Run the training job with primus-cli
|
||||
@@ -265,7 +265,7 @@ benchmark results:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
./primus-cli container --image rocm/jax-training:maxtext-v26.3 \
|
||||
./primus-cli container --image rocm/jax-training:maxtext-v26.2 \
|
||||
-- train pretrain \
|
||||
--config examples/maxtext/configs/MI300X/{{ model.primus_config_name }}
|
||||
|
||||
@@ -301,7 +301,7 @@ benchmark results:
|
||||
.. tab-item:: MAD-integrated benchmarking
|
||||
|
||||
The following run command is tailored to {{ model.model }}.
|
||||
See :ref:`amd-maxtext-model-support-v26.3` to switch to another available model.
|
||||
See :ref:`amd-maxtext-model-support-v26.2` to switch to another available model.
|
||||
|
||||
1. Clone the ROCm Model Automation and Dashboarding (`<https://github.com/ROCm/MAD>`__) repository to a local
|
||||
directory and install the required packages on the host machine.
|
||||
@@ -332,7 +332,7 @@ benchmark results:
|
||||
.. tab-item:: Standalone benchmarking
|
||||
|
||||
The following commands are optimized for {{ model.model }}. See
|
||||
:ref:`amd-maxtext-model-support-v26.3` to switch to another
|
||||
:ref:`amd-maxtext-model-support-v26.2` to switch to another
|
||||
available model. Some instructions and resources might not be
|
||||
available for all models and configurations.
|
||||
|
||||
@@ -452,7 +452,7 @@ benchmark results:
|
||||
|
||||
[docker_image] (optional)
|
||||
The Docker image to use. If not specified, it defaults to
|
||||
``rocm/jax-training:maxtext-v26.3``.
|
||||
``rocm/jax-training:maxtext-v26.2``.
|
||||
|
||||
For example, to run a multi-node training benchmark on {{ model.model }}:
|
||||
|
||||
@@ -477,192 +477,18 @@ benchmark results:
|
||||
{% else %}
|
||||
.. rubric:: Multi-node training
|
||||
|
||||
For multi-node training examples, choose a model from :ref:`amd-maxtext-model-support-v26.3`
|
||||
For multi-node training examples, choose a model from :ref:`amd-maxtext-model-support-v26.2`
|
||||
with an available `multi-node training script <https://github.com/ROCm/MAD/tree/develop/scripts/jax-maxtext/env_scripts>`__.
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
Profiling with JAX XPlane Profiler
|
||||
===================================
|
||||
|
||||
MaxText has built-in XPlane profiling support via JAX's profiler. Traces
|
||||
capture GPU kernel timelines, RCCL collectives, HLO graphs, and more. The
|
||||
output can be viewed in TensorBoard's Trace Viewer or analyzed with
|
||||
TraceLens.
|
||||
|
||||
Key MaxText profiler flags
|
||||
--------------------------
|
||||
|
||||
The following MaxText config keys control profiling:
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
profiler=xplane # Use xplane format (produces .xplane.pb files)
|
||||
skip_first_n_steps_for_profiler=2 # Skip compilation/warmup steps
|
||||
profiler_steps=5 # Number of steps to profile
|
||||
upload_all_profiler_results=True # Save all GPU profiles (not just GPU0)
|
||||
|
||||
``steps`` should be greater than ``skip_first_n_steps_for_profiler`` +
|
||||
``profiler_steps`` (for example, ``steps=12`` with ``skip=2`` and
|
||||
``profile=5`` gives 5 warmup + 5 profiled + 2 cooldown).
|
||||
``skip_first_n_steps_for_profiler=2`` skips step 0 (compilation) and step
|
||||
1 (warmup). ``profiler_steps=5`` is typically sufficient; more steps
|
||||
produce larger ``.xplane.pb`` files.
|
||||
|
||||
Profiling with MAD or madengine
|
||||
-------------------------------
|
||||
|
||||
The model YAML configs under ``scripts/jax-maxtext/env_scripts/`` include
|
||||
a ``profiler`` key (set to ``""`` by default). To enable profiling when
|
||||
running through MAD or madengine, edit the YAML config for your model and
|
||||
set the profiler fields:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
profiler: "xplane"
|
||||
skip_first_n_steps_for_profiler: 2
|
||||
profiler_steps: 5
|
||||
upload_all_profiler_results: True
|
||||
steps: 12
|
||||
|
||||
Then run the benchmark as usual:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
|
||||
madengine run \
|
||||
--tags jax_maxtext_train_llama-3.1-8b \
|
||||
--keep-model-dir \
|
||||
--live-output \
|
||||
--timeout 28800
|
||||
|
||||
Use ``--keep-model-dir`` so the container's output directory is preserved
|
||||
after the run. Profile output is written under the ``base_output_directory``
|
||||
specified in the YAML.
|
||||
|
||||
Example: Profile a model standalone in Docker
|
||||
----------------------------------------------
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
IMAGE="$1" # Docker image, e.g. rocm/jax-training:maxtext-v26.3
|
||||
TAG="$2" # Short tag for output folder, e.g. v26.3_llama2_7b
|
||||
PROFILE_DIR="/path/to/profiles/${TAG}"
|
||||
|
||||
mkdir -p "${PROFILE_DIR}"
|
||||
|
||||
docker run --rm --privileged --network=host \
|
||||
--device=/dev/dri --device=/dev/kfd --ipc=host \
|
||||
-v "${PROFILE_DIR}:/mnt/profile" \
|
||||
"${IMAGE}" bash -c '
|
||||
export XLA_PYTHON_CLIENT_MEM_FRACTION=.97
|
||||
export LD_LIBRARY_PATH=/usr/local/lib/:/opt/rocm/lib:$LD_LIBRARY_PATH
|
||||
export XLA_FLAGS="--xla_gpu_enable_latency_hiding_scheduler=True --xla_gpu_enable_command_buffer= <your other XLA flags>"
|
||||
export GPU_MAX_HW_QUEUES=2
|
||||
|
||||
cd /workspace/maxtext
|
||||
|
||||
python3 -m MaxText.train src/MaxText/configs/base.yml \
|
||||
run_name=profile \
|
||||
base_output_directory=/mnt/profile \
|
||||
hardware=gpu \
|
||||
steps=12 \
|
||||
model_name=<your-model> \
|
||||
dataset_type=synthetic \
|
||||
enable_checkpointing=False \
|
||||
enable_goodput_recording=False \
|
||||
monitor_goodput=False \
|
||||
<your model-specific flags> \
|
||||
profiler=xplane \
|
||||
skip_first_n_steps_for_profiler=2 \
|
||||
profiler_steps=5 \
|
||||
upload_all_profiler_results=True
|
||||
' 2>&1 | tee "${PROFILE_DIR}/run.log"
|
||||
|
||||
echo "Profile files:"
|
||||
find "${PROFILE_DIR}" -name "*.xplane.pb" -o -name "*.trace.json.gz" 2>/dev/null
|
||||
|
||||
Output structure
|
||||
----------------
|
||||
|
||||
MaxText writes profiles in TensorBoard format:
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
<base_output_directory>/
|
||||
└── profile/
|
||||
└── tensorboard/
|
||||
└── plugins/
|
||||
└── profile/
|
||||
└── <YYYY_MM_DD_HH_MM_SS>/
|
||||
├── <hostname>.xplane.pb # Raw XPlane proto (GPU timelines)
|
||||
├── <hostname>.trace.json.gz # Trace viewer data
|
||||
└── *.hlo_proto.pb # HLO graphs for each compiled module
|
||||
|
||||
Viewing traces in TensorBoard
|
||||
-----------------------------
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
pip install tensorboard tensorboard-plugin-profile
|
||||
|
||||
# Point --logdir at the directory containing the tensorboard/ folder
|
||||
tensorboard --logdir /path/to/profiles/<TAG>/profile --port 6006
|
||||
|
||||
Navigate to **Profile > Trace Viewer** in the TensorBoard UI. Zoom into a
|
||||
single training step (skip the first profiled step as it may have residual
|
||||
warmup) and look at individual GPU streams to see compute/RCCL overlap.
|
||||
|
||||
To keep profile files small, use ``profiler_steps=5`` to keep
|
||||
``.xplane.pb`` files under approximately 100 MB. Too many steps can produce
|
||||
files over 500 MB that TensorBoard struggles to load. Use
|
||||
``enable_checkpointing=False`` to avoid checkpoint I/O noise in the trace,
|
||||
and ``dataset_type=synthetic`` to eliminate data loading variability.
|
||||
|
||||
Profiling with rocprofv3
|
||||
========================
|
||||
|
||||
If you need to collect a trace without the JAX profiler, use ``rocprofv3``:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
rocprofv3 --hip-trace --kernel-trace --memory-copy-trace --rccl-trace \
|
||||
--output-format pftrace -d ./v3_traces -- <command>
|
||||
|
||||
Replace ``<command>`` with the command you want to profile, such as
|
||||
``./jax-maxtext_benchmark_report.sh -m Llama-2-7B``. Use ``-d
|
||||
<TRACE_DIRECTORY>`` to specify where the ``.json`` traces are saved. The
|
||||
resulting traces can be opened in `Perfetto <https://ui.perfetto.dev/>`__.
|
||||
|
||||
Known issues
|
||||
============
|
||||
|
||||
- You might see NaNs in the losses while using real data (not synthetic
|
||||
data) when setting ``packing=True`` and ``NVTE_CK_IS_V3_ATOMIC_FP32=0``.
|
||||
Set ``NVTE_CK_IS_V3_ATOMIC_FP32=1`` for production training when using
|
||||
real data and input sequence packing (``packing=True``).
|
||||
|
||||
- There is a known slight performance regression for DeepSeek-V2-lite
|
||||
(16B) in v26.3. This is being tracked and will be addressed in a future
|
||||
release.
|
||||
|
||||
- **JAX 0.9.1 Early Access known issues:**
|
||||
|
||||
- There is a known performance regression for MoE models
|
||||
(DeepSeek-V2-lite and Mixtral-8x7B).
|
||||
|
||||
- The trace viewer in profiling may be missing some information in the
|
||||
flame graph.
|
||||
|
||||
- Shardy is a new config in JAX 0.6.0. You might get related errors if
|
||||
it's not configured correctly. To disable it, set ``shardy=False``
|
||||
during the training run. See the `Shardy migration guide
|
||||
<https://docs.jax.dev/en/latest/shardy_jax_migration.html>`__ to
|
||||
enable it.
|
||||
- You might see NaNs in the losses when setting ``packing=True``. As
|
||||
a workaround, turn off input sequence packing (``packing=False``).
|
||||
This will be fixed in a future release.
|
||||
|
||||
Further reading
|
||||
===============
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using JAX MaxText for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using JAX MaxText for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using JAX MaxText for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using JAX MaxText for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using JAX MaxText for ROCm.
|
||||
|
||||
+1
-2
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using JAX MaxText for ROCm.
|
||||
@@ -377,5 +376,5 @@ Further reading
|
||||
Previous versions
|
||||
=================
|
||||
|
||||
See :doc:`jax-maxtext-history` to find documentation for previous releases
|
||||
See :doc:`previous-versions/jax-maxtext-history` to find documentation for previous releases
|
||||
of the ``ROCm/jax-training`` Docker image.
|
||||
|
||||
-516
@@ -1,516 +0,0 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using JAX MaxText for ROCm.
|
||||
:keywords: ROCm, AI, LLM, train, jax, torch, Llama, flux, tutorial, docker
|
||||
|
||||
********************************************
|
||||
Training a model with Primus and JAX MaxText
|
||||
********************************************
|
||||
|
||||
.. caution::
|
||||
|
||||
This documentation does not reflect the latest version of ROCm JAX MaxText
|
||||
training performance documentation. See :doc:`../jax-maxtext` for the latest version.
|
||||
|
||||
The JAX MaxText for ROCm training Docker image provides a prebuilt environment
|
||||
for training on AMD Instinct MI355X, MI350X, MI325X, and MI300X GPUs, with
|
||||
essential components such as JAX, XLA, ROCm libraries, and MaxText utilities.
|
||||
|
||||
The image also integrates with `Primus <https://github.com/AMD-AGI/Primus>`__,
|
||||
a high-level training framework that supports multiple backends. You can use
|
||||
the unified ``primus-cli`` to run training jobs using the JAX MaxText backend.
|
||||
|
||||
It includes the following software components:
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/jax-maxtext-v26.2-benchmark-models.yaml
|
||||
|
||||
{% set dockers = data.dockers %}
|
||||
.. tab-set::
|
||||
|
||||
{% for docker in dockers %}
|
||||
{% set jax_version = docker.components["JAX"] %}
|
||||
|
||||
.. tab-item:: ``{{ docker.pull_tag }}``
|
||||
:sync: {{ docker.pull_tag }}
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Software component
|
||||
- Version
|
||||
|
||||
{% for component_name, component_version in docker.components.items() %}
|
||||
* - {{ component_name }}
|
||||
- {{ component_version }}
|
||||
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
MaxText with on ROCm provides the following key features to train large language models efficiently:
|
||||
|
||||
- Transformer Engine (TE)
|
||||
|
||||
- Flash Attention (FA) 3 -- with or without sequence input packing
|
||||
|
||||
- GEMM tuning
|
||||
|
||||
- Multi-node support
|
||||
|
||||
- NANOO FP8 (for MI300X series GPUs) and FP8 (for MI355X and MI350X) quantization support
|
||||
|
||||
.. _amd-maxtext-model-support-v26.2:
|
||||
|
||||
Supported models
|
||||
================
|
||||
|
||||
The following models are pre-optimized for performance on AMD Instinct
|
||||
GPUs. Some instructions, commands, and available training
|
||||
configurations in this documentation might vary by model -- select one to get
|
||||
started.
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/jax-maxtext-v26.2-benchmark-models.yaml
|
||||
|
||||
{% set model_groups = data.model_groups %}
|
||||
.. raw:: html
|
||||
|
||||
<div id="vllm-benchmark-ud-params-picker" class="container-fluid">
|
||||
<div class="row gx-0">
|
||||
<div class="col-2 me-1 px-2 model-param-head">Model</div>
|
||||
<div class="row col-10 pe-0">
|
||||
{% for model_group in model_groups %}
|
||||
<div class="col-4 px-2 model-param" data-param-k="model-group" data-param-v="{{ model_group.tag }}" tabindex="0">{{ model_group.group }}</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row gx-0 pt-1">
|
||||
<div class="col-2 me-1 px-2 model-param-head">Variant</div>
|
||||
<div class="row col-10 pe-0">
|
||||
{% for model_group in model_groups %}
|
||||
{% set models = model_group.models %}
|
||||
{% for model in models %}
|
||||
{% if models|length % 3 == 0 %}
|
||||
<div class="col-4 px-2 model-param" data-param-k="model" data-param-v="{{ model.mad_tag }}" data-param-group="{{ model_group.tag }}" tabindex="0">{{ model.model }}</div>
|
||||
{% else %}
|
||||
<div class="col-6 px-2 model-param" data-param-k="model" data-param-v="{{ model.mad_tag }}" data-param-group="{{ model_group.tag }}" tabindex="0">{{ model.model }}</div>
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
.. note::
|
||||
|
||||
Some models, such as Llama 3, require an external license agreement through
|
||||
a third party (for example, Meta).
|
||||
|
||||
System validation
|
||||
=================
|
||||
|
||||
Before running AI workloads, it's important to validate that your AMD hardware is configured
|
||||
correctly and performing optimally.
|
||||
|
||||
If you have already validated your system settings, including aspects like NUMA auto-balancing, you
|
||||
can skip this step. Otherwise, complete the procedures in the :ref:`System validation and
|
||||
optimization <rocm-for-ai-system-optimization>` guide to properly configure your system settings
|
||||
before starting training.
|
||||
|
||||
To test for optimal performance, consult the recommended :ref:`System health benchmarks
|
||||
<rocm-for-ai-system-health-bench>`. This suite of tests will help you verify and fine-tune your
|
||||
system's configuration.
|
||||
|
||||
Environment setup
|
||||
=================
|
||||
|
||||
This Docker image is optimized for specific model configurations outlined
|
||||
as follows. Performance can vary for other training workloads, as AMD
|
||||
doesn’t validate configurations and run conditions outside those described.
|
||||
|
||||
Pull the Docker image
|
||||
---------------------
|
||||
|
||||
Use the following command to pull the Docker image from Docker Hub.
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/jax-maxtext-v26.2-benchmark-models.yaml
|
||||
|
||||
{% set docker = data.dockers[0] %}
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker pull {{ docker.pull_tag }}
|
||||
|
||||
.. _amd-maxtext-multi-node-setup-v26.2:
|
||||
|
||||
Multi-node configuration
|
||||
------------------------
|
||||
|
||||
See :doc:`/how-to/rocm-for-ai/system-setup/multi-node-setup` to configure your
|
||||
environment for multi-node training.
|
||||
|
||||
.. _amd-maxtext-get-started-v26.2:
|
||||
|
||||
Benchmarking
|
||||
============
|
||||
|
||||
Once the setup is complete, choose between two options to reproduce the
|
||||
benchmark results:
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/jax-maxtext-v26.2-benchmark-models.yaml
|
||||
|
||||
.. _vllm-benchmark-mad:
|
||||
|
||||
{% set docker = data.dockers[0] %}
|
||||
{% set model_groups = data.model_groups %}
|
||||
{% for model_group in model_groups %}
|
||||
{% for model in model_group.models %}
|
||||
|
||||
.. container:: model-doc {{model.mad_tag}}
|
||||
|
||||
.. tab-set::
|
||||
|
||||
{% if model.primus_config_name %}
|
||||
.. tab-item:: Primus benchmarking
|
||||
|
||||
.. container:: model-doc {{ model.mad_tag }}
|
||||
|
||||
The following run commands are tailored to {{ model.model }}.
|
||||
See :ref:`amd-maxtext-model-support-v26.2` to switch to another available model.
|
||||
|
||||
.. rubric:: Download the Docker image and required packages
|
||||
|
||||
1. Pull the ``{{ docker.pull_tag }}`` Docker image from Docker Hub.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker pull {{ docker.pull_tag }}
|
||||
|
||||
2. Run the Docker container.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker run -it \
|
||||
--device /dev/dri \
|
||||
--device /dev/kfd \
|
||||
--network host \
|
||||
--ipc host \
|
||||
--group-add video \
|
||||
--cap-add SYS_PTRACE \
|
||||
--security-opt seccomp=unconfined \
|
||||
--privileged \
|
||||
-v $HOME:$HOME \
|
||||
-v $HOME/.ssh:/root/.ssh \
|
||||
-v $HF_HOME:/hf_cache \
|
||||
-e HF_HOME=/hf_cache \
|
||||
-e MAD_SECRETS_HFTOKEN=$MAD_SECRETS_HFTOKEN
|
||||
--shm-size 64G \
|
||||
--name training_env \
|
||||
{{ docker.pull_tag }}
|
||||
|
||||
Use these commands if you exit the ``training_env`` container and need to return to it.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker start training_env
|
||||
docker exec -it training_env bash
|
||||
|
||||
3. Clone the Primus repository.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git clone https://github.com/AMD-AIG-AIMA/Primus.git
|
||||
cd Primus
|
||||
git checkout dev/fuyuajin/maxtext-backend-test
|
||||
git submodule update --init third_party/maxtext/
|
||||
|
||||
.. rubric:: Run the training job with primus-cli
|
||||
|
||||
For detailed usage instructions for ``primus-cli``, see the
|
||||
`Primus CLI User Guide
|
||||
<https://github.com/AMD-AGI/Primus/blob/main/docs/cli/PRIMUS-CLI-GUIDE.md>`__.
|
||||
|
||||
Use the following examples to run training with ``primus-cli``:
|
||||
|
||||
- Direct mode: run directly on the current host or within an existing Docker container
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MI355X
|
||||
:sync: mi355x
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
./primus-cli direct \
|
||||
-- train pretrain \
|
||||
--config examples/maxtext/configs/MI355X/{{ model.primus_config_name }}
|
||||
|
||||
.. tab-item:: MI300X
|
||||
:sync: mi300x
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
./primus-cli direct \
|
||||
-- train pretrain \
|
||||
--config examples/maxtext/configs/MI300X/{{ model.primus_config_name }}
|
||||
|
||||
- Container mode: run in Docker containers
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MI355X
|
||||
:sync: mi355x
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
./primus-cli container --image {{ docker.pull_tag }} \
|
||||
-- train pretrain \
|
||||
--config examples/maxtext/configs/MI355X/{{ model.primus_config_name }}
|
||||
|
||||
.. tab-item:: MI300X
|
||||
:sync: mi300x
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
./primus-cli container --image rocm/jax-training:maxtext-v26.2 \
|
||||
-- train pretrain \
|
||||
--config examples/maxtext/configs/MI300X/{{ model.primus_config_name }}
|
||||
|
||||
|
||||
- Slurm mode: run distributed training on a Slurm cluster
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MI355X
|
||||
:sync: mi355x
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
# Use a custom config file, where you can specify
|
||||
# the Docker image and set environment variables.
|
||||
./primus-cli --config my_maxtext_config.yaml slurm srun -N 8 \
|
||||
-- train pretrain \
|
||||
--config examples/maxtext/configs/MI355X/{{ model.primus_config_name }}
|
||||
|
||||
.. tab-item:: MI300X
|
||||
:sync: mi300x
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
# Use a custom config file, where you can specify
|
||||
# the Docker image and set environment variables.
|
||||
./primus-cli --config my_maxtext_config.yaml slurm srun -N 8 \
|
||||
-- train pretrain \
|
||||
--config examples/maxtext/configs/MI300X/{{ model.primus_config_name }}
|
||||
{% endif %}
|
||||
|
||||
{% if model.mad_tag and "single-node" in model.doc_options %}
|
||||
.. tab-item:: MAD-integrated benchmarking
|
||||
|
||||
The following run command is tailored to {{ model.model }}.
|
||||
See :ref:`amd-maxtext-model-support-v26.2` to switch to another available model.
|
||||
|
||||
1. Clone the ROCm Model Automation and Dashboarding (`<https://github.com/ROCm/MAD>`__) repository to a local
|
||||
directory and install the required packages on the host machine.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git clone https://github.com/ROCm/MAD
|
||||
cd MAD
|
||||
pip install -r requirements.txt
|
||||
|
||||
2. Use this command to run the performance benchmark test on the {{ model.model }} model
|
||||
using one GPU with the :literal:`{{model.precision}}` data type on the host machine.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
|
||||
madengine run \
|
||||
--tags {{model.mad_tag}} \
|
||||
--keep-model-dir \
|
||||
--live-output \
|
||||
--timeout 28800
|
||||
|
||||
MAD launches a Docker container with the name
|
||||
``container_ci-{{model.mad_tag}}``. The latency and throughput reports of the
|
||||
model are collected in the following path: ``~/MAD/perf.csv/``.
|
||||
{% endif %}
|
||||
|
||||
.. tab-item:: Standalone benchmarking
|
||||
|
||||
The following commands are optimized for {{ model.model }}. See
|
||||
:ref:`amd-maxtext-model-support-v26.2` to switch to another
|
||||
available model. Some instructions and resources might not be
|
||||
available for all models and configurations.
|
||||
|
||||
.. rubric:: Download the Docker image and required scripts
|
||||
|
||||
Run the JAX MaxText benchmark tool independently by starting the
|
||||
Docker container as shown in the following snippet.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker pull {{ docker.pull_tag }}
|
||||
|
||||
{% if model.model_repo and "single-node" in model.doc_options %}
|
||||
.. rubric:: Single node training
|
||||
|
||||
1. Set up environment variables.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
export MAD_SECRETS_HFTOKEN=<Your Hugging Face token>
|
||||
export HF_HOME=<Location of saved/cached Hugging Face models>
|
||||
|
||||
``MAD_SECRETS_HFTOKEN`` is your Hugging Face access token to access models, tokenizers, and data.
|
||||
See `User access tokens <https://huggingface.co/docs/hub/en/security-tokens>`__.
|
||||
|
||||
``HF_HOME`` is where ``huggingface_hub`` will store local data. See `huggingface_hub CLI <https://huggingface.co/docs/huggingface_hub/main/en/guides/cli#huggingface-cli-download>`__.
|
||||
If you already have downloaded or cached Hugging Face artifacts, set this variable to that path.
|
||||
Downloaded files typically get cached to ``~/.cache/huggingface``.
|
||||
|
||||
2. Launch the Docker container.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker run -it \
|
||||
--device=/dev/dri \
|
||||
--device=/dev/kfd \
|
||||
--network host \
|
||||
--ipc host \
|
||||
--group-add video \
|
||||
--cap-add=SYS_PTRACE \
|
||||
--security-opt seccomp=unconfined \
|
||||
--privileged \
|
||||
-v $HOME:$HOME \
|
||||
-v $HOME/.ssh:/root/.ssh \
|
||||
-v $HF_HOME:/hf_cache \
|
||||
-e HF_HOME=/hf_cache \
|
||||
-e MAD_SECRETS_HFTOKEN=$MAD_SECRETS_HFTOKEN
|
||||
--shm-size 64G \
|
||||
--name training_env \
|
||||
{{ docker.pull_tag }}
|
||||
|
||||
3. In the Docker container, clone the ROCm MAD repository and navigate to the
|
||||
benchmark scripts directory at ``MAD/scripts/jax-maxtext``.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git clone https://github.com/ROCm/MAD
|
||||
cd MAD/scripts/jax-maxtext
|
||||
|
||||
4. Run the setup scripts to install libraries and datasets needed
|
||||
for benchmarking.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
./jax-maxtext_benchmark_setup.sh -m {{ model.model_repo }}
|
||||
|
||||
5. To run the training benchmark without quantization, use the following command:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
./jax-maxtext_benchmark_report.sh -m {{ model.model_repo }}
|
||||
|
||||
For quantized training, run the script with the appropriate option for your Instinct GPU.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MI355X and MI350X
|
||||
|
||||
For ``fp8`` quantized training on MI355X and MI350X GPUs, use the following command:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
./jax-maxtext_benchmark_report.sh -m {{ model.model_repo }} -q fp8
|
||||
|
||||
{% if model.model_repo not in ["Llama-3.1-70B", "Llama-3.3-70B"] %}
|
||||
.. tab-item:: MI325X and MI300X
|
||||
|
||||
For ``nanoo_fp8`` quantized training on MI300X series GPUs, use the following command:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
./jax-maxtext_benchmark_report.sh -m {{ model.model_repo }} -q nanoo_fp8
|
||||
{% endif %}
|
||||
|
||||
{% endif %}
|
||||
{% if model.multinode_config and "multi-node" in model.doc_options %}
|
||||
.. rubric:: Multi-node training
|
||||
|
||||
The following SLURM scripts will launch the Docker container and
|
||||
run the benchmark. Run them outside of any Docker container. The
|
||||
unified multi-node benchmark script accepts a configuration file
|
||||
that specifies the model and training parameters.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
sbatch -N <NUM_NODES> jax_maxtext_multinode_benchmark.sh <config_file.yml> [docker_image]
|
||||
|
||||
<NUM_NODES>
|
||||
The number of nodes to use for training (for example, 2, 4,
|
||||
8).
|
||||
|
||||
<config_file.yml>
|
||||
Path to the YAML configuration file containing model and
|
||||
training parameters. Configuration files are available in the
|
||||
``scripts/jax-maxtext/env_scripts/`` directory for different
|
||||
models and GPU architectures.
|
||||
|
||||
[docker_image] (optional)
|
||||
The Docker image to use. If not specified, it defaults to
|
||||
``rocm/jax-training:maxtext-v26.2``.
|
||||
|
||||
For example, to run a multi-node training benchmark on {{ model.model }}:
|
||||
|
||||
.. tab-set::
|
||||
|
||||
{% if model.multinode_config.gfx950 %}
|
||||
.. tab-item:: MI355X and MI350X (gfx950)
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sbatch -N 4 jax_maxtext_multinode_benchmark.sh {{ model.multinode_config.gfx950 }}
|
||||
{% endif %}
|
||||
|
||||
{% if model.multinode_config.gfx942 %}
|
||||
.. tab-item:: MI325X and MI300X (gfx942)
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sbatch -N 4 jax_maxtext_multinode_benchmark.sh {{ model.multinode_config.gfx942 }}
|
||||
{% endif %}
|
||||
|
||||
{% else %}
|
||||
.. rubric:: Multi-node training
|
||||
|
||||
For multi-node training examples, choose a model from :ref:`amd-maxtext-model-support-v26.2`
|
||||
with an available `multi-node training script <https://github.com/ROCm/MAD/tree/develop/scripts/jax-maxtext/env_scripts>`__.
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
Known issues
|
||||
============
|
||||
|
||||
- You might see NaNs in the losses when setting ``packing=True``. As
|
||||
a workaround, turn off input sequence packing (``packing=False``).
|
||||
This will be fixed in a future release.
|
||||
|
||||
Further reading
|
||||
===============
|
||||
|
||||
- To learn more about MAD and the ``madengine`` CLI, see the `MAD usage guide <https://github.com/ROCm/MAD?tab=readme-ov-file#usage-guide>`__.
|
||||
|
||||
- To learn more about system settings and management practices to configure your system for
|
||||
AMD Instinct MI300X Series GPUs, see `AMD Instinct MI300X system optimization <https://instinct.docs.amd.com/projects/amdgpu-docs/en/latest/system-optimization/mi300x.html>`_.
|
||||
|
||||
- For a list of other ready-made Docker images for AI with ROCm, see
|
||||
`AMD Infinity Hub <https://www.amd.com/en/developer/resources/infinity-hub.html#f-amd_hub_category=AI%20%26%20ML%20Models>`_.
|
||||
|
||||
Previous versions
|
||||
=================
|
||||
|
||||
See :doc:`jax-maxtext-history` to find documentation for previous releases
|
||||
of the ``ROCm/jax-training`` Docker image.
|
||||
+1
-9
@@ -16,20 +16,12 @@ previous releases of the ``ROCm/primus`` Docker image on `Docker Hub <https://hu
|
||||
- Components
|
||||
- Resources
|
||||
|
||||
* - v26.3 (latest)
|
||||
* - v26.2 (latest)
|
||||
-
|
||||
* ROCm 7.2.0
|
||||
* PyTorch 2.10.0+git94c6e04
|
||||
-
|
||||
* :doc:`Primus Megatron documentation <../primus-megatron>`
|
||||
* `Docker Hub <https://hub.docker.com/layers/rocm/primus/v26.3/images/sha256-da50bfe9dc4bb70ad683d3ee4a176dccb66f596f48d30e8e52323b41892759b1>`__
|
||||
|
||||
* - v26.2
|
||||
-
|
||||
* ROCm 7.2.0
|
||||
* PyTorch 2.10.0+git94c6e04
|
||||
-
|
||||
* :doc:`Primus Megatron documentation <primus-megatron-v26.2>`
|
||||
* `Docker Hub <https://hub.docker.com/layers/rocm/primus/v26.2/images/sha256-9148d1bfcd579bf92f44bd89090e0d8c958f149c134b4b34b9674ab559244585>`__
|
||||
|
||||
* - v26.1
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
*****************************************************************
|
||||
Migrating workloads to Primus (Megatron backend) from Megatron-LM
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using ROCm Megatron-LM
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
|
||||
+3
-4
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using Megatron-LM for ROCm.
|
||||
@@ -26,12 +25,12 @@ model training. Performance acceleration is powered by `Primus Turbo
|
||||
<https://hub.docker.com/r/rocm/megatron-lm/>`__ Docker Hub registry will be
|
||||
deprecated soon in favor of `rocm/primus <https://hub.docker.com/r/rocm/primus>`__.
|
||||
The ``rocm/primus`` Docker containers will cover PyTorch training ecosystem frameworks,
|
||||
including Megatron-LM and :doc:`torchtitan <../primus-pytorch>`.
|
||||
including Megatron-LM and :doc:`torchtitan <primus-pytorch>`.
|
||||
|
||||
Primus with Megatron is designed to replace the :doc:`ROCm Megatron-LM
|
||||
training <../megatron-lm>` workflow. To learn how to migrate workloads from
|
||||
training <megatron-lm>` workflow. To learn how to migrate workloads from
|
||||
Megatron-LM to Primus with Megatron, see
|
||||
:doc:`megatron-lm-primus-migration-guide`.
|
||||
:doc:`previous-versions/megatron-lm-primus-migration-guide`.
|
||||
|
||||
AMD provides a ready-to-use Docker images for MI355X, MI350X,
|
||||
MI325X, and MI300X GPUs containing essential components for Primus, ROCm, and
|
||||
|
||||
-1240
File diff suppressed because it is too large
Load Diff
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
|
||||
+5
-6
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
@@ -25,11 +24,11 @@ Primus now supports the PyTorch torchtitan backend.
|
||||
<https://hub.docker.com/r/rocm/pytorch-training/>`__ Docker Hub registry will be
|
||||
deprecated soon in favor of `rocm/primus <https://hub.docker.com/r/rocm/primus>`__.
|
||||
The ``rocm/primus`` Docker containers will cover PyTorch training ecosystem frameworks,
|
||||
including torchtitan and :doc:`Megatron-LM <../primus-megatron>`.
|
||||
including torchtitan and :doc:`Megatron-LM <primus-megatron>`.
|
||||
|
||||
Primus with the PyTorch torchtitan backend is designed to replace the
|
||||
:doc:`ROCm PyTorch training <../pytorch-training>` workflow. See
|
||||
:doc:`../pytorch-training` to see steps to run workloads without Primus.
|
||||
:doc:`ROCm PyTorch training <pytorch-training>` workflow. See
|
||||
:doc:`pytorch-training` to see steps to run workloads without Primus.
|
||||
|
||||
AMD provides a ready-to-use Docker image for MI355X, MI350X, MI325X, and
|
||||
MI300X GPUs containing essential components for Primus and PyTorch training
|
||||
@@ -97,7 +96,7 @@ vary by model -- select one to get started.
|
||||
.. seealso::
|
||||
|
||||
For additional workloads, including Llama 3.3, Llama 3.2, Llama 2, GPT OSS, Qwen, and Flux models,
|
||||
see the documentation :doc:`../pytorch-training` (without Primus)
|
||||
see the documentation :doc:`pytorch-training` (without Primus)
|
||||
|
||||
.. _amd-primus-pytorch-performance-measurements-v26.01:
|
||||
|
||||
@@ -135,7 +134,7 @@ Run training
|
||||
============
|
||||
|
||||
Once the setup is complete, choose between the following two workflows to start benchmarking training.
|
||||
For fine-tuning workloads and multi-node training examples, see :doc:`../pytorch-training` (without Primus).
|
||||
For fine-tuning workloads and multi-node training examples, see :doc:`pytorch-training` (without Primus).
|
||||
For best performance on MI325X, MI350X, and MI355X GPUs, you might need to
|
||||
tweak some configurations (such as batch sizes).
|
||||
|
||||
|
||||
-403
@@ -1,403 +0,0 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
:keywords: ROCm, AI, LLM, train, PyTorch, torch, Llama, flux, tutorial, docker
|
||||
|
||||
****************************************
|
||||
Training a model with Primus and PyTorch
|
||||
****************************************
|
||||
|
||||
.. caution::
|
||||
|
||||
This documentation does not reflect the latest version of ROCm Primus PyTorch training
|
||||
performance benchmark documentation. See :doc:`../primus-pytorch` for the latest version.
|
||||
|
||||
`Primus <https://github.com/AMD-AGI/Primus>`__ is a unified and flexible
|
||||
LLM training framework designed to streamline training. It streamlines LLM
|
||||
training on AMD Instinct GPUs using a modular, reproducible configuration paradigm.
|
||||
Primus now supports the PyTorch torchtitan backend.
|
||||
|
||||
.. note::
|
||||
|
||||
For a unified training solution on AMD GPUs with ROCm, the `rocm/pytorch-training
|
||||
<https://hub.docker.com/r/rocm/pytorch-training/>`__ Docker Hub registry will be
|
||||
deprecated soon in favor of `rocm/primus <https://hub.docker.com/r/rocm/primus>`__.
|
||||
The ``rocm/primus`` Docker containers will cover PyTorch training ecosystem frameworks,
|
||||
including torchtitan and :doc:`Megatron-LM <../primus-megatron>`.
|
||||
|
||||
Primus with the PyTorch torchtitan backend is designed to replace the
|
||||
:doc:`ROCm PyTorch training <../pytorch-training>` workflow. See
|
||||
:doc:`../pytorch-training` to see steps to run workloads without Primus.
|
||||
|
||||
AMD provides a ready-to-use Docker image for MI355X, MI350X, MI325X, and
|
||||
MI300X GPUs containing essential components for Primus and PyTorch training
|
||||
with Primus Turbo optimizations.
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/primus-pytorch-v26.2-benchmark-models.yaml
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: {{ data.docker.pull_tag }}
|
||||
:sync: {{ data.docker.pull_tag }}
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Software component
|
||||
- Version
|
||||
|
||||
{% for component_name, component_version in data.docker.components.items() %}
|
||||
* - {{ component_name }}
|
||||
- {{ component_version }}
|
||||
{% endfor %}
|
||||
|
||||
.. _amd-primus-pytorch-model-support-v26.2:
|
||||
|
||||
Supported models
|
||||
================
|
||||
|
||||
The following models are pre-optimized for performance on the AMD Instinct MI325X and MI300X GPUs.
|
||||
Some instructions, commands, and training recommendations in this documentation might
|
||||
vary by model -- select one to get started.
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/primus-pytorch-v26.2-benchmark-models.yaml
|
||||
|
||||
{% set model_groups = data.model_groups %}
|
||||
.. raw:: html
|
||||
|
||||
<div id="vllm-benchmark-ud-params-picker" class="container-fluid">
|
||||
<div class="row gx-0">
|
||||
<div class="col-2 me-1 px-2 model-param-head">Model</div>
|
||||
<div class="row col-10 pe-0">
|
||||
{% for model_group in model_groups %}
|
||||
<div class="col-6 px-2 model-param" data-param-k="model-group" data-param-v="{{ model_group.tag }}" tabindex="0">{{ model_group.group }}</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row gx-0 pt-1">
|
||||
<div class="col-2 me-1 px-2 model-param-head">Variant</div>
|
||||
<div class="row col-10 pe-0">
|
||||
{% for model_group in model_groups %}
|
||||
{% set models = model_group.models %}
|
||||
{% for model in models %}
|
||||
{% if models|length % 3 == 0 %}
|
||||
<div class="col-4 px-2 model-param" data-param-k="model" data-param-v="{{ model.mad_tag }}" data-param-group="{{ model_group.tag }}" tabindex="0">{{ model.model }}</div>
|
||||
{% else %}
|
||||
<div class="col-6 px-2 model-param" data-param-k="model" data-param-v="{{ model.mad_tag }}" data-param-group="{{ model_group.tag }}" tabindex="0">{{ model.model }}</div>
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
.. seealso::
|
||||
|
||||
For additional workloads, including Llama 3.3, Llama 3.2, Llama 2, GPT OSS, Qwen, and Flux models,
|
||||
see the documentation :doc:`../pytorch-training` (without Primus)
|
||||
|
||||
.. _amd-primus-pytorch-performance-measurements-v26.2:
|
||||
|
||||
System validation
|
||||
=================
|
||||
|
||||
Before running AI workloads, it's important to validate that your AMD hardware is configured
|
||||
correctly and performing optimally.
|
||||
|
||||
If you have already validated your system settings, including aspects like NUMA auto-balancing, you
|
||||
can skip this step. Otherwise, complete the procedures in the :ref:`System validation and
|
||||
optimization <rocm-for-ai-system-optimization>` guide to properly configure your system settings
|
||||
before starting training.
|
||||
|
||||
To test for optimal performance, consult the recommended :ref:`System health benchmarks
|
||||
<rocm-for-ai-system-health-bench>`. This suite of tests will help you verify and fine-tune your
|
||||
system's configuration.
|
||||
|
||||
This Docker image is optimized for specific model configurations outlined
|
||||
below. Performance can vary for other training workloads, as AMD
|
||||
doesn’t test configurations and run conditions outside those described.
|
||||
|
||||
Pull the Docker image
|
||||
=====================
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/primus-pytorch-v26.2-benchmark-models.yaml
|
||||
|
||||
Use the following command to pull the Docker image from Docker Hub.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker pull {{ data.docker.pull_tag }}
|
||||
|
||||
Run training
|
||||
============
|
||||
|
||||
Once the setup is complete, choose between the following two workflows to start benchmarking training.
|
||||
For fine-tuning workloads and multi-node training examples, see :doc:`../pytorch-training` (without Primus).
|
||||
For best performance on MI325X, MI350X, and MI355X GPUs, you might need to
|
||||
tweak some configurations (such as batch sizes).
|
||||
|
||||
.. datatemplate:yaml:: /data/how-to/rocm-for-ai/training/previous-versions/primus-pytorch-v26.2-benchmark-models.yaml
|
||||
|
||||
{% set docker = data.docker %}
|
||||
{% set model_groups = data.model_groups %}
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Primus benchmarking
|
||||
|
||||
{% for model_group in model_groups %}
|
||||
{% for model in model_group.models %}
|
||||
|
||||
.. container:: model-doc {{ model.mad_tag }}
|
||||
|
||||
The following run commands are tailored to {{ model.model }}.
|
||||
See :ref:`amd-primus-pytorch-model-support-v26.2` to switch to another available model.
|
||||
|
||||
.. rubric:: Download the Docker image and required packages
|
||||
|
||||
1. Pull the ``{{ docker.pull_tag }}`` Docker image from Docker Hub.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker pull {{ docker.pull_tag }}
|
||||
|
||||
2. Run the Docker container.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker run -it \
|
||||
--device /dev/dri \
|
||||
--device /dev/kfd \
|
||||
--network host \
|
||||
--ipc host \
|
||||
--group-add video \
|
||||
--cap-add SYS_PTRACE \
|
||||
--security-opt seccomp=unconfined \
|
||||
--privileged \
|
||||
-v $HOME:$HOME \
|
||||
-v $HOME/.ssh:/root/.ssh \
|
||||
--shm-size 64G \
|
||||
--name training_env \
|
||||
{{ docker.pull_tag }}
|
||||
|
||||
Use these commands if you exit the ``training_env`` container and need to return to it.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
docker start training_env
|
||||
docker exec -it training_env bash
|
||||
|
||||
The Docker container hosts verified commit ``9c529cd4`` of the `Primus
|
||||
<https://github.com/AMD-AGI/Primus/tree/9c529cd4a934a68a880ede036c3e97b792e38167/>`__ repository.
|
||||
|
||||
.. rubric:: Prepare training datasets and dependencies
|
||||
|
||||
The following benchmarking examples require downloading models and datasets
|
||||
from Hugging Face. To ensure successful access to gated repos, set your
|
||||
``HF_TOKEN``.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
export HF_TOKEN=$your_personal_hugging_face_access_token
|
||||
|
||||
.. rubric:: Pretraining
|
||||
|
||||
To get started, navigate to the ``Primus`` directory in your container.
|
||||
|
||||
.. code-block::
|
||||
|
||||
cd /workspace/Primus
|
||||
|
||||
Now, to start the pretraining benchmark, use the ``run_pretrain.sh`` script
|
||||
included with Primus with the appropriate options.
|
||||
|
||||
.. rubric:: Benchmarking examples
|
||||
|
||||
.. container:: model-doc primus_pyt_train_llama-3.1-8b
|
||||
|
||||
Use the following command to run train Llama 3.1 8B with BF16 precision using Primus torchtitan.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MI355X and MI350X
|
||||
:sync: MI355X
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
bash runner/primus-cli direct \
|
||||
--log_file /tmp/primus_llama3.1_8B.log \
|
||||
-- train pretrain \
|
||||
--config examples/torchtitan/configs/MI355X/llama3.1_8B-BF16-pretrain.yaml
|
||||
|
||||
.. tab-item:: MI300X
|
||||
:sync: MI300X
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
bash runner/primus-cli direct \
|
||||
--log_file /tmp/primus_llama3.1_8B.log \
|
||||
-- train pretrain \
|
||||
--config examples/torchtitan/configs/MI300X/llama3.1_8B-BF16-pretrain.yaml
|
||||
|
||||
To train Llama 3.1 8B with FP8 precision, use the following command.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MI355X and MI350X
|
||||
:sync: MI355X
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
bash runner/primus-cli direct \
|
||||
--log_file /tmp/primus_llama3.1_8B_fp8.log \
|
||||
-- train pretrain \
|
||||
--config examples/torchtitan/configs/MI355X/llama3.1_8B-FP8-pretrain.yaml
|
||||
|
||||
.. tab-item:: MI300X
|
||||
:sync: MI300X
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
bash runner/primus-cli direct \
|
||||
--log_file /tmp/primus_llama3.1_8B_fp8.log \
|
||||
-- train pretrain \
|
||||
--config examples/torchtitan/configs/MI300X/llama3.1_8B-FP8-pretrain.yaml
|
||||
|
||||
.. container:: model-doc primus_pyt_train_llama-3.1-70b
|
||||
|
||||
Use the following command to run train Llama 3.1 70B with BF16 precision using Primus torchtitan.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MI355X and MI350X
|
||||
:sync: MI355X and MI300X
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
bash runner/primus-cli direct \
|
||||
--log_file /tmp/primus_llama3.1_70B.log \
|
||||
-- train pretrain \
|
||||
--config examples/torchtitan/configs/MI355X/llama3.1_70B-BF16-pretrain.yaml
|
||||
|
||||
.. tab-item:: MI300X
|
||||
:sync: MI300X
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
bash runner/primus-cli direct \
|
||||
--log_file /tmp/primus_llama3.1_70B.log \
|
||||
-- train pretrain \
|
||||
--config examples/torchtitan/configs/MI300X/llama3.1_70B-BF16-pretrain.yaml
|
||||
|
||||
To train Llama 3.1 70B with FP8 precision, use the following command.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MI355X and MI350X
|
||||
:sync: MI355X
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
bash runner/primus-cli direct \
|
||||
--log_file /tmp/primus_llama3.1_70B_fp8.log \
|
||||
-- train pretrain \
|
||||
--config examples/torchtitan/configs/MI355X/llama3.1_70B-FP8-pretrain.yaml
|
||||
|
||||
.. tab-item:: MI300X
|
||||
:sync: MI300X
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
bash runner/primus-cli direct \
|
||||
--log_file /tmp/primus_llama3.1_70B_fp8.log \
|
||||
-- train pretrain \
|
||||
--config examples/torchtitan/configs/MI300X/llama3.1_70B-FP8-pretrain.yaml
|
||||
|
||||
.. container:: model-doc primus_pyt_train_deepseek-v3-16b
|
||||
|
||||
Use the following command to run train DeepSeek V3 16B with BF16 precision using Primus torchtitan.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: MI355X and MI350X
|
||||
:sync: MI355X and MI300X
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
bash runner/primus-cli direct \
|
||||
--log_file /tmp/primus_deepseek_v3_16b.log \
|
||||
-- train pretrain \
|
||||
--config examples/torchtitan/configs/MI355X/deepseek_v3_16b-pretrain.yaml
|
||||
|
||||
.. tab-item:: MI300X
|
||||
:sync: MI300X
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
bash runner/primus-cli direct \
|
||||
--log_file /tmp/primus_deepseek_v3_16b.log \
|
||||
-- train pretrain \
|
||||
--config examples/torchtitan/configs/MI300X/deepseek_v3_16b-pretrain.yaml
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
.. tab-item:: MAD-integrated benchmarking
|
||||
|
||||
{% for model_group in model_groups %}
|
||||
{% for model in model_group.models %}
|
||||
|
||||
.. container:: model-doc {{ model.mad_tag }}
|
||||
|
||||
The following run command is tailored to {{ model.model }}.
|
||||
See :ref:`amd-primus-pytorch-model-support-v26.2` to switch to another available model.
|
||||
|
||||
1. Clone the ROCm Model Automation and Dashboarding (`<https://github.com/ROCm/MAD>`__) repository to a local
|
||||
directory and install the required packages on the host machine.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git clone https://github.com/ROCm/MAD
|
||||
cd MAD
|
||||
pip install -r requirements.txt
|
||||
|
||||
2. For example, use this command to run the performance benchmark test on the {{ model.model }} model
|
||||
using one node with the {{ model.precision }} data type on the host machine.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
|
||||
madengine run \
|
||||
--tags {{ model.mad_tag }} \
|
||||
--keep-model-dir \
|
||||
--live-output \
|
||||
--timeout 28800
|
||||
|
||||
MAD launches a Docker container with the name
|
||||
``container_ci-{{ model.mad_tag }}``. The latency and throughput reports of the
|
||||
model are collected in ``~/MAD/perf.csv``.
|
||||
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
|
||||
Further reading
|
||||
===============
|
||||
|
||||
- For an introduction to Primus, see `Primus: A Lightweight, Unified Training
|
||||
Framework for Large Models on AMD GPUs <https://rocm.blogs.amd.com/software-tools-optimization/primus/README.html>`__.
|
||||
|
||||
- To learn more about MAD and the ``madengine`` CLI, see the `MAD usage guide <https://github.com/ROCm/MAD?tab=readme-ov-file#usage-guide>`__.
|
||||
|
||||
- To learn more about system settings and management practices to configure your system for
|
||||
AMD Instinct MI300X Series GPUs, see `AMD Instinct MI300X system optimization <https://instinct.docs.amd.com/projects/amdgpu-docs/en/latest/system-optimization/mi300x.html>`_.
|
||||
|
||||
- For a list of other ready-made Docker images for AI with ROCm, see
|
||||
`AMD Infinity Hub <https://www.amd.com/en/developer/resources/infinity-hub.html#f-amd_hub_category=AI%20%26%20ML%20Models>`_.
|
||||
|
||||
Previous versions
|
||||
=================
|
||||
|
||||
See :doc:`pytorch-training-history` to find documentation for previous releases
|
||||
of the ``ROCm/pytorch-training`` Docker image.
|
||||
+2
-10
@@ -16,20 +16,12 @@ previous releases of the ``ROCm/primus`` Docker image on `Docker Hub <https://hu
|
||||
- Components
|
||||
- Resources
|
||||
|
||||
* - v26.3 (latest)
|
||||
-
|
||||
* ROCm 7.2.1
|
||||
* PyTorch 2.10.0+git94c6e04
|
||||
-
|
||||
* :doc:`Primus PyTorch training documentation <../primus-pytorch>`
|
||||
* `Docker Hub <https://hub.docker.com/layers/rocm/primus/v26.3/images/sha256-da50bfe9dc4bb70ad683d3ee4a176dccb66f596f48d30e8e52323b41892759b1>`__
|
||||
|
||||
* - v26.2
|
||||
* - v26.2 (latest)
|
||||
-
|
||||
* ROCm 7.2.0
|
||||
* PyTorch 2.10.0+git94c6e04
|
||||
-
|
||||
* :doc:`Primus PyTorch training documentation <primus-pytorch-v26.2>`
|
||||
* :doc:`Primus PyTorch training documentation <../primus-pytorch>`
|
||||
* `Docker Hub <https://hub.docker.com/layers/rocm/primus/v26.2/images/sha256-9148d1bfcd579bf92f44bd89090e0d8c958f149c134b4b34b9674ab559244585>`__
|
||||
|
||||
* - v26.1
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
:orphan:
|
||||
:no-search:
|
||||
|
||||
.. meta::
|
||||
:description: How to train a model using PyTorch for ROCm.
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user