From 8b163d01556784d51593d3d2b6d0a7d50d81b26b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=B4=B9=E6=94=BF=E8=81=AA?= <3300949577@qq.com> Date: Wed, 17 Jul 2024 02:22:29 -0500 Subject: [PATCH] Update README.md --- README.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index f5d1e86..200d3d4 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -## Scaling Diffusion Transformers with Mixture of Experts
Official PyTorch Implementation +![image](https://github.com/user-attachments/assets/bf922145-48b2-40cc-9d15-bd1cc608d785)## Scaling Diffusion Transformers with Mixture of Experts
Official PyTorch Implementation [![arXiv](https://img.shields.io/badge/arXiv-2407.11633-b31b1b.svg)](https://arxiv.org/abs/2407.11633) @@ -32,6 +32,19 @@ torchrun --nnodes=1 --nproc_per_node=N train.py \ --global-batch-size 256 ``` +For multiple node training, we solve the [bug](https://github.com/facebookresearch/DiT/blob/main/train.py#L149) at original DiT repository, and you can run with 8 nodes as: +```bash +torchrun --nnodes=8 \ + --node_rank=0 \ + --nproc_per_node=8 \ + --master_addr="10.0.0.0" \ + --master_port=1234 \ + train.py \ + --model DiT-B/2 \ + --global-batch-size 1024 \ + --data-path /path/to/imagenet/train +``` + ### 2. Inference