Can someone point out where in the week 1 list of topics the concept of 2D parallelism and 3D parallelism is covered … I do not recall seeing those phrases “2D parallelism” and “3D parallelism” in any of the videos.
Where is 2D parallelism and 3D parallelism mentioned in the week 1 topics?
These concepts seem to relate to “Model Parallelism” where large models are split between GPUs.