Interesting read: How Meta trains large language models at scale

 Interesting... https://engineering.fb.com/2024/06/12/data-infrastructure/training-large-language-models-at-scale-meta/