Sharding

The last decade has witnessed growth in the computational requirements for training deep neural networks. Current approaches (e.g., …