Scale neural network training

Train neural networks with distributed training and experiment tracking

When you want to scale PyTorch training across GPUs without rewriting boilerplate.

• · PyTorch model
• · DataLoader
• · loss function
• · optimizer

• · trained model checkpoint
• · metrics logs
• · inference-ready weights

• · Python
• · PyTorch Lightning
• · PyTorch
• · optional: Tensorboard / Weights & Biases

• · data in DataLoader format
• · model inherits LightningModule

• · out-of-memory during distributed training
• · learning rate too high/low

• · automatic mixed precision (AMP)
• · checkpoint/resume
• · gradient accumulation