Pytorch Lightning GuideSAFE

🔬 A curated collection of 23,000+ agent skills for empirical research across 8 social science disciplines. | 精选 23,000+ AI Agent 技能库,覆盖8大社会科学学科的实证研究。CoPaper.AI 20分钟完成一篇可复现的规范实证论文,并支持用户上传 Skills。-- Maintained by CoPaper.AI from Stanford REAP.

The instruction file, verbatim from the audited commit — this is the text the model reads, and the surface the audit's instruction layer examines. Quoted here so you can judge it without cloning anything.

---
name: pytorch-lightning-guide
description: "PyTorch Lightning framework for scalable model training and research"
metadata:
  openclaw:
    emoji: "⚡"
    category: "domains"
    subcategory: "ai-ml"
    keywords: ["pytorch-lightning", "training", "distributed", "finetuning", "scalability", "research"]
    source: "https://github.com/Lightning-AI/pytorch-lightning"
---

# PyTorch Lightning Guide

## Overview

PyTorch Lightning is a deep learning framework with over 31,000 GitHub stars that provides a high-level interface for PyTorch, enabling researchers to focus on model design rather than engineering boilerplate. Developed by Lightning AI, it decouples the science (model architecture, loss functions, data processing) from the engineering (distributed training, mixed precision, gradient accumulation, checkpointing) through a structured `LightningModule` abstraction.

For academic researchers, Lightning eliminates the need to write repetitive training loops, device management code, and distributed training logic. You define your model, training step, and data loaders, and Lightning handles everything else -- from single GPU to multi-node distributed training, from FP32 to mixed precision, from local development to cloud deployment. This means faster iteration on research ideas with production-quality training infrastructure.

Lightning is used extensively in AI research labs and has become a standard tool for reproducible deep learning experiments. It integrates seamlessly with experiment tracking tools like Weights & Biases, MLflow, and TensorBoard, and supports all PyTorch-compatible model architectures.

## Installation and Setup

```bash
# Install PyTorch Lightning
pip install lightning

# Or install with specific extras
pip install lightning[extra]

# For development/research with all features
pip install lightning[all]
```

Lightning requires Python 3.9+ and PyTorch 2.1+. For GPU training, ensure your PyTorch installation includes CUDA support:

```bash
# Check GPU availability
python -c "import torch; print(torch.cuda.is_available())"
```

Verify your installation:

```python
import lightning as L
print(L.__version__)
```

## Core Architecture

### The LightningModule

The `LightningModule` is the central abstraction. It organizes your PyTorch code into clearly defined methods:

```python
import lightning as L
import torch
import torch.nn.functional as F
from torch import nn

class ResearchModel(L.LightningModule):
    def __init__(self, input_dim, hidden_dim, output_dim, lr=1e-3):
        super().__init__()
        self.save_hyperparameters()
        self.encoder = nn.Sequential(
            nn.Linear(input_dim, hidden_dim),
            nn.ReLU(),
            nn.Dropout(0.2),
            nn.Linear(hidden_dim, hidden_dim),
            nn.ReLU(),
        )
        self.classifier = nn.Linear(hidden_dim, output_dim)
        self.lr = lr

    def forward(self, x):
        features = self.encoder(x)
        return self.classifier(features)

    def training_step(self, batch, batch_idx):
        x, y = batch
        logits = self(x)
        loss = F.cross_entropy(logits, y)
        acc = (logits.argmax(dim=-1) == y).float().mean()
        self.log("train_loss", loss, prog_bar=True)
        self.log("train_acc", acc, prog_bar=True)
        return loss

    def validation_step(self, batch, batch_idx):
        x, y = batch
        logits = self(x)
        loss = F.cross_entropy(logits, y)
        acc = (logits.argmax(dim=-1) == y).float().mean()
        self.log("val_loss", loss, prog_bar=True)
        self.log("val_acc", acc, prog_bar=True)

    def configure_optimizers(self):
        optimizer = torch.optim.AdamW(self.parameters(), lr=self.lr)
        scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(
            optimizer, T_max=self.trainer.max_epochs
        )
        return [optimizer], [scheduler]
```

### The LightningDataModule

Encapsulate all data processing in a reusable `LightningDataModule`:

```python
class ResearchDataModule(L.LightningDataModule):
    def __init__(self, data_dir, batch_size=32, num_workers=4):
        super().__init__()
        self.data_dir = data_dir
        self.batch_size = batch_size
        self.num_workers = num_workers

    def setup(self, stage=None):
        # Load and split data
        dataset = load_research_dataset(self.data_dir)
        self.train_data, self.val_data, self.test_data = random_split(
            dataset, [0.8, 0.1, 0.1]
        )

    def train_dataloader(self):
        return DataLoader(self.train_data, batch_size=self.batch_size,
                         shuffle=True, num_workers=self.num_workers)

    def val_dataloader(self):
        return DataLoader(self.val_data, batch_size=self.batch_size,
                         num_workers=self.num_workers)
```

### The Trainer

The `Trainer` orchestrates everything with a rich set of configuration options:

```python
trainer = L.Trainer(
    max_epochs=100,
    accelerator="gpu",
    devices=4,
    strategy="ddp",
    precision="16-mixed",
    gradient_clip_val=1.0,
    accumulate_grad_batches=4,
    callbacks=[
        L.callbacks.EarlyStopping(monitor="val_loss", patience=10),
        L.callbacks.ModelCheckpoint(monitor="val_loss", save_top_k=3),
        L.callbacks.LearningRateMonitor(),
    ],
    logger=L.loggers.WandbLogger(project="my-research"),
)

# Train the model
trainer.fit(model, datamodule=data_module)

# Test with best checkpoint
trainer.test(model, datamodule=data_module, ckpt_path="best")
```

## Advanced Research Features

### Distributed Training Strategies

Lightning supports multiple distributed training strategies out of the box:

- **DDP (Distributed Data Parallel)**: Standard multi-GPU training
- **FSDP (Fully Sharded Data Parallel)**: Memory-efficient training for large models
- **DeepSpeed**: ZeRO optimization stages 1, 2, and 3

```python
# FSDP for large model training
trainer = L.Trainer(
    strategy="fsdp",
    devices=8,
    precision="bf16

SAFEgrade B · trust 89/100 Nothing in the source contradicts what it says it does. Grade A is reserved for packages that have also passed the behavioural sandbox.

What the source does

Filesystem

none-observed

Network

none-observed

Shell

none-observed

Dependencies

pinned

Secrets in source

none-found

Findings (0)

No findings outside the package's declared scope.

Gates applied: no_behavioural_pass.