Data Visualization and Model Evaluation: Essential Skills for PyTorch Beginners and Advanced Learners¶

Why Learn Data Visualization and Model Evaluation?¶

In PyTorch learning, the effectiveness of a model often needs to be intuitively understood through data visualization, while model evaluation is the key to testing how well a model performs. Whether checking data validity, troubleshooting training anomalies, or analyzing model performance across different scenarios, these skills help you quickly identify issues and optimize the model.

1. Data Visualization: Unveiling the “True Face” of Data¶

1. Why Visualize Data?¶

• Understand Data Distribution: For example, pixel ranges of image data or distribution patterns of numerical data.
• Detect Anomalies: Visualization can quickly reveal odd samples or values in the data.
• Validate Data Preprocessing: E.g., whether normalized data meets expectations.

2. Common Visualization Tools in PyTorch¶

• Matplotlib: A fundamental Python visualization library, ideal for quickly plotting images and statistical charts.
• TensorBoard: An official PyTorch-recommended tool for dynamic visualization during training (e.g., loss curves, model structures).

3. Matplotlib Practice: From Images to Statistics¶

Install Dependencies:

pip install matplotlib torchvision  # torchvision for loading classic datasets

Example 1: Visualize MNIST Dataset Samples

import matplotlib.pyplot as plt
from torchvision import datasets, transforms

# Load MNIST dataset (handwritten digits 0-9)
transform = transforms.Compose([transforms.ToTensor()])
train_dataset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)

# Get the first 5 images and labels
images, labels = next(iter(train_dataset))
plt.figure(figsize=(10, 2))
for i in range(5):
    img = images[i].squeeze().numpy()  # Remove batch and channel dimensions
    plt.subplot(1, 5, i+1)
    plt.imshow(img, cmap='gray')  # Display as grayscale
    plt.title(f"Label: {labels[i].item()}")
    plt.axis('off')  # Hide axes
plt.show()

Example 2: Data Distribution Histogram

# Count label distribution in the training set
import numpy as np
labels_np = np.array([data[1] for data in train_dataset])
plt.figure(figsize=(8, 4))
plt.hist(labels_np, bins=10, edgecolor='black')
plt.xlabel('Digit')
plt.ylabel('Count')
plt.title('MNIST Label Distribution')
plt.show()

4. TensorBoard: “Monitoring Dashboard” for Training Process¶

Install TensorBoard:

pip install tensorboard

Core Functions:
- Visualize scalars (e.g., loss, accuracy) over training steps.
- Display model input images and computational graph structure.

Example: Visualizing Training Process

from torch.utils.tensorboard import SummaryWriter
import torch
import time

# Initialize the logger
writer = SummaryWriter(log_dir=f"runs/demo_{time.strftime('%Y%m%d_%H%M%S')}")

# Simulate training loop (using simple linear regression as an example)
model = torch.nn.Linear(1, 1)  # Input 1D, Output 1D
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
criterion = torch.nn.MSELoss()

for epoch in range(100):
    # Generate synthetic data: y = 2x + 3 + noise
    x = torch.randn(100, 1) * 5  # Input data
    y = 2 * x + 3 + torch.randn(100, 1) * 0.5  # True labels with noise
    optimizer.zero_grad()
    y_pred = model(x)
    loss = criterion(y_pred, y)
    loss.backward()
    optimizer.step()

    # Log scalars: training loss and validation loss (assumed validation loss = loss * 0.9)
    writer.add_scalar('Train Loss', loss.item(), epoch)
    writer.add_scalar('Val Loss', loss.item() * 0.9, epoch)

# Visualize model structure (requires defining input shape in advance)
writer.add_graph(model, torch.randn(1, 1))  # Input 1 sample with dimension 1
writer.close()

Start TensorBoard:
Run the command in the terminal: tensorboard --logdir=runs, then open http://localhost:6006 in your browser to view loss curves and model structures.

2. Model Evaluation: Quantifying the “True Performance” of the Model¶

1. Core Metrics for Classification Tasks¶

• Accuracy: Proportion of correctly predicted samples.
• Confusion Matrix: Shows prediction outcomes for each class (TP/FP/TN/FN).
• Precision/Recall: For specific classes (e.g., performance in “identifying digit 8”).

2. Core Metrics for Regression Tasks¶

• Mean Squared Error (MSE): Mean of squared differences between predictions and true values.
• Mean Absolute Error (MAE): Mean of absolute differences between predictions and true values.

3. PyTorch Implementation Example: Evaluating an MNIST Classification Model¶

Assuming a pre-trained simple CNN model, here is the evaluation code:

import torch
from sklearn.metrics import confusion_matrix
import seaborn as sns
import matplotlib.pyplot as plt

# 1. Load test dataset
test_dataset = datasets.MNIST(root='./data', train=False, transform=transforms.ToTensor())
test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=100, shuffle=False)

# 2. Model inference (get all predictions and labels)
model.eval()  # Disable training-specific layers (e.g., Dropout)
all_preds = []
all_labels = []
with torch.no_grad():  # Disable gradient computation to save memory
    for images, labels in test_loader:
        outputs = model(images)
        preds = torch.argmax(outputs, dim=1)  # Get the class with maximum probability
        all_preds.extend(preds.cpu().numpy())
        all_labels.extend(labels.numpy())

# 3. Calculate accuracy
correct = sum(np.array(all_preds) == np.array(all_labels))
accuracy = correct / len(all_labels)
print(f"Test Accuracy: {accuracy:.4f}")

# 4. Visualize confusion matrix
cm = confusion_matrix(all_labels, all_preds)
plt.figure(figsize=(10, 8))
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', 
            xticklabels=range(10), yticklabels=range(10))
plt.xlabel('Predicted Label')
plt.ylabel('True Label')
plt.title('Confusion Matrix')
plt.show()

3. Practical Summary: The Closed Loop from Data to Model¶

1. Data Visualization: Use Matplotlib to observe data distribution first, then TensorBoard to monitor the training process.
2. Model Evaluation: For classification tasks, use accuracy + confusion matrix; for regression tasks, use MSE + MAE.
3. Iterative Optimization: If the confusion matrix shows “8” and “9” are often confused, adjust data augmentation or model complexity.

4. Advanced Directions¶

• Advanced Visualization: Use TensorBoard’s add_image to visualize model-generated images (e.g., fake images generated by GANs).
• Dynamic Evaluation: Use the torchmetrics library to compute precision, F1-score, etc., in real time.
• Visualization Tool Comparison: Matplotlib is suitable for flexible plotting, while TensorBoard excels at training process monitoring.

By mastering these skills, you can more efficiently debug models and understand data, laying a solid foundation for building complex models later. When practicing, remember to experiment with different parameters and visualization combinations to gradually develop an “intuition” for models!