bio-applied-deep-learning-for-biology

name	bio-applied-deep-learning-for-biology
description	Train PyTorch CNN/LSTM/Transformer/VAE on DNA/protein sequences: one-hot encoding, motif filters, saliency. Use when classifying sequences, predicting TF binding sites, denoising scRNA-seq, or choosing DL vs ML.
tool_type	python
primary_tool	pytorch

Deep Learning for Biology

When to Use

• Classifying raw DNA/protein sequences (promoter vs non-promoter, TF binding site prediction) where hand-crafted k-mer features would lose information a CNN/Transformer can learn directly.
• Deciding between classical ML (random forest, SVM, logistic regression) and a neural network for a given biological dataset and sample size.
• Denoising or compressing high-dimensional gene expression / single-cell data with a variational autoencoder (VAE).
• Building the standard PyTorch train/eval loop for any tabular-, sequence-, or image-shaped biological input.
• Interpreting a trained model — extracting learned CNN filters as motifs, or computing saliency maps to see which positions drove a prediction.

Version Compatibility

• PyTorch >= 2.2 (CPU or CUDA build), Python >= 3.10
• scikit-learn >= 1.3 (train_test_split, StandardScaler), NumPy >= 1.24, pandas >= 2.0

Prerequisites

• pip install torch numpy pandas scikit-learn matplotlib (CPU wheel: --index-url https://download.pytorch.org/whl/cpu)
• Familiarity with classical ML workflow (see bio-applied-machine-learning-for-biology) — this skill assumes you already know when feature engineering + RF/SVM is the right call and are past that point.

Classical ML vs Deep Learning Decision Table

Criterion	Classical ML	Deep Learning
Sample size	100s-1000s	10,000+ (or transfer learning)
Feature engineering	Manual (k-mers, physicochemical)	Learned automatically
Input type	Tabular features	Raw sequences / images
Interpretability	High (feature importance)	Lower (needs SHAP/saliency/attention)
Training time	Minutes	Hours-days
Hardware	CPU	GPU recommended

Use classical ML when: tabular features, small dataset, interpretability required. Use DL when: raw sequence/image input, large dataset, hierarchical patterns (motifs within motifs), or a relevant pre-trained model exists for transfer learning.

Core Training Loop and Autograd

Goal: build, train, and evaluate a feedforward classifier — the pattern every architecture below reuses. Approach: define the model as an nn.Module, then run the standard 5-step loop (zero_grad → forward → loss → backward → step) inside model.train(), and switch to model.eval() + torch.no_grad() for evaluation.

import numpy as np
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')


class SimpleClassifier(nn.Module):
    """Feedforward network for binary classification from tabular features."""

    def __init__(self, input_dim, hidden_dim=64):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(input_dim, hidden_dim),
            nn.ReLU(),
            nn.Dropout(0.3),
            nn.Linear(hidden_dim, hidden_dim // 2),
            nn.ReLU(),
            nn.Linear(hidden_dim // 2, 1),
        )  # returns logits; pair with BCEWithLogitsLoss

    def forward(self, x):
        return self.net(x)


def train_classifier(X_train, y_train, input_dim, epochs=100, lr=1e-3, batch_size=32):
    """Standard PyTorch training loop. X_train/y_train are numpy arrays."""
    X_t = torch.FloatTensor(X_train)
    y_t = torch.FloatTensor(y_train).unsqueeze(1)
    loader = DataLoader(TensorDataset(X_t, y_t), batch_size=batch_size, shuffle=True)

    model = SimpleClassifier(input_dim).to(device)
    criterion = nn.BCEWithLogitsLoss()  
    optimizer = optim.Adam(model.parameters(), lr=lr)

    losses = []
     epoch  (epochs):
        model.train()
        epoch_loss = 
         X_batch, y_batch  loader:
            X_batch, y_batch = X_batch.to(device), y_batch.to(device)
            optimizer.zero_grad()              
            logits = model(X_batch)            
            loss = criterion(logits, y_batch)  
            loss.backward()                    
            optimizer.step()                   
            epoch_loss += loss.item() * X_batch.size()
        losses.append(epoch_loss / (loader.dataset))
     model, losses



 ():
    
    model.()
    logits = model(torch.FloatTensor(X_test).to(device))
    preds = (torch.sigmoid(logits) > ).().cpu().squeeze()
     (preds == torch.FloatTensor(y_test)).().mean().item()

One-Hot Encoding and 1D CNN for Motif Detection

Goal: classify raw DNA sequences (e.g., promoter vs non-promoter) without hand-crafted k-mer features. Approach: one-hot encode to (N, 4, L) (Conv1d expects (batch, channels, length)), then stack Conv1d → ReLU → MaxPool blocks so filters learn motifs and pooling gives position invariance.

def one_hot_encode(sequences, alphabet='ACGT'):
    """One-hot encode DNA/RNA sequences to shape (N, C, L) for Conv1d."""
    mapping = {c: i for i, c in enumerate(alphabet)}
    n, seq_len = len(sequences), len(sequences[0])
    encoded = np.zeros((n, len(alphabet), seq_len), dtype=np.float32)
    for i, seq in enumerate(sequences):
        for j, char in enumerate(seq[:seq_len]):
            if char in mapping:
                encoded[i, mapping[char], j] = 1.0
    return torch.FloatTensor(encoded)


class SequenceCNN(nn.Module):
    """1D CNN: Conv(detect motifs) -> ReLU -> MaxPool(position invariance) -> Dense."""

    def __init__(self, seq_length=200):
        super().__init__()
        self.conv_layers = nn.Sequential(
            nn.Conv1d(4, 32, kernel_size=8, padding=3),
            nn.ReLU(),
            nn.MaxPool1d(4),
            nn.Conv1d(32, 64, kernel_size=6, padding=2),
            nn.ReLU(),
            nn.AdaptiveAvgPool1d(1),  # global pooling -> fixed-size vector regardless of L
        )
        self.classifier = nn.Sequential(
            nn.Flatten(),
            nn.Linear(, ),
            nn.ReLU(),
            nn.Linear(, ),
        )  

     ():
         .classifier(.conv_layers(x))


 ():
    
     model.conv_layers[].weight.data.cpu().numpy()


 ():
    
    x = x.unsqueeze().clone().requires_grad_()
    model.()
    output = model(x.to(device))
    output.backward()
     x.grad.data.().squeeze().(dim=).cpu().numpy()  

VAE for Gene Expression Denoising

Goal: learn a low-dimensional latent space that separates cell types despite dropout noise in scRNA-seq-like data. Approach: encoder outputs mean/log-variance of a Gaussian latent, reparameterization trick samples z, decoder reconstructs input; loss = reconstruction + beta-weighted KL divergence.

class VAE(nn.Module):
    """Variational autoencoder for high-dimensional expression data."""

    def __init__(self, input_dim, hidden_dim=128, latent_dim=10):
        super().__init__()
        self.encoder = nn.Sequential(
            nn.Linear(input_dim, hidden_dim), nn.ReLU(),
            nn.Linear(hidden_dim, hidden_dim // 2), nn.ReLU(),
        )
        self.fc_mu = nn.Linear(hidden_dim // 2, latent_dim)
        self.fc_logvar = nn.Linear(hidden_dim // 2, latent_dim)
        self.decoder = nn.Sequential(
            nn.Linear(latent_dim, hidden_dim // 2), nn.ReLU(),
            nn.Linear(hidden_dim // 2, hidden_dim), nn.ReLU(),
            nn.Linear(hidden_dim, input_dim),
        )

    def encode(self, x):
        h = self.encoder(x)
        return self.fc_mu(h), self.fc_logvar(h)

    def reparameterize(self, mu, logvar):
        std = torch.exp(0.5 * logvar)
        return mu + std * torch.randn_like(std)  # reparam trick: keeps sampling differentiable

    def forward(self, x):
        mu, logvar = self.encode(x)
        z = self.reparameterize(mu, logvar)
        return self.decoder(z), mu, logvar


def vae_loss():
    
    recon_loss = nn.functional.mse_loss(recon, x, reduction=)
    kl_div = - * torch.( + logvar - mu.() - logvar.exp())
     recon_loss + beta * kl_div

Pitfalls

• model.eval() + torch.no_grad(): both required for inference — eval() disables dropout/batchnorm; no_grad() prevents gradient tracking (memory/speed).
• optimizer.zero_grad() before backward: forgetting this accumulates gradients across batches and produces wrong updates.
• Conv1d input shape: expects (batch, channels, length); one-hot DNA is (N, 4, L) — transpose from (N, L, 4) if your encoder produced the other layout.
• Loss/output layer mismatch: prefer BCEWithLogitsLoss on raw logits over Sigmoid() + BCELoss — the fused version is numerically stable near 0/1. Use CrossEntropyLoss (not Sigmoid+BCELoss) for multiclass.
• Class imbalance: use pos_weight in BCEWithLogitsLoss or a weighted sampler — biological datasets are often highly imbalanced (e.g., 1% binding sites vs background).
• Data leakage: split train/test BEFORE any scaling; fit StandardScaler on train only, then .transform() (not .fit_transform()) on test.
• Overfitting on small biological datasets: dropout, early stopping, weight decay, and data augmentation (e.g., reverse-complement for DNA) all help when samples are in the hundreds.
• GPU/CPU device mismatch: move model AND every tensor to the same device; a cryptic RuntimeError about tensors on different devices means you forgot a .to(device).

See Also

• bio-applied-machine-learning-for-biology — classical ML baseline (RF/SVM, feature engineering) to compare against before reaching for DL.
• bio-applied-structural-methods / structural-bioinformatics — for transformer-based structure prediction (AlphaFold-style) rather than sequence classification.
• bio-applied-single-cell-scanpy — upstream QC/normalization for the expression data fed into the VAE example.