bioinf-jku/SNNs

Latest recorded source activity: May 12, 2026 at 14:16

SkillsMP catalog refreshed: October 26, 2026 at 11:37

skills collected: 1

GitHub stars: 1,589
GitHub forks: 196

Install with an AI assistant

Copy this prompt into the AI assistant you're using.

Follow https://skillsmp.com/skill-install/prompt.md to install Agent Skills from https://github.com/bioinf-jku/SNNs.

Repository README

Repository README source: README.md

Read the source README

Expand full README

# Self-Normalizing Networks Tutorials and implementations for Self-Normalizing Networks (SNNs) as suggested by Klambauer et al. ([arXiv pre-print](https://arxiv.org/pdf/1706.02515.pdf)). The repository contains TensorFlow/Keras scripts and PyTorch notebooks for demonstrating the main SNN recipe: - SELU activations - LeCun normal initialization - AlphaDropout when dropout is used ## Environment The current examples use TensorFlow 2.x/Keras and PyTorch. Create the Conda environment from the root environment file: ```bash conda env create -f environment.yml conda activate snn ``` The environment is intended for current TensorFlow and PyTorch versions. On Linux/NVIDIA systems, TensorFlow GPU dependencies are installed via the `tensorflow[and-cuda]` pip extra. ## Implementation Notes ### TensorFlow/Keras Keras provides SELU, LeCun normal initialization, and AlphaDropout: - [`tf.keras.activations.selu`](https://www.tensorflow.org/api_docs/python/tf/keras/activations/selu) - [`tf.keras.initializers.LecunNormal`](https://www.tensorflow.org/api_docs/python/tf/keras/initializers/LecunNormal) - [`tf.keras.layers.AlphaDropout`](https://www.tensorflow.org/api_docs/python/tf/keras/layers/AlphaDropout) ### PyTorch PyTorch provides SELU and AlphaDropout: - [`torch.nn.SELU`](https://pytorch.org/docs/stable/generated/torch.nn.SELU.html) - [`torch.nn.AlphaDropout`](https://pytorch.org/docs/stable/generated/torch.nn.AlphaDropout.html) For SELU networks, initialize linear and convolutional layers with: ```python torch.nn.init.kaiming_normal_(weight, mode="fan_in", nonlinearity="linear") ``` This corresponds to LeCun normal initialization because `fan_in` is used with a gain of 1. ## Tutorials ### TensorFlow 2.x / Keras - Multilayer Perceptron on MNIST ([python script](TF_2_x/MNIST-MLP-SELU.py)) - Convolutional Neural Network on MNIST ([python script](TF_2_x/MNIST-Conv-SELU.py)) - Convolutional Neural Network on CIFAR10 ([python script](TF_2_x/CIFAR10-Conv-SELU.py)) ### PyTorch - Multilayer Perceptron on MNIST ([notebook](Pytorch/SelfNormalizingNetworks_MLP_MNIST.ipynb)) - Convolutional Neural Network on MNIST ([notebook](Pytorch/SelfNormalizingNetworks_CNN_MNIST.ipynb)) - Convolutional Neural Network on CIFAR10 ([notebook](Pytorch/SelfNormalizingNetworks_CNN_CIFAR10.ipynb)) ## Further Material ### Figure 1 Notebooks and code to produce Figure 1 are provided in [figure1](figure1/). This material builds on top of the [biutils](https://github.com/untom/biutils) package. ### Calculations and Numeric Checks Mathematica notebooks for calculations and numeric checks of the theorems are provided here: - [Mathematica notebook](Calculations/SELU_calculations.nb) - [Mathematica PDF](Calculations/SELU_calculations.pdf) ### UCI, Tox21, and HTRU2 Data Sets - [UCI](http://persoal.citius.usc.es/manuel.fernandez.delgado/papers/jmlr/data.tar.gz) - [Tox21](http://bioinf.jku.at/research/DeepTox/tox21.zip) - [HTRU2](https://archive.ics.uci.edu/ml/machine-learning-databases/00372/HTRU2.zip)

Loading the repository README…

Skills in this repository

Data Scientists

1 occupation category · 100% classified

skill

occupation

description

updated

self-normalizing-networks

Data Scientists

Use this skill when working in the Self-Normalizing Networks tutorial repository, especially for TensorFlow, PyTorch, Conda environment, or SELU/AlphaDropout implementation tasks.

2026-05-12

Showing 1 of 1 collected skill.