hormone-t5-emotion-layer

name

hormone-t5-emotion-layer

description

Hormone-inspired Emotion Layer for Transformer language models (HELT / HormoneT5). Biologically-inspired architecture augmenting transformers with a Hormone Emotion Block simulating the endocrine system's role in emotional processing. Six continuous hormone-like values computed via specialized per-hormone attention heads with orthogonally initialized learnable queries, temperature-scaled attention, and deep output projections. Emotional embedding modulates encoder hidden states for emotionally-appropriate response generation. Multi-objective training: sequence-to-sequence loss + hormone prediction loss with margin penalties + diversity regularization. Achieves 85%+ per-hormone accuracy. Use when: designing emotionally intelligent LLMs, affective computing, hormone-inspired neural architectures, bio-inspired emotion modeling in AI, transformer emotion augmentation. Keywords: HormoneT5, HELT, emotion layer, hormone attention, affective computing, endocrine-inspired AI, emotional language models, bio-inspired emotion.

HormoneT5: Hormone-inspired Emotion Layer for Transformers

arXiv: 2605.13858 | Reda & El-Metwally (April 2026)

Core Contribution

Augments transformer language models with a Hormone Emotion Block that simulates the human endocrine system's role in emotional processing, enabling continuous multi-dimensional emotion representation rather than discrete emotion classification.

Architecture

Hormone Emotion Block (HEB)

Input: Encoder hidden states (seq_len, d_model)
        ↓
┌─────────────────────────────────────────────┐
│         Hormone Emotion Block               │
│                                             │
│  6 Per-Hormone Attention Heads:            │
│  - Orthogonally initialized learnable queries│
│  - Temperature-scaled attention             │
│  - Deep output projections                  │
│                                             │
│  Output: 6 continuous hormone values        │
└─────────────────────────────────────────────┘
        ↓
Hormone values → Emotional embedding (d_model)
        ↓
Modulate encoder hidden states via addition/concatenation
        ↓
Emotionally-appropriate response generation