// Explain the complete X recommendation algorithm pipeline. Use when users ask how posts are ranked, how the algorithm works, or want an overview of the recommendation system.
Reference for X algorithm engagement types and signals. Use when analyzing engagement metrics, action predictions, or understanding what signals the algorithm tracks.
Explain why posts get filtered from the X feed. Use when analyzing why a post was removed, not shown, or filtered out of recommendations.
Explain the Phoenix ML model architecture for X recommendations. Use when users ask about embeddings, transformers, how predictions work, or ML model details.
Calculate and explain X algorithm engagement scores. Use when analyzing post ranking, understanding score weights, engagement potential, or why one post ranks higher than another.
| name | x-algo-pipeline |
| description | Explain the complete X recommendation algorithm pipeline. Use when users ask how posts are ranked, how the algorithm works, or want an overview of the recommendation system. |
The X recommendation algorithm processes posts through an 8-stage pipeline to generate the "For You" feed. Each stage transforms, filters, or scores the candidate posts.
┌─────────────────────────────────────────────────────────────────────────────┐
│ X RECOMMENDATION PIPELINE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ User Request │
│ │ │
│ ▼ │
│ ┌─────────────┐ │
│ │ 1. Query │ Hydrate user features, action history, socialgraph │
│ │ Hydration │ │
│ └──────┬──────┘ │
│ ▼ │
│ ┌─────────────┐ Thunder (in-network) + Phoenix (out-of-network) │
│ │ 2. Sources │ In-network: Posts from followed accounts │
│ │ │ Out-of-network: ML retrieval from all posts │
│ └──────┬──────┘ │
│ ▼ │
│ ┌─────────────┐ │
│ │ 3. Candidate│ Fetch tweet text, author data, visibility status │
│ │ Hydration │ │
│ └──────┬──────┘ │
│ ▼ │
│ ┌─────────────┐ │
│ │ 4. Pre-Score│ Age, duplicates, safety, blocked authors │
│ │ Filtering │ │
│ └──────┬──────┘ │
│ ▼ │
│ ┌─────────────┐ Phoenix ML → WeightedScorer → AuthorDiversity → OON │
│ │ 5. Scoring │ Each scorer adds/adjusts candidate.score │
│ │ │ │
│ └──────┬──────┘ │
│ ▼ │
│ ┌─────────────┐ │
│ │ 6. Selection│ TopKScoreSelector: Keep top N by final score │
│ │ │ │
│ └──────┬──────┘ │
│ ▼ │
│ ┌─────────────┐ │
│ │ 7. Post- │ Conversation dedup, previously seen, keywords │
│ │ Filtering │ │
│ └──────┬──────┘ │
│ ▼ │
│ ┌─────────────┐ │
│ │ 8. Side │ Logging, analytics, impression tracking │
│ │ Effects │ │
│ └──────┬──────┘ │
│ ▼ │
│ Feed Response │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Enriches the request with user context:
Two candidate sources provide posts:
// home-mixer/sources/thunder_source.rs
// Posts from accounts the user follows
served_type: Some(pb::ServedType::ForYouInNetwork)
// home-mixer/sources/phoenix_source.rs
fn enable(&self, query: &ScoredPostsQuery) -> bool {
!query.in_network_only // Disabled for "Following" tab
}
served_type: Some(pb::ServedType::ForYouPhoenixRetrieval)
Fetches full post data:
Removes ineligible candidates before expensive ML scoring:
AgeFilter - Too oldDropDuplicatesFilter - Duplicate IDsVFFilter - Safety violationsAuthorSocialgraphFilter - Blocked/muted authorsCoreDataHydrationFilter - Missing dataIneligibleSubscriptionFilter - Subscription required// home-mixer/scorers/phoenix_scorer.rs
// Calls Phoenix ML to predict engagement probabilities
Produces phoenix_scores with 18 action probabilities.
// home-mixer/scorers/weighted_scorer.rs
// Combines probabilities into single score
weighted_score = Σ(weight × P(action))
Produces weighted_score from action predictions.
// home-mixer/scorers/author_diversity_scorer.rs
// Penalizes multiple posts from same author
multiplier = (1 - floor) × decay^position + floor
Adjusts scores to promote variety.
// home-mixer/scorers/oon_scorer.rs
// Adjusts out-of-network post scores
if !in_network: score *= OON_WEIGHT_FACTOR
Balances in-network vs out-of-network content.
// home-mixer/selectors/top_k_score_selector.rs
pub struct TopKScoreSelector;
impl Selector<ScoredPostsQuery, PostCandidate> for TopKScoreSelector {
fn score(&self, candidate: &PostCandidate) -> f64 {
candidate.score.unwrap_or(f64::NEG_INFINITY)
}
fn size(&self) -> Option<usize> {
Some(params::TOP_K_CANDIDATES_TO_SELECT)
}
}
Keeps top K posts by final score.
Fine-grained filtering after selection:
DedupConversationFilter - One post per conversationRetweetDeduplicationFilter - One version per underlying postPreviouslySeenPostsFilter - Remove seen postsPreviouslyServedPostsFilter - Remove from current sessionMutedKeywordFilter - User keyword mutesSelfTweetFilter - Remove own postsNon-blocking operations after response:
Candidates start with:
├── tweet_id, author_id (from Sources)
├── tweet_text, metadata (from Hydration)
├── phoenix_scores (from PhoenixScorer)
├── weighted_score (from WeightedScorer)
├── score (from AuthorDiversity + OON)
└── Final ranking by score
pub struct PostCandidate {
pub tweet_id: i64,
pub author_id: u64,
pub tweet_text: String,
pub in_reply_to_tweet_id: Option<u64>,
pub retweeted_tweet_id: Option<u64>,
pub retweeted_user_id: Option<u64>,
pub phoenix_scores: PhoenixScores, // ML predictions
pub weighted_score: Option<f64>, // After WeightedScorer
pub score: Option<f64>, // Final score
pub served_type: Option<ServedType>, // Source type
pub in_network: Option<bool>, // Following or not
pub ancestors: Vec<u64>, // Conversation context
pub video_duration_ms: Option<i32>, // For VQV eligibility
pub visibility_reason: Option<FilteredReason>,
pub subscription_author_id: Option<u64>,
// ...
}
| Tab | Thunder (In-Network) | Phoenix (Out-of-Network) |
|---|---|---|
| For You | Enabled | Enabled |
| Following | Enabled | Disabled |
/x-algo-scoring - Detailed scoring formula/x-algo-filters - All filter implementations/x-algo-engagement - Action types and signals/x-algo-ml - Phoenix ML model architecture