Why filtered: Duplicate tweet ID from multiple sources.
5. RetweetDeduplicationFilter
Prevents showing the same underlying post multiple times (as original or as different retweets).
// home-mixer/filters/retweet_deduplication_filter.rsmatch candidate.retweeted_tweet_id {
Some(retweeted_id) => {
// Remove if we've already seen this tweet (as original or retweet)if seen_tweet_ids.insert(retweeted_id) {
kept.push(candidate);
} else {
removed.push(candidate);
}
}
None => {
// Mark original tweet ID as seen
seen_tweet_ids.insert(candidate.tweet_id asu64);
kept.push(candidate);
}
}
Why filtered: Another version of this post (original or retweet) already included.
6. DedupConversationFilter
Keeps only the highest-scored post per conversation thread.
// home-mixer/filters/dedup_conversation_filter.rsfnget_conversation_id(candidate: &PostCandidate) ->u64 {
// Conversation root = minimum ancestor ID, or self if no ancestors
candidate
.ancestors
.iter()
.copied()
.min()
.unwrap_or(candidate.tweet_id asu64)
}
// Keeps highest score per conversation_id
Why filtered: Another post in same conversation thread has higher score.
7. SelfTweetFilter
Removes the user's own posts from their "For You" feed.
Why filtered: Missing author ID or empty tweet text (hydration failed).
12. IneligibleSubscriptionFilter
Removes subscription-only posts from authors the user isn't subscribed to.
// home-mixer/filters/ineligible_subscription_filter.rslet (kept, removed) = candidates.into_iter().partition(|candidate| {
match candidate.subscription_author_id {
Some(author_id) => subscribed_user_ids.contains(&author_id),
None => true, // Not a subscription post, keep it
}
});
Why filtered: Post requires subscription to author, user not subscribed.
Filter Result Structure
All filters return:
pubstructFilterResult<T> {
pub kept: Vec<T>, // Candidates that passedpub removed: Vec<T>, // Candidates that were filtered out
}
Conditional Filter Enabling
Some filters only run in certain contexts:
// PreviouslyServedPostsFilter only runs on paginationfnenable(&self, query: &ScoredPostsQuery) ->bool {
query.is_bottom_request
}
Bloom Filter Deduplication
PreviouslySeenPostsFilter uses Bloom filters for efficient "seen" tracking:
Client sends Bloom filter entries with request
Server reconstructs filters via BloomFilter::from_entry
Uses may_contain() (probabilistic) for fast lookup
Falls back to explicit seen_ids for definitive checks
Related Skills
/x-algo-pipeline - Where filters fit in the full pipeline
/x-algo-engagement - Understanding what data filters check against
