| name | wikimedia-eventstreams |
| description | Consume real-time streams of Wikimedia events (edits, page creations, deletions, moves, log entries) via Server-Sent Events (SSE). Covers the EventStreams HTTP service, stream schemas, client libraries (Python/JS/curl), filtering, historical replay, canary handling, auto-reconnect, and building live dashboards, patrol monitors, and cross-wiki trackers |
| license | MIT |
| compatibility | opencode |
| depends_on | ["wikimedia-api-access","pywikibot"] |
| skill_discovery_hints | [{"keywords":["real-time","stream","SSE","EventStreams","live","recent change","recentchange"]},{"keywords":["new page","page creation","patrol","monitor","watch","live feed"]},{"keywords":["edit event","revision create","log event","streaming"]}] |
| last_verified | "2026-06-10T00:00:00.000Z" |
⚠️ User-Agent required: All examples in this skill connect to stream.wikimedia.org. Requests without a descriptive User-Agent header may be throttled. See the wikimedia-api-access skill for the correct format and rate-limiting patterns. Before writing code, load that skill for the required User-Agent boilerplate.
💡 Related skills for common workflows:
- wikimedia-ml-services — Score events from the stream (revert risk, goodfaith, damaging) for real-time patrol and vandalism detection
- wikimedia-diffs — Fetch and classify diffs for edits detected in the stream (addition-heavy, deletion-heavy, replacement patterns)
- wikipedia-reference-verifiability — Check PageTriage review status for new pages detected in the stream
- wikimedia-api-access — Required User-Agent headers and rate limiting for any follow-up API calls
Table of Contents
- What EventStreams Is
- Where It Fits in the API Universe
- Stream Catalog
- Quick Start
- Event Schema Reference
- Client-Side Filtering
- Canary Events — Mandatory Handling
- Connection Reliability & Auto-Reconnect
- Historical Replay
- Python Clients in Depth
- JavaScript Clients in Depth
- Command-Line & Browser
- Building Real-Time Tools
- When NOT to Use EventStreams
- Reference Links
What EventStreams Is
EventStreams is a real-time event streaming service operated by the Wikimedia Foundation. Unlike every other Wikimedia API (which follows a request-response pattern), EventStreams uses Server-Sent Events (SSE) — the server pushes structured JSON events to you over HTTP as they happen.
Key facts:
- Backend: Apache Kafka clusters (hundreds of thousands of messages/second across multiple data centers)
- Protocol: Standard HTTP SSE (
text/event-stream) via chunked transfer encoding
- Supersedes: RCStream (the old Socket.IO-based service), and may eventually replace
irc.wikimedia.org
- Read-only: Events flow server → client. You cannot publish events through EventStreams.
- External use only: WMF production services should consume Kafka directly — the public HTTP service is for external tool developers.
How SSE Works
SSE is a standard web protocol (part of HTML5) where a server opens a long-lived HTTP connection and writes lines of data as events occur:
event: message
id: [{"topic":"eqiad.mediawiki.recentchange","partition":0,"timestamp":1532031066001},...]
data: {"event": "data", "is": "here"}
Each event has:
event: message for data, error for errorsid: JSON array of Kafka topic/partition/timestamp metadata (used for reconnection)data: The actual JSON payload (the event)
The connection stays open until the client disconnects or a 15-minute server-side timeout kicks in. Good clients auto-reconnect using the Last-Event-ID header to resume where they left off.
Where It Fits in the API Universe
EventStreams occupies a unique position — it's the only push-based, real-time data source in the Wikimedia ecosystem.
| Data Source | Paradigm | Latency | Use Case |
|---|
| REST API v1 | Request-response (HTTP GET) | Minutes old | Read page content, summaries |
Action API (api.php) | Request-response | Minutes old | Query, edit, upload, delete |
| EventStreams | Push (SSE / Kafka) | Sub-second | Real-time monitoring, live dashboards |
| SQL Replicas | Poll (SQL queries) | Hours–days old | Deep analysis, JOINs, bulk stats |
| Dumps | Batch download | Days–months old | Offline/ML training, full corpus |
| SPARQL | Request-response | Minutes old | Graph queries over Wikidata |
Complementary to existing skills:
| Skill | How EventStreams relates |
|---|
wikimedia-diffs | Diffs are static comparisons of two revisions. EventStreams gives you the live firehose of changes — pipe events into diff-analysis for real-time monitoring. |
wikipedia-edit-history | History is retrospective (looking at past edits). EventStreams is prospective (watching edits happen in real-time). |
pywikibot | Pywikibot's comms.eventstreams.EventStreams is the most Pythonic way to consume EventStreams. See the Python Clients section. |
wikimedia-api-access | Sets the User-Agent and rate-limiting patterns for all HTTP-based access, including EventStreams. |
Stream Catalog
These are all available streams as of v2 of the EventStreams HTTP service. Streams are grouped by what they emit.
MediaWiki Core Streams
| Canonical Stream | Aliases | What It Emits | Schema |
|---|
mediawiki.recentchange | recentchange | Every edit, page creation, log entry, categorize action, or external change — the main firehose | /mediawiki/recentchange |
mediawiki.revision-create | revision-create | Every new revision (page saves). Does NOT include log events or categorizations. | /mediawiki/revision/create |
mediawiki.page-create | page-create | First revision of a newly created page (subset of revision-create) | /mediawiki/revision/create |
mediawiki.page-delete | page-delete | Page deletions | /mediawiki/page/delete |
mediawiki.page-undelete | page-undelete | Page restorations | /mediawiki/page/undelete |
mediawiki.page-move | page-move | Page renames/moves | /mediawiki/page/move |
mediawiki.page-links-change | page-links-change | When a page's outgoing links change (triggers update of WhatLinksHere) | /mediawiki/page/links-change |
mediawiki.page-properties-change | page-properties-change | When page properties change | /mediawiki/page/properties-change |
mediawiki.page_change.v1 | — | Unified page change event | /mediawiki/page/change |
mediawiki.revision-tags-change | — | When revision tags are added or removed | /mediawiki/revision/tags-change |
mediawiki.revision-visibility-change | — | When revision visibility is suppressed or restored | /mediawiki/revision/visibility-change |
Machine Learning / Prediction Streams
| Canonical Stream | What It Emits | Schema |
|---|
mediawiki.page_outlink_topic_prediction_change.v1 | Topic predictions for outlinks change (MOR/ORES model) | /mediawiki/page/prediction_classification_change |
mediawiki.page_revert_risk_prediction_change.v1 | Revert-risk scores change (ORES model) | /mediawiki/page/prediction_classification_change |
mediawiki.page_revert_risk_multilingual_prediction_change.v1 | Multilingual revert-risk predictions change | /mediawiki/page/prediction_classification_change |
Wikidata / Wikibase Streams
| Canonical Stream | What It Emits |
|---|
rdf-streaming-updater.mutation.v2 | Wikidata RDF changes (consumed by the Wikidata Query Service updater) |
rdf-streaming-updater.mutation-main.v2 | Wikidata main namespace entity changes |
rdf-streaming-updater.mutation-scholarly.v2 | Scholarly entity changes |
mediainfo-streaming-updater.mutation.v2 | Commons media info entity changes (structured data on Commons) |
Test Stream
| Canonical Stream | Aliases | What It Emits |
|---|
eventgate-main.test.event | test | One artificial event per minute for testing connectivity |
Composite Streams
To subscribe to multiple streams simultaneously, join them with commas in the URL:
https://stream.wikimedia.org/v2/stream/recentchange,page-create,page-delete
The {streams} path parameter in the Swagger spec accepts comma-separated stream names. Each event carries its own stream name in meta.stream.
Getting the Live List
The list of streams changes over time. To see the current catalog:
open https://stream.wikimedia.org/?doc
curl -s https://stream.wikimedia.org/?spec | jq '.paths | keys'
Quick Start
1. curl + jq (simplest, no dependencies)
curl -s -H 'Accept: application/json' \
-H 'User-Agent: MyBot/1.0 (me@example.com)' \
https://stream.wikimedia.org/v2/stream/recentchange
With Accept: application/json, EventStreams sends newline-delimited JSON instead of SSE format, making it pipeable to jq:
curl -sN -H 'Accept: application/json' \
-H 'User-Agent: MyBot/1.0 (me@example.com)' \
https://stream.wikimedia.org/v2/stream/recentchange \
| jq 'select(.wiki == "enwiki" and .type == "edit") | {user, title}'
Note: The -sN (silent, no-buffering) flag is important — without it curl may buffer output.
2. Python (with requests-sse)
import json
from requests_sse import EventSource
url = 'https://stream.wikimedia.org/v2/stream/recentchange'
headers = {'User-Agent': 'MyBot/1.0 (me@example.com)'}
with EventSource(url, headers=headers) as stream:
for event in stream:
if event.type == 'message':
change = json.loads(event.data)
if change['meta']['domain'] == 'canary':
continue
print(f"{change['user']} edited {change['title']}")
3. Python (with Pywikibot — recommended)
from pywikibot.comms.eventstreams import EventStreams
stream = EventStreams(streams='recentchange')
stream.register_filter(server_name='en.wikipedia.org', type='edit')
for change in stream:
print(f"{change['user']} edited {change['title']}")
4. JavaScript (browser)
const url = 'https://stream.wikimedia.org/v2/stream/recentchange';
const eventSource = new EventSource(url);
eventSource.onmessage = (event) => {
const data = JSON.parse(event.data);
const eventTime = new Date(data.timestamp * 1000);
if (data.meta.domain === 'canary') return;
if (data.server_name === 'en.wikipedia.org') {
console.log(`${data.user} edited ${data.title} @ ${eventTime.toISOString()}`);
}
};
5. Node.js (with wikimedia-streams)
import WikimediaStream from 'wikimedia-streams';
const stream = new WikimediaStream('recentchange');
stream
.filter('mediawiki.recentchange')
.all({ wiki: 'enwiki' })
.on('recentchange', (data) => {
console.log(`${data.user} edited ${data.title}`);
});
Event Schema Reference
The recentchange Event (the main stream)
This is the most commonly consumed stream. All fields from Manual:RCFeed are preserved. The schema version is mediawiki/recentchange/1.0.1.
{
"$schema": "/mediawiki/recentchange/1.0.1",
"meta": {
"dt": "2024-06-04T21:00:00Z",
"id": "uuid-string",
"stream": "mediawiki.recentchange",
"domain": "en.wikipedia.org",
"request_id": "uuid",
"uri": "https://en.wikipedia.org/wiki/Albert_Einstein",
"partition": 0,
"offset": 4000957901
},
"title": "Albert Einstein",
"type": "edit",
"bot": false,
"comment": "/* lede */ added citation",
"id": 123456789,
"length": {
"new": 84721,
"old": 84650
},
"minor": false,
"namespace": 0,
"patrolled": true,
"revision": {
"new": 123456789,
"old": 123456788
},
"server_name": "en.wikipedia.org",
"server_script_path": "/w",
"server_url": "https://en.wikipedia.org",
"timestamp": 1717540800,
"user": "ExampleEditor",
"wiki": "enwiki",
"parsedcomment": "<p>comment</p>",
"log_action": null,
"log_action_comment": null,
"log_id": null,
"log_params": null,
"log_type": null
}
The type field values:
| Value | Meaning |
|---|
"edit" | An existing page was edited |
"new" | A new page was created |
"log" | A log entry was made (delete, move, protect, block, upload, etc.) |
"categorize" | A page was added/removed from a category |
"external" | An external change was recorded |
The wiki field is the internal database name — the key identifier for filtering by wiki:
| Wiki | wiki value | server_name |
|---|
| English Wikipedia | enwiki | en.wikipedia.org |
| Wikimedia Commons | commonswiki | commons.wikimedia.org |
| Wikidata | wikidatawiki | www.wikidata.org |
| German Wikipedia | dewiki | de.wikipedia.org |
| French Wikipedia | frwiki | fr.wikipedia.org |
Notable Fields in Other Streams
revision-create (schema: /mediawiki/revision/create): Similar to recentchange but only for revision creations. Has page_id, rev_id, page_title, page_namespace, user, comment, bot flags. Does NOT include log events or categorize events.
page-delete (schema: /mediawiki/page/delete): Has page_id, page_title, page_namespace, rev_id of the last revision before deletion, suppressed boolean (for oversight deletions), user, and comment.
page-move (schema: /mediawiki/page/move): Has page_id, page_title (old title), page_namespace, new_title, new_namespace, user, comment, suppressed_redirect boolean.
page-links-change (schema: /mediawiki/page/links-change): Has page_id, page_title, page_namespace, added_links array, removed_links array. Useful for tracking what links to what in real time.
Client-Side Filtering
EventStreams has no server-side filtering. You must filter on the client side. The most common patterns:
By Wiki (most common)
wiki = 'enwiki'
with EventSource(url, headers=headers) as stream:
for event in stream:
if event.type == 'message':
change = json.loads(event.data)
if change['meta']['domain'] == 'canary':
continue
if change['wiki'] == wiki:
print(change['title'])
By Wiki Server Name
if change['server_name'] == 'en.wikipedia.org':
...
By Event Type
if change['type'] == 'edit':
...
By Namespace
if change['namespace'] == 0:
...
if change['namespace'] % 2 == 1:
...
By Bot Flag
if not change.get('bot', False):
...
if change.get('bot', False):
...
By User
if change.get('user') == 'VandalBot123':
alert('Suspicious account active!')
By Page Title Pattern
import fnmatch
if fnmatch.fnmatch(change.get('title', ''), 'Albert*'):
...
Combining Filters (real-world pattern)
def relevant(change):
"""Track human edits to main-namespace enwiki articles."""
if change['meta']['domain'] == 'canary':
return False
if change['wiki'] != 'enwiki':
return False
if change['namespace'] != 0:
return False
if change.get('bot', False):
return False
if change['type'] not in ('edit', 'new'):
return False
return True
Using Pywikibot's Built-In Filtering
Pywikibot's EventStreams class provides a declarative filtering API:
stream = EventStreams(streams='recentchange')
stream.register_filter(
server_name='en.wikipedia.org',
type='edit',
bot=False
)
Canary Events — Mandatory Handling
The Wikimedia Data Engineering team injects artificial "canary" events into every stream multiple times per hour. These events exist to distinguish between a broken stream and an empty one (monitoring).
As a consumer, you MUST discard all canary events. They are not real changes. The simplest and fastest check:
if change['meta']['domain'] == 'canary':
continue
Pywikibot's EventStreams class handles this automatically — canary events are dropped before they reach your code. If you use a raw SSE client or curl, you must check manually.
Canary events copy their fields from the example event in the stream's JSON schema. For recentchange, they look like a real edit but with meta.domain: "canary".
Connection Reliability & Auto-Reconnect
15-Minute Timeout
WMF's HTTP connection termination layer enforces a 15-minute timeout on idle SSE connections. A well-behaved client must:
- Detect the disconnection
- Reconnect to the stream URL
- Send a
Last-Event-ID header to resume from where it left off
Standard SSE Reconnection (browser EventSource)
Browsers handle this automatically. EventSource reconnects on error and sends the Last-Event-ID header with the last received event ID. This is transparent to your code.
Python Reconnection (requests-sse)
import json
import time
from requests_sse import EventSource
url = 'https://stream.wikimedia.org/v2/stream/recentchange'
headers = {'User-Agent': 'MyBot/1.0 (me@example.com)'}
last_id = None
max_retries = 10
retry_delay = 1
for attempt in range(max_retries):
try:
with EventSource(url, headers=headers, last_event_id=last_id) as stream:
for event in stream:
if event.type == 'message':
change = json.loads(event.data)
if change['meta']['domain'] == 'canary':
continue
print(f"{change['user']} edited {change['title']}")
last_id = event.last_event_id
retry_delay = 1
except (ConnectionError, TimeoutError) as e:
print(f'Connection lost ({e}), reconnecting in {retry_delay}s...')
time.sleep(retry_delay)
retry_delay = min(retry_delay * 2, 60)
Pywikibot Auto-Reconnection
Pywikibot's EventStreams handles reconnection transparently — it uses requests-sse under the hood with the last_event_id parameter and automatically retries on connection errors.
Detecting Stale Connections
If you receive no events for 10 minutes (but the connection is still open), consider proactively closing and reconnecting:
import time
last_event_time = time.time()
timeout_threshold = 600
with EventSource(url, headers=headers) as stream:
for event in stream:
if event.type == 'message':
last_event_time = time.time()
if time.time() - last_event_time > timeout_threshold:
print('Stream appears stale, reconnecting...')
break
Historical Replay
Since June 2018, EventStreams supports timestamp-based historical consumption. This lets you replay events from a past point in time.
Using the since Query Parameter
Add ?since=<timestamp> to the stream URL:
curl -sN -H 'Accept: application/json' \
'https://stream.wikimedia.org/v2/stream/recentchange?since=2026-06-01T00:00:00Z'
The since parameter accepts anything parseable by JavaScript Date.parse() — ISO-8601 datetimes work best.
Using Last-Event-ID for Manual Offset
For precise control, set Last-Event-ID to a JSON array specifying topic, partition, and offset or timestamp:
[{"topic": "eqiad.mediawiki.recentchange", "partition": 0, "offset": 1234567}]
In Python:
with EventSource(url, headers=headers, last_event_id='[{"topic":"eqiad.mediawiki.recentchange","partition":0,"offset":1234567}]') as stream:
...
Retention Limits
Events are not kept indefinitely. Depending on the stream's Kafka topic configuration:
| Stream type | Typical retention |
|---|
mediawiki.recentchange | ~7–31 days |
mediawiki.revision-create | ~7–31 days |
rdf-streaming-updater.mutation.v2 | May vary (consumed by WDQS) |
Check with the Data Engineering team or the stream's configuration for exact retention.
Rate Limiting on Replay
Be kind when requesting historical data. Replaying many days of events can be compute-intensive for the service. Only request the minimum data you need. If you need bulk historical analysis, use API queries, SQL replicas, or XML dumps instead.
Python Clients in Depth
Option 1: requests-sse (low-level, no dependencies beyond requests)
pip install requests-sse
Full example with all features:
import json
import time
from requests_sse import EventSource
CONFIG = {
'url': 'https://stream.wikimedia.org/v2/stream/recentchange',
'headers': {'User-Agent': 'MyBot/1.0 (me@example.com)'},
}
def process_change(change):
"""Handle a single recentchange event."""
if change.get('meta', {}).get('domain') == 'canary':
return
if (change.get('wiki') == 'enwiki'
and change.get('namespace') == 0
and change.get('type') == 'edit'
and not change.get('bot')):
print(f"[{change['meta']['dt']}] "
f"{change['user']} edited {change['title']} "
f"(+{change.get('length', {}).get('new', 0) - change.get('length', {}).get('old', 0)} bytes)")
def main():
"""Run the EventStreams consumer with auto-reconnect."""
last_id = None
retry_delay = 1
while True:
try:
with EventSource(
CONFIG['url'],
headers=CONFIG['headers'],
last_event_id=last_id
) as stream:
print('Connected to EventStreams')
for event in stream:
if event.type == 'message':
change = json.loads(event.data)
process_change(change)
last_id = event.last_event_id
retry_delay = 1
except (ConnectionError, TimeoutError, Exception) as e:
print(f'Disconnected: {e}. Reconnecting in {retry_delay}s...')
time.sleep(retry_delay)
retry_delay = min(retry_delay * 2, 30)
if __name__ == '__main__':
main()
Option 2: Pywikibot EventStreams (recommended for Python)
from pywikibot.comms.eventstreams import EventStreams
This is the most feature-rich Python client:
stream = EventStreams(
streams='recentchange',
since='2026-06-01T00:00:00Z',
)
stream.register_filter(
server_name='en.wikipedia.org',
type='edit',
bot=False
)
for change in stream:
print(f"{change['user']} edited {change['title']}")
stream.set_maximum_items(10)
for change in stream:
print(change['title'])
Pywikibot's EventStreams advantages:
- Automatic canary filtering ✓
- Automatic reconnection ✓
- Declarative filter API ✓
- Composite streams from a list ✓
- Generator interface (works with
next(), for, etc.) ✓
- Historical replay via
since= ✓
- Uses Pywikibot's configured User-Agent ✓
stream = EventStreams(streams='recentchange')
stream.register_filter(
server_name=['en.wikipedia.org', 'de.wikipedia.org'],
type='edit',
bot=False
)
for change in stream:
print(f"[{change['wiki']}] {change['user']} → {change['title']}")
Option 3: Asynchronous (aiohttp-sse-client)
For high-throughput applications that need async IO:
import json
import aiohttp
from aiohttp_sse_client import client as sse_client
async def listen():
async with aiohttp.ClientSession(
headers={'User-Agent': 'MyBot/1.0 (me@example.com)'}
) as session:
async with sse_client.EventSource(
'https://stream.wikimedia.org/v2/stream/recentchange',
session=session
) as event_source:
async for event in event_source:
if event.type == 'message':
change = json.loads(event.data)
if change.get('meta', {}).get('domain') == 'canary':
continue
print(f"{change['user']} edited {change['title']}")
JavaScript Clients in Depth
Browser — EventSource API
Built into every modern browser. Zero dependencies.
<!DOCTYPE html>
<html>
<head>
<title>Live Wikipedia Edits</title>
</head>
<body>
<ul id="edits"></ul>
<script>
const url = 'https://stream.wikimedia.org/v2/stream/recentchange';
const es = new EventSource(url);
const list = document.getElementById('edits');
es.onmessage = (event) => {
const data = JSON.parse(event.data);
if (data.meta.domain === 'canary') return;
if (data.server_name === 'en.wikipedia.org' && data.type === 'edit') {
const li = document.createElement('li');
li.textContent = `${data.user} edited ${data.title}`;
list.prepend(li);
while (list.children.length > 100) {
list.removeChild(list.lastChild);
}
}
};
es.onerror = () => {
console.error('Connection lost, browser will auto-reconnect');
};
</script>
</body>
</html>
Node.js — wikimedia-streams package
npm install wikimedia-streams
This library provides type-safe event handlers, fluent filtering, and automatic canary event filtering:
import WikimediaStream from 'wikimedia-streams';
const stream = new WikimediaStream('recentchange');
stream
.filter('mediawiki.recentchange')
.all({ wiki: 'enwiki' })
.on('recentchange', (data) => {
console.log(`[${data.meta.dt}] ${data.user} → ${data.title}`);
});
stream.on('open', () => console.log('Connected'));
stream.on('error', (err) => console.error('Error:', err));
Node.js — eventsource package (standard EventSource polyfill)
npm install eventsource
⚠️ Import note: eventsource v3+ exports EventSource as a named export,
not a default export. Use destructured import in both ESM and CommonJS.
The constructor accepts a second argument { headers, rejectUnauthorized }
for configuration — always pass a User-Agent header.
ESM:
import { EventSource } from 'eventsource';
CommonJS:
const { EventSource } = require('eventsource');
Full example with User-Agent (both ESM and CJS):
import { EventSource } from 'eventsource';
const url = 'https://stream.wikimedia.org/v2/stream/recentchange';
const es = new EventSource(url, {
headers: {
'User-Agent': 'MyBot/1.0 (me@example.com) EventStreamsExample'
}
});
es.onmessage = (event) => {
const data = JSON.parse(event.data);
if (data.meta?.domain === 'canary') return;
const eventTime = new Date(data.timestamp * 1000);
if (data.wiki === 'enwiki') {
console.log(`${data.user} edited ${data.title} @ ${eventTime.toISOString()}`);
}
};
Command-Line & Browser
curl + jq — one-liner for inspection
curl -sN -H 'Accept: application/json' \
https://stream.wikimedia.org/v2/stream/recentchange \
| jq 'select(.wiki == "enwiki" and .type == "edit" and .bot == false) | {user, title, comment}'
Browser Swagger UI — interactive exploration
open https://stream.wikimedia.org/?doc
The Swagger UI lets you:
- See all available streams with descriptions
- Click "Try it out" to connect to a stream directly in the browser
- View sample event data
- Discover schemas via the linked schema registry
Browser — Codepen demos
These are live, editable demos you can run in the browser:
Building Real-Time Tools
EventStreams is the foundation for a wide range of real-time applications. Here are proven patterns:
Live Edit Counter
from pywikibot.comms.eventstreams import EventStreams
from collections import Counter
import time
stream = EventStreams(streams='recentchange')
stream.register_filter(server_name='en.wikipedia.org')
counts = Counter()
window_start = time.time()
for change in stream:
counts[change['user']] += 1
elapsed = time.time() - window_start
if elapsed >= 30:
print(f"\nTop editors in last {elapsed:.0f}s:")
for user, count in counts.most_common(10):
print(f" {user}: {count}")
counts.clear()
window_start = time.time()
Cross-Wiki Patrol Monitor
Track edits by the same user across multiple wikis simultaneously:
from pywikibot.comms.eventstreams import EventStreams
from collections import defaultdict
stream = EventStreams(streams='recentchange')
user_edits = defaultdict(list)
for change in stream:
if change['meta']['domain'] == 'canary':
continue
user = change['user']
wiki = change['wiki']
title = change['title']
user_edits[user].append((wiki, title, change['meta']['dt']))
recent = [e for e in user_edits[user]
if (time.time() - parse_timestamp(e[2])) < 60]
if len(set(w for w, _, _ in recent)) >= 3:
print(f"⚠️ Cross-wiki activity: {user}")
for w, t, dt in recent:
print(f" {w}: {t} @ {dt}")
Wikidata Batch Edit Detector
Detect when a single user makes many edits to Wikidata items in a short period (useful for spotting automated imports or vandalism):
from pywikibot.comms.eventstreams import EventStreams
from collections import Counter
import time
stream = EventStreams(streams='recentchange')
stream.register_filter(server_name='www.wikidata.org')
user_counter = Counter()
window = 60
for change in stream:
user = change['user']
user_counter[user] += 1
if user_counter[user] > 50:
print(f"🚨 High-volume editor: {user} "
f"({user_counter[user]} edits/min)")
Live Edit Feed for a Specific Page or Category
from pywikibot.comms.eventstreams import EventStreams
from fnmatch import fnmatch
physics_pages = [
'Physics',
'Quantum mechanics',
'Theory of relativity',
'Thermodynamics',
]
stream = EventStreams(streams='recentchange')
stream.register_filter(server_name='en.wikipedia.org')
for change in stream:
title = change.get('title', '')
if any(fnmatch(title, f'{pattern}*') for pattern in physics_pages):
print(f"🔬 {change['user']} edited {title}")
Commons File Upload Monitor
Watch new files as they're uploaded to Wikimedia Commons — useful for patrol,
gallery curation, and batch-upload monitoring. This requires a distinct filter
combo (type=log, log_type=upload, log_action=upload) not used in other
real-time examples:
from pywikibot.comms.eventstreams import EventStreams
stream = EventStreams(streams='recentchange')
for change in stream:
if (change.get('wiki') == 'commonswiki'
and change.get('type') == 'log'
and change.get('log_type') == 'upload'
and change.get('log_action') == 'upload'):
title = change.get('title', '')
user = change.get('user', '')
print(f"📤 {user} uploaded {title}")
Note: log_action distinguishes new uploads ('upload') from overwrites
('overwrite'). Filter for log_action='upload' to only see brand-new files.
The title field includes the File: prefix namespace.
For a real-time gallery that displays the uploaded images, combine this event
stream with the thumbnail methods from the
wikimedia-commons-thumbnails skill
— specifically the Special:FilePath approach which needs only the filename
and no extra API calls.
SSE Relay: EventStreams → Your Server → Browser Clients
The most common architecture for production real-time web apps: a single
server-side connection to EventStreams fans out filtered events to many
browser SSE clients. This avoids direct browser connections to
stream.wikimedia.org (CORS concerns) and centralizes filtering/transformation.
Ready-to-run implementations are provided in this skill's scripts/ directory:
Node.js quick start:
cd scripts
npm install express eventsource
node sse-relay-server.js
Python quick start:
cd scripts
pip install fastapi uvicorn aiohttp aiohttp-sse-client
python sse-relay-server.py
Key benefits of the relay pattern:
- One upstream EventStreams connection serves many browser clients
- Filter, transform, and enrich events before forwarding
- No direct browser dependency on
stream.wikimedia.org
- Reconnection logic is centralized (handled by the
eventsource library)
- Can add authentication, rate-limiting, or event persistence at the relay
Real-Time Dashboard Widget (HTML/JS)
Build a live counter that shows edits per minute, latest edits, or bot vs. human ratio. See the example at codepen.io/Krinkle/pen/BwEKgW.
When NOT to Use EventStreams
| Scenario | Better Approach |
|---|
| WMF production service | Consume Kafka directly (the public HTTP service is for external tools only) |
| Historical analysis | SQL replicas, XML dumps, or Action API queries — EventStreams only retains 7–31 days |
| Single page read | REST API v1 GET /page/{title}/html — simpler and stateless |
| Bulk data export | Download XML dumps — gigabytes of data for offline processing |
| Complex queries across wikis | SQL replicas on Toolforge — JOINs are impossible with a stream |
| Need server-side filtering | Use the Action API list=recentchanges with rcprop= parameters (but you lose real-time speed) |
| One-off data check | curl the Action API — no long-lived connection needed |
| High-frequency, low-effort tasks | EventStreams is for continuous monitoring, not batch jobs |
| Offline / batch ML training | XML dumps — you need the full revision history, not a real-time sample |
Rule of thumb: EventStreams is for real-time visibility — live dashboards, patrol monitors, cross-wiki trackers, and tools that need to react within seconds. If your data need is retrospective, bulk, or infrequent, use the request-response APIs or database replicas.
Reference Links
