| name | pywikibot |
| description | Use Pywikibot — the Python library and CLI tool suite for automating work on MediaWiki sites (Wikipedia, Wikidata, Commons, and any other MediaWiki wiki). Covers installation, configuration, the core object model, page generators, the bot framework, built-in scripts, and Wikidata/Commons integration |
| license | MIT |
| compatibility | opencode |
| depends_on | ["wikimedia-api-access","wikidata"] |
| skill_discovery_hints | [{"keywords":["Pywikibot","bot","automated editing","page generator","pwb.py","MediaWiki bot"]},{"keywords":["bulk edit","category operations","template harvesting","archive bot","replace.py"]}] |
| last_verified | "2026-06-10T00:00:00.000Z" |
⚠️ User-Agent required: Pywikibot sets User-Agent automatically based on your user-config.py settings, but any direct curl/requests calls in this skill still need a proper header. See the wikimedia-api-access skill for format and rate-limiting patterns. Before writing custom code alongside Pywikibot, load that skill for the required User-Agent boilerplate.
Table of Contents
- What Pywikibot Is
- Where It Fits in the API Universe
- Installation & Setup
- Core Object Model
- Page Generators — The Superpower
- Bot Framework
- Key Built-in Scripts Catalog
- Wikidata & Wikibase Integration
- Commons / File Uploads
- Running on Toolforge & PAWS
- One-of-a-Kind Capabilities
- When NOT to Use Pywikibot
- Security & Bot Credentials
- Troubleshooting & FAQ
- Reference Links
What Pywikibot Is
Pywikibot is a Python library and collection of ready-to-run CLI scripts that automates work on MediaWiki sites. It started in 2003 and is at version 11.3+ (MIT license). It wraps the MediaWiki Action API (api.php) into a high-level Python object model so you work with Page, Site, ItemPage, Category, User objects instead of raw HTTP/JSON.
It is the de-facto standard bot framework for the Wikimedia ecosystem — used on Wikipedia, Wikidata, Commons, Wiktionary, and thousands of third-party MediaWiki wikis.
Three-Layer Architecture
User Code (your script or custom bot)
↓ Page / User / Category / ItemPage objects
Pywikibot Library
├── site — Site objects (connection, auth, per-wiki config)
├── page — Page, FilePage, Category, User, ItemPage, PropertyPage, LexemePage
├── pagegenerators — 40+ generator types (cat, links, search, transclude, ...)
├── bot — Bot class hierarchy (BaseBot → ExistingPageBot → CurrentPageBot)
├── textlib — Wikitext parsing (section splitting, template extraction, category links)
├── cosmetic_changes — Auto-fix wikitext formatting (whitespace, HTML, deprecated syntax)
├── data.api — Low-level API request wrapper with auto-retry + continuation
├── diff — Delta computation helpers
└── i18n — Internationalization support
↓
MediaWiki Action API (api.php) — query | edit | parse | upload | delete | move | protect
Where It Fits in the API Universe
Pywikibot is not an alternative to the other skills — it is a meta-layer that consumes the Action API and provides a framework on top of it. Many tasks that could be done with raw API calls are much easier with Pywikibot.
| Layer | Tools / Skills | Best For |
|---|
| REST APIs (v1) | wikimedia-api-access | Single-page reads, quick GET requests |
Action API (api.php) | wikimedia-api-access, wikimedia-diffs, wikipedia-edit-history | Low-level queries, diffs, edit history |
| SPARQL (Wikidata Query Service) | wikidata | Complex graph queries, aggregations, joins |
| SQL replicas (Toolforge DB) | wikimedia-database | Deep analytics, JOINs across wiki metadata |
| Pywikibot | ⬅️ This skill | Full bot lifecycle: edit pages, run mass operations, scrape templates into Wikidata, archive talk pages, move categories, write custom bots |
API Coverage (Action → Method Mapping)
| API Action | Pywikibot Method |
|---|
action=edit | Page.save(), Page.put(), Page.touch() |
action=query | Site.pagegenerators, Site.allpages(), Site.search() |
action=delete | Page.delete() |
action=move | Page.move() |
action=protect | Page.protect() |
action=upload | Site.upload(), FilePage.upload() |
action=parse | Page.get_parsed_page(), Site.expand_text() |
action=rollback | Page.rollback() |
action=watch | Page.watch() |
action=block | User.block() |
action=emailuser | User.send_email() |
action=compare | Site.compare() |
action=mergehistory | Page.merge_history() |
action=purge | Page.purge() |
action=thank | Site.thank_revision() |
Full cross-reference table: doc.wikimedia.org/pywikibot/stable/mwapi.html
Installation & Setup
Method 1: pip (library-only, no built-in scripts)
pip install pywikibot
This gives you the library. You must create user-config.py manually (see below). The ready-to-run scripts (archivebot, replace, etc.) are not included in this mode.
Method 2: Repository (full installation with scripts)
git clone https://gerrit.wikimedia.org/r/pywikibot/core.git
cd core
git submodule update --init
pip install -r requirements.txt
This gives you the pwb.py wrapper and all built-in scripts.
user-config.py
Create a file named user-config.py in your working directory (or home directory if using pip mode):
mylang = 'en'
family = 'wikipedia'
usernames['wikipedia']['en'] = 'MyBotUsername'
For multiple wikis:
mylang = 'en'
family = 'wikipedia'
usernames['wikipedia']['en'] = 'MyBot'
usernames['wikidata']['wikidata'] = 'MyBot'
usernames['commons']['commons'] = 'MyBot'
put_throttle = 1
max_retries = 5
retry_wait = 30
user_agent_description = 'MyBot/1.0 (my@email.com)'
Or use the interactive setup script:
python pwb.py generate_user_files
Quick Test
import pywikibot
site = pywikibot.Site('en', 'wikipedia')
page = pywikibot.Page(site, 'Wikipedia:Sandbox')
print(page.text[:500])
Core Object Model
Site — a wiki connection
site = pywikibot.Site('en', 'wikipedia')
repo = pywikibot.Site('wikidata', 'wikidata')
commons = pywikibot.Site('commons', 'commons')
custom = pywikibot.Site('en', 'mywiki')
Page — a wiki article
page = pywikibot.Page(site, 'Albert Einstein')
print(page.text)
print(page.title())
print(page.namespace())
print(page.exists())
print(page.isRedirectPage())
print(page.toggleTalkPage())
page.text = page.text.replace('foo', 'bar')
page.save('Bot: fixing typo')
page.move('New Title', 'reason')
page.delete('deletion reason')
page.protect(edit='sysop', move='sysop', reason='protection reason')
for rev in page.revisions(total=5):
print(rev['user'], rev['timestamp'], rev['comment'])
print(page.getCreator())
print(page.getLatestRevisionId())
Category — category pages
cat = pywikibot.Category(site, 'Category:Physics')
for article in cat.articles():
print(article.title())
for subcat in cat.subcategories():
print(subcat.title())
for page in cat.articles(recurse=True):
...
User — user pages and contributions
user = pywikibot.User(site, 'ExampleUser')
print(user.editCount())
print(user.registrationTime())
for contrib in user.contributions(total=10):
print(contrib['title'], contrib['timestamp'])
user.block(reason='vandalism', expiry='24 hours')
user.unblock(reason='appeal granted')
user.send_email(subject='Greetings', text='Hello!')
FilePage — image/file pages
fp = pywikibot.FilePage(commons, 'File:Example.jpg')
print(fp.fileUrl)
print(fp.getImageSize())
print(fp.getFileVersionHistory())
fp.download('local.jpg')
fp.upload('source.jpg', comment='Uploading', file_key='...')
Link — internal / interwiki links
from pywikibot import Link
link = Link('Albert Einstein', site)
print(link.title)
print(link.namespace)
print(link.site)
Page Generators — The Superpower
This is the single most powerful abstraction in Pywikibot. A generator is an iterable that yields Page objects, composable with command-line options. You can chain them to build complex page lists.
40+ Generator Types (all usable via CLI flags too)
| CLI Flag | Generator | Example |
|---|
-cat: | Pages in a category | -cat:Physics |
-catr: | Pages in category + subcategories | -catr:Physics |
-links: | Pages linking to a page | -links:"Albert Einstein" |
-transcludes: | Pages that use a template | -transcludes:"Infobox person" |
-search: | Full-text search results | -search:"quantum mechanics" |
-subcats: | Subcategories of a category | -subcats:Physics |
-filelinks: | Pages using a file | -filelinks:Example.jpg |
-newimages: | Recently uploaded files | -newimages:50 |
-random: | Random pages | -random:10 |
-recentchanges: | Recent changes | -recentchanges:100 |
-page: | Single page by name | -page:"Albert Einstein" |
-pageid: | Single page by ID | -pageid:12345 |
-namespace: | Filter by namespace | -namespace:0 (main) |
-unusedfiles: | Unused files | -unusedfiles |
-uncatfiles: | Uncategorized files | -uncatfiles |
-uncatcat: | Uncategorized categories | -uncatcat |
-watchlist: | Current user's watchlist | -watchlist |
-logevents: | Log events (move, delete, etc.) | -logevents:move |
-liverecentchanges: | Live RC feed | -liverecentchanges |
-xml: | From a dump file | -xml:dump.xml |
-imageused: | Pages using an image | -imageused:Example.jpg |
-newpages: | Newly created pages | -newpages:50 |
-interwiki: | Same page on other language wikis | -interwiki |
Usage in Code
from pywikibot import pagegenerators
site = pywikibot.Site('en', 'wikipedia')
gen = site.categorizedpages('Physics', recurse=True)
gen = site.pagebacklinks('Albert Einstein', namespaces=[0])
gen = site.search('quantum mechanics', namespaces=[0])
gen_factory = pagegenerators.GeneratorFactory()
gen_factory.handle_args(['-cat:Physics', '-namespace:0'])
gen = gen_factory.getCombinedGenerator()
for page in gen:
print(page.title())
Chaining + Filters
gen = site.categorizedpages('Physics', recurse=True)
gen = site.pagebacklinks('Albert Einstein', namespaces=[0])
gen = pagegenerators.IntersectPageGenerator(gen1, gen2)
Bot Framework
Pywikibot provides a class hierarchy so you only write the per-page logic:
BaseBot
└── ExistingPageBot (iterates, calls treat_page for each)
└── CurrentPageBot (also provides current_page attribute)
└── ConfigParserBot (reads options from .ini file)
Minimal Custom Bot
import pywikibot
from pywikibot import pagegenerators
from pywikibot.bot import ExistingPageBot
class MyBot(ExistingPageBot):
update_options = {
'text': 'Appending this line',
'summary': 'Bot: appending text'
}
def treat_page(self):
"""Called once per page."""
self.current_page.text += '\n' + self.opt.text
self.put_current(summary=self.opt.summary)
def main():
options = {}
gen_factory = pagegenerators.GeneratorFactory()
local_args = pywikibot.handle_args()
local_args = gen_factory.handle_args(local_args)
for arg in local_args:
opt, sep, value = arg.partition(':')
if opt in ('-summary', '-text'):
options[opt[1:]] = value
bot = MyBot(generator=gen_factory.getCombinedGenerator(), **options)
bot.run()
if __name__ == '__main__':
main()
Running It
python mybot.py -cat:Physics -summary:"Bot test" -text:"Hello"
The bot framework handles:
- Page iteration from any generator
- Save throttling (
put_throttle)
- Edit conflict detection and retry
- User confirmation prompts (
-always to skip)
- Logging and error handling
- Dry-run mode (
-simulate)
Key Built-in Scripts Catalog
Scripts are available only in repository mode (git clone). They are run via python pwb.py <scriptname>.
Category Scripts
| Script | What It Does | One-Liner |
|---|
category_redirect | Move pages out of redirected categories, fix double redirects, log manual cases | pwb.py category_redirect -cat:"Category:Redirects" |
category_graph | Visualize category hierarchy as DOT/SVG/HTML | pwb.py category_graph -from Physics -depth 3 |
commonscat | Add {{commonscat}} to categories by following interwiki links | pwb.py commonscat -start:Category:! |
Template Scripts
| Script | What It Does | One-Liner |
|---|
template | Replace/substitute/remove templates across pages | pwb.py template "OldTemplate" "NewTemplate" -namespace:0 |
templatecount | Count/list pages transcluding one or more templates | pwb.py templatecount -count -namespace:0 ref note |
General Page Scripts
| Script | What It Does | One-Liner |
|---|
archivebot | Archive old discussion threads from talk pages | pwb.py archivebot |
replace | Find-and-replace text across many pages | pwb.py replace -cat:Physics "old" "new" |
add_text | Append/prepend text to pages | pwb.py add_text -cat:Physics -text:"{{stub}}" |
redirect | Fix double redirects or delete broken ones | pwb.py redirect double |
disambredir | Fix redirects to disambiguation pages | pwb.py disambredir -cat:Disambiguation |
patrol | Mark recent changes as patrolled | pwb.py patrol -page:RecentChanges |
Image/File Scripts
| Script | What It Does | One-Liner |
|---|
checkimages | Audit recently uploaded files for description problems | pwb.py checkimages -limit:50 |
commons_information | Add language templates to file descriptions on Commons | pwb.py commons_information -start:File:! |
image_transfer | Transfer images between wikis (e.g. to Commons) | pwb.py image_transfer -from:en -to:commons |
Admin Scripts
| Script | What It Does | One-Liner |
|---|
delete | Mass delete or undelete pages | pwb.py delete -cat:"Category:To delete" -always |
blockpageschecker | Remove stale protection templates from unprotected pages | pwb.py blockpageschecker -always |
Non-Editing Scripts
| Script | What It Does | One-Liner |
|---|
listpages | List pages matching a generator, with format options | pwb.py listpages -cat:Physics -format:"{page.title}" |
pagefromfile | Create pages from a text file | pwb.py pagefromfile -file:newpages.txt |
touch | Touch pages (null-edit to update caches) | pwb.py touch -cat:Physics |
Utility Scripts
| Script | What It Does |
|---|
generate_user_files | Interactive user-config.py generator |
shell | Interactive Python shell with Pywikibot pre-loaded |
version | Print Pywikibot version and dependency info |
login | Authenticate (store cookies) |
Wikidata & Wikibase Integration
This is where Pywikibot has no real competitor — no other tool offers this depth of structured data manipulation.
Core Classes
| Class | Purpose |
|---|
pywikibot.ItemPage | A Wikidata item (Q937 = Einstein) |
pywikibot.PropertyPage | A Wikidata property (P106 = occupation) |
pywikibot.Claim | A statement (item + property + value) |
pywikibot.LexemePage | A Wikibase Lexeme (L-ID) |
pywikibot.LexemeForm | A form within a Lexeme |
pywikibot.LexemeSense | A sense within a Lexeme |
pywikibot.SiteLink | A sitelink object (page on a specific wiki) |
Reading Items
repo = pywikibot.Site('wikidata', 'wikidata')
item = pywikibot.ItemPage(repo, 'Q937')
data = item.get()
print(item.labels)
for prop_id, claims in data['claims'].items():
for claim in claims:
print(f"{prop_id}: {claim.getTarget()}")
for site_id, sitelink in data['sitelinks'].items():
print(f"{site_id}: {sitelink.title}")
Editing Items
item.editLabels({'en': 'Albert Einstein', 'de': 'Albert Einstein'})
item.editDescriptions({'en': 'German-born theoretical physicist'})
claim = pywikibot.Claim(repo, 'P106')
target = pywikibot.ItemPage(repo, 'Q169470')
claim.setTarget(target)
item.addClaim(claim)
qualifier = pywikibot.Claim(repo, 'P580')
qualifier.setTarget(pywikibot.WbTime(year=1905))
claim.addQualifier(qualifier)
source = pywikibot.Claim(repo, 'P854')
source.setTarget('https://example.com/source')
claim.addSource(source)
Scripts for Wikidata
| Script | What It Does |
|---|
harvest_template | Scrape template parameters from wikitext → Wikidata claims. Uniquely powerful — maps infobox fields to P-IDs, handles links, multiple values, inverse claims, and cross-wiki harvesting. |
claimit | Add claims in bulk from a list of property→value pairs. Supports geo-coordinates, string values, item values. |
illustrate_wikidata | Add images to Wikidata items from Wikipedia page images (PageImages extension). |
merge_items | Merge duplicate Wikidata items. |
interwikidata | Manage interwiki links through Wikidata. |
harvest_template — The Killer Feature
This script reads template parameters from Wikipedia articles and writes them as structured Wikidata claims. Example:
pwb.py harvest_template \
-site:wikipedia:en \
-namespace:0 \
-template:"Infobox person" \
image P18
pwb.py harvest_template \
-site:wikipedia:en \
-template:"Infobox musical artist" \
current_members P527 \
-exists:p -multi
Commons / File Uploads
from pywikibot import FilePage, Site
commons = Site('commons', 'commons')
fp = FilePage(commons, 'File:Example.jpg')
print(fp.fileUrl)
print(fp.getFileVersionHistory())
fp.download('local.jpg')
uploader = commons.upload('local.jpg',
description='== Summary==\nDescription here\n==Licensing==\n{{self|cc-by-sa-4.0}}',
comment='Uploading via bot')
Image Transfer Script
pwb.py image_transfer -from:en -to:commons \
-cat:"Category:Images" \
-description:"My description" \
-always
Running on Toolforge & PAWS
Toolforge (Kubernetes)
Pywikibot is pre-installed on Toolforge. See the wikimedia-toolforge skill for detailed setup.
ssh <toolname>@login.toolforge.org
become <toolname>
python3 -m venv venv
source venv/bin/activate
pip install pywikibot
source /data/project/shared/pywikibot/venv/bin/activate
PAWS (Jupyter Notebooks)
PAWS is the easiest way to try Pywikibot — no installation needed:
- Go to paws.wmflabs.org
- Start a new Python 3 notebook
!pip install pywikibot or use the pre-installed version
- Authenticate interactively with
pywikibot.Site().login()
One-of-a-Kind Capabilities
These are tasks that are extremely difficult or impractical with any other approach (raw API calls, SPARQL, or SQL):
★★★★★ Template-to-Wikidata Harvesting
The harvest_template script reads wikitext templates (infoboxes) and writes structured Wikidata claims. It handles:
- Multiple values from multi-valued template parameters
- Link detection (
-islink: auto-link plain text)
- Inverse claims (
-inverse:P910 — if A has property X pointing to B, also set property Y on B pointing to A)
- Exists-checking (
-exists:ptq — only add if target/source/qualifier don't match)
- Cross-wiki harvesting (same template on different language Wikipedias → same Wikidata item)
No other tool can do this. Raw API calls would require reimplementing template parsing, link resolution, claim deduplication, and cross-wiki coordination.
★★★★★ Mass Category Redirect Fixing
The category_redirect script finds redirected categories, moves all member pages, fixes double redirects, converts hard→soft redirects, and logs cases that need manual attention. It respects a cooldown period (default 7 days) to avoid edit-warring.
★★★★★ Talk Page Archiving
The archivebot script parses timestamps in talk page threads, identifies old discussions, moves them to subpages with configurable thresholds, and increments archive counters. Configuration is done entirely via a transcluded template (no CLI flags needed). Variables like %(counter)d and %(page)s allow dynamic archive page names.
★★★★★ Template Substitution / Removal Across Entire Wikis
The template script can -subst (replace {{Foo}} with {{subst:Foo}}) or -remove templates across millions of pages. The substitution-aware mode (-assubst) combines both operations. Process-by-user filters let you restrict to pages edited by a given user within a time window.
★★★★★ Cross-Wiki Batch Operations
The interwiki generator yields pages across all language versions simultaneously. Combined with any script, this lets you fix interwiki spam or update templates on 300+ Wikipedias from a single command.
★★★★★ Geo-Coordinate Claims from Template Data
The claimit script accepts decimal coordinates directly:
pwb.py claimit [generators] P625 -23.3991,-52.0910,0.0001
This is otherwise very fiddly with raw API calls (requires coordinate precision handling).
★★★★★ Wikitext Cosmetic Cleanup
The cosmetic_changes module automatically fixes:
- HTML-style
<br> → <br /> and similar
- Deprecated
<center> → <div style="text-align: center;">
- Missing whitespace around headings
- Obsolete self-closing tags
- Double-hyphen → em-dash where appropriate
Running replace -cosmeticchanges across a wiki applies these fixes automatically.
★★★★★ Batch Image Transfer to Commons
The image_transfer script handles the full pipeline: download from source wiki, check license compatibility, build description page with attribution history, upload to Commons. This is a multi-step, multi-API process that Pywikibot coordinates end-to-end.
When NOT to Use Pywikibot
| Scenario | Better Approach |
|---|
| Read a single page | curl + REST API v1 (~2 lines vs Pywikibot's import + site setup) |
| Simple SPARQL query | Wikidata Query Service directly (SPARQL is cleaner than Pywikibot's SPARQLQuery) |
| High-concurrency scraping | Raw aiohttp + async (Pywikibot is synchronous/blocking) |
| Real-time event stream | EventStreams API (Server-Sent Events, not Pywikibot) |
| JOIN-heavy analytics | Toolforge SQL replicas (wikimedia-database skill) |
| One-shot data extraction | python3 -c "import requests; print(requests.get(...).json())" |
| Non-Python environment | Direct Action API calls from any language |
| Minimal serverless function | REST API v1 (no auth needed for reads, tiny dependency) |
| Just re-upload one file | UploadWizard or direct API upload |
Rule of thumb: If your task touches fewer than 50 pages and involves no edits, use a lighter tool. If you need to edit 500+ pages, scrape templates into Wikidata, or run a recurring maintenance bot, Pywikibot is the right choice.
Security & Bot Credentials
Bot Passwords (Recommended for Bots)
- Go to
Special:BotPasswords on the wiki
- Create a bot password with only the permissions you need
- In
user-config.py:
usernames['wikipedia']['en'] = 'MyBot'
authenticate['en.wikipedia.org'] = ('MyBot@BotName', 'password_generated_by_special_botpasswords')
OAuth (for tools hosted on Toolforge)
authenticate['en.wikipedia.org'] = (
consumer_token,
consumer_secret,
access_token,
access_secret
)
OAuth 2.0 (Pywikibot 9.0+)
authenticate['*.wikipedia.org'] = (
'oauth2_client_id',
'oauth2_client_secret',
'oauth2_access_token'
)
Never Hardcode Credentials
- Use environment variables or a separate
secrets.py (gitignored)
- On Toolforge, use
$HOME/.pywikibot/user-config.py with restrictive permissions (chmod 600)
- Bot passwords expire — monitor and rotate them
Troubleshooting & FAQ
"No module named 'pywikibot'"
Make sure you installed it: pip install pywikibot. If using the repository, run pip install -r requirements.txt.
"Script not found" when running pwb.py scriptname
Scripts are only available in repository mode. Run from the core/ directory. Or use python -m pywikibot.scripts.<scriptname> from pip mode if the script supports it.
Login fails / "Not logged in"
- Check
user-config.py for correct username and family
- Bot passwords must match the format
Username@BotName
- Some wikis require OAuth — check wiki's bot policy
- Run
python pwb.py login explicitly to test authentication
⚠️ MediaWiki API username quirk: spaces → underscores
The MediaWiki action=login API requires the internal database form of usernames,
where spaces are stored as underscores (_). The lgname parameter will fail if you
pass a username with literal spaces (e.g., "AL Wiki MIT"), even though that's how
the username displays on wiki pages and in the Special:BotPasswords confirmation message.
Always normalize usernames by replacing spaces with underscores before passing
them to lgname (or let Pywikibot handle it — it does this internally via Site.login()).
If you're calling the Action API directly:
lgname = "AL_Wiki_MIT"
lgname = "AL Wiki MIT"
Pywikibot handles this automatically in most cases, but if you're writing raw API
calls or a custom auth flow, remember to normalize the username.
Rate limiting / 429 errors
Adjust in user-config.py:
put_throttle = 2
max_retries = 10
retry_wait = 60
Edit conflicts / "Page was changed while editing"
Pywikibot detects badtimestamp errors and retries automatically by re-fetching the page. You can also call page.save() with force=True to skip the check (use with caution).
Cross-wiki: "Unknown family"
Add a family file or use the built-in families. Common families: wikipedia, wikidata, commons, wiktionary, wikisource, wikiquote, wikivoyage, wikinews, wikiversity, mediawiki.
"The module pywikibot is not installed" in pip mode
Scripts are excluded from pip installs. Use library mode: write your own script importing pywikibot directly. Alternatively, use pip install pywikibot[scripts] where available.
Reference Links