| name | bolton-rapid-software-testing |
| description | Test software in the style of Michael Bolton, Rapid Software Testing co-creator with James Bach. Emphasizes the distinction between testing and checking, critical thinking, oracles, and the social nature of quality. Use when designing test strategies, evaluating test automation, or developing critical thinking in testers. |
| tags | rapid-testing, context-driven, heuristics, oracles, risk-based, exploratory, skills, critical-thinking |
Michael Bolton Rapid Software Testing Style Guide
Overview
Michael Bolton is the co-creator of Rapid Software Testing (with James Bach) and one of the most influential voices in the testing community. He is known for making crucial distinctions—most famously between "testing" (a human cognitive activity) and "checking" (a machine-executable verification). His work emphasizes that quality is a relationship between people, and that testing is fundamentally about critical thinking and learning.
Core Philosophy
"Testing is the process of evaluating a product by learning about it through experiencing, exploring, and experimenting."
"Checking is the process of making evaluations by applying algorithmic decision rules to specific observations of a product."
"Quality is value to some person(s) who matter(s)."
Bolton argues that much of what we call "automated testing" is actually "automated checking"—valuable, but not the same as the skilled cognitive work of testing. Real testing requires a human mind to recognize problems that we couldn't specify in advance.
Design Principles
-
Testing ≠ Checking: Distinguish human investigation from algorithmic verification.
-
Quality Is Relational: Quality means value to someone who matters—understand who.
-
Oracles Are Heuristic: We recognize problems through fallible principles, not perfect rules.
-
Testing Is Social: Communicate findings in terms stakeholders understand and care about.
-
Critical Thinking First: Question everything, including your own assumptions.
Testing vs. Checking
┌─────────────────────────────────────────────────────────────┐
│ TESTING │
│ │
│ Human cognitive activity: │
│ • Learning about the product │
│ • Exploring unknown territories │
│ • Recognizing problems we couldn't predict │
│ • Applying judgment and sapience │
│ • Adapting to what we discover │
│ • Modeling and questioning │
│ │
│ Cannot be automated (requires human mind) │
├─────────────────────────────────────────────────────────────┤
│ CHECKING │
│ │
│ Algorithmic verification: │
│ • Confirming expected behaviors │
│ • Applying decision rules │
│ • Binary pass/fail outcomes │
│ • Repeatable observations │
│ • Regression detection │
│ • Fast feedback on known behaviors │
│ │
│ CAN be automated (but requires human design and oversight) │
└─────────────────────────────────────────────────────────────┘
CRITICAL: "Automated testing" is mostly automated checking.
A human DESIGNS the checks.
A human INTERPRETS the results.
A human DECIDES what problems matter.
When Testing
Always
- Distinguish what you're testing from what you're checking
- Identify who cares about quality and what they value
- Apply multiple oracles to recognize problems
- Report problems in terms of risk and stakeholder impact
- Question the requirements—they're someone's best guess
- Document your mental model of the product
- Learn continuously throughout testing
Never
- Confuse "tests passed" with "product is good"
- Assume automation replaces testing
- Report only binary pass/fail without context
- Stop exploring when you find one problem
- Trust that absence of failures means absence of problems
- Ignore problems because they're not in requirements
- Treat testing as a phase that ends
Prefer
- Exploring over scripted confirmation
- Learning over executing
- Thinking over clicking
- Questioning over accepting
- Modeling over listing
- Communicating over documenting
- Understanding over measuring
Code Patterns
Testing vs. Checking in Practice
def check_login_returns_token():
"""
A check: Confirms a specific expected behavior.
Valuable, but not testing.
"""
response = api.login(username="valid", password="valid")
assert response.status_code == 200
assert "token" in response.json()
class LoginTesting:
"""
Testing: A human explores, learns, and discovers.
Cannot be fully automated.
"""
def explore_login_behavior(self):
"""
A tester might wonder:
- What happens with Unicode usernames?
- What if I login from two devices simultaneously?
- What if the password is extremely long?
- What if I include SQL in the username?
- What happens under high load?
- What does the UI do while waiting?
- How does a confused user perceive this?
These questions emerge from a human mind engaging
with the product. The answers require judgment to evaluate.
"""
pass
def recognize_problems(self, observation):
"""
Applying oracles - human judgment about what's problematic.
Is a 3-second login response time a problem?
- For a banking app: probably yes
- For a first login setting up encryption: probably fine
Context and stakeholder value matter.
"""
pass
class TestingStrategy:
"""
Bolton's approach: automate checking, support testing.
"""
def __init__(self):
.checks = []
.test_ideas = []
():
.checks.append(check_function)
():
.test_ideas.append({
: idea,
: oracles,
: ,
: []
})
():
Quality as Value to Stakeholders
class QualityPerspective:
"""
Quality is relational: value to some person(s) who matter(s).
Different stakeholders have different quality criteria.
"""
def __init__(self, product: str):
self.product = product
self.stakeholders = {}
def identify_stakeholder(self,
name: str,
role: str,
values: list,
fears: list):
"""
Identify what quality means to each stakeholder.
"""
self.stakeholders[name] = {
'role': role,
'values': values,
'fears': fears,
}
def analyze_quality_dimensions(self):
"""
Example stakeholder analysis for a banking app.
"""
stakeholders = {
'end_user': {
'values': ['fast transactions', 'easy to use', 'works offline'],
'fears': ['losing money', 'confusing interface', 'downtime'],
},
'security_officer': {
'values': ['encryption', 'audit trails', 'compliance'],
: [, , ],
},
: {
: [, , ],
: [, , ],
},
: {
: [, , ],
: [, , ],
},
: {
: [, , ],
: [, , ],
},
}
stakeholders
() -> :
impacts = []
stakeholder affected_stakeholders:
impact = .assess_impact(bug, stakeholder)
impacts.append()
+ .join(impacts)
Oracle Design
class OracleFramework:
"""
Oracles: Principles by which we recognize problems.
Bolton emphasizes that all oracles are heuristic (fallible).
"""
def __init__(self):
self.applied_oracles = []
ORACLE_CATEGORIES = {
'reference': {
'description': 'Compare to authoritative sources',
'examples': [
'Requirements documents',
'Specifications',
'Standards (ISO, RFC)',
'Regulations (GDPR, HIPAA)',
'Contracts',
],
'fallibility': 'Requirements can be wrong or incomplete'
},
'comparable': {
'description': 'Compare to similar things',
'examples': [
'Previous versions',
'Competitor products',
'Similar features in same product',
'Industry norms',
],
'fallibility': 'Similar is not identical; context differs'
},
'model': {
'description': 'Compare to mental or formal models',
'examples': [
'User mental models',
'Developer mental models',
'State machines',
'Mathematical models',
],
'fallibility':
},
: {
: ,
: [
,
,
,
],
:
},
: {
: ,
: [
,
,
,
,
],
:
},
: {
: ,
: [
,
,
,
,
],
:
},
}
() -> :
{
: observation,
: category,
: reference,
: ,
: ,
: .ORACLE_CATEGORIES[category][]
}
() -> :
questions = [
,
,
,
,
,
]
questions
Critical Thinking in Testing
class CriticalThinkingFramework:
"""
Bolton emphasizes critical thinking as the core testing skill.
"""
CRITICAL_QUESTIONS = {
'source': [
"Who is making this claim?",
"What is their expertise?",
"What might they be wrong about?",
"What might they not know?",
],
'evidence': [
"What evidence supports this?",
"How was the evidence gathered?",
"Could the evidence be misleading?",
"What evidence would contradict this?",
],
'assumptions': [
"What assumptions does this rely on?",
"Are the assumptions valid?",
"What if the assumptions are wrong?",
],
'implications': [
"If this is true, what follows?",
"If this is false, what follows?",
"What are the consequences of being wrong?",
],
'alternatives': [
"What other explanations are possible?",
"What are we not considering?",
"What would someone who disagrees say?",
],
}
def analyze_test_result(self, result: dict) -> dict:
"""
Apply critical thinking to a test result.
"""
analysis = {
'result': result,
'questions': [],
'alternative_explanations': [],
'confidence': ,
}
result[] == :
analysis[] = [
,
,
,
,
]
:
analysis[] = [
,
,
,
,
]
analysis
() -> :
[
,
,
,
,
,
,
]
() -> :
[
,
,
,
,
,
,
]
Bug Advocacy
class BugAdvocacy:
"""
Bolton's approach: Advocate for bugs to be understood and fixed.
A bug report is a work of technical writing.
"""
def write_bug_report(self,
summary: str,
observation: str,
oracles: list,
stakeholder_impact: str,
reproduction: str = None) -> dict:
"""
A good bug report tells a story.
"""
return {
'summary': summary,
'what_i_did': reproduction or "See attached session notes",
'what_happened': observation,
'why_this_matters': stakeholder_impact,
'oracles_applied': oracles,
'additional_context': {
'environment': "...",
'related_observations': [],
'questions': [],
},
'suggested_priority': None,
}
def frame_bug_for_stakeholder(self,
bug: dict,
stakeholder: str) -> :
frames = {
: ,
: ,
: ,
}
frames.get(stakeholder, frames[])
Session-Based Testing with RST
class RapidTestingSession:
"""
A Rapid Software Testing session combines
exploration, checking, and critical thinking.
"""
def __init__(self,
charter: str,
time_box_minutes: int = 90):
self.charter = charter
self.time_box = time_box_minutes
self.notes = []
self.checks_performed = []
self.oracles_applied = []
self.problems_found = []
self.questions_raised = []
def record_observation(self,
what_i_did: str,
what_happened: str,
my_interpretation: str):
"""
Record observations with interpretation.
"""
self.notes.append({
'time': datetime.now(),
'action': what_i_did,
'result': what_happened,
'interpretation': my_interpretation,
'oracle_used': None,
})
def recognize_problem(self,
observation: str,
oracle: str,
stakeholders_affected: list,
my_reasoning: str):
"""
Document problem recognition with full reasoning.
"""
.problems_found.append({
: observation,
: oracle,
: my_reasoning,
: stakeholders_affected,
: ,
: [],
})
():
{
: [
n n .notes
n[].lower()
n[].lower()
],
: .checks_performed,
:
,
}
() -> :
{
: .charter,
: .time_box,
: [n[] n .notes],
: [
n[] n .notes
n[].lower()
n[].lower()
],
: .problems_found,
: .questions_raised,
: [],
: {
: ,
: ,
: ,
},
}
Mental Model
Bolton approaches testing by asking:
- Am I testing or checking? Know the difference
- Who matters? Quality is value to someone
- How do I recognize problems? Apply oracles consciously
- What am I assuming? Question everything
- How do I communicate this? Frame for your audience
The RST Checklist
□ Charter defined with clear scope
□ Stakeholders identified (who cares?)
□ Oracles selected (how will I recognize problems?)
□ Testing distinguished from checking
□ Observations recorded with interpretation
□ Problems framed in stakeholder terms
□ Questions captured for follow-up
□ Session debriefed and learnings captured
Signature Bolton Moves
- Testing vs. Checking distinction
- Quality as value to stakeholders
- Oracles are heuristic (fallible)
- Critical thinking as core skill
- Bug advocacy (not just reporting)
- Framing problems for audiences
- Questioning metrics and coverage
- Testing is a social activity
