| name | administration-documentation |
| description | Use when preparing the administration documentation / manual for an operational selection procedure — the materials administrators and users need to administer, score, interpret, secure, and communicate the procedure consistently. Covers administrator qualifications, the testing environment, scoring/interpretation, security, candidate communications and feedback, reassessment, nonstandard administrations, data retention, and review/updating. Triggers: "administration manual", "test administration documentation", "test security", "candidate feedback", "retest policy", "reassessment", "data retention for test scores", "proctoring / unproctored internet testing". |
| version | 1.0.0 |
| author | OpenMatter-Network |
| license | MIT |
| category | research |
| tags | ["Community","io-psychology","personnel-selection"] |
| permissions | [] |
Administration documentation
The operational counterpart to the technical-validation-report. It tells administrators and
users how to deliver the procedure so that scores mean what the validation says they mean. Keep it
separate from the technical report so confidential research information is protected and
administrators get only what they need. A publisher's manual often must be supplemented with
organization-specific addenda for local decisions.
A test is valid for an inference only if administered, scored, and used as it was validated.
Deviations from prescribed procedures threaten the inference.
Who administers — qualifications & training
State the qualifications required for each administrative responsibility (administering, scoring,
maintaining scoring algorithms, handling retest requests). Train administrators on the specific
procedure and on why standardized administration matters — failures to follow protocols can
render research results and operational scores partly meaningless. Build observational checks/
quality control; periodically review pass rates / mean scores for spikes or dips signaling a
compromised key or administration problems. Administrators should be aware of personal limitations
(physical, perceptual, cognitive) and decline to administer/score when they can't meet role demands.
What to document
- Introduction & overview — purpose; why the org uses validated procedures; importance of
security and consistency. Don't oversell the program's applicability.
- Contact information — whom to contact for questions/problems.
- Selection procedures — names, editions, forms; ordering and security; sample items (not full
secure content); for proprietary tools, a description of items and constructs.
- Applicability — to whom the procedure applies, exceptions/exemptions, and rules about when
it's administered (e.g., only when a vacancy exists vs. periodic pooling). Include applicability to
candidates with disabilities and different cultural/linguistic groups.
- Administration responsibilities & guidelines — detailed administration instructions; rules/tips
for an appropriate testing environment and verifying candidate identity; for remote/unproctored
internet testing (UIT/RPIT), the administrator still must inform test takers of environmental
factors and security expectations.
- Administration environment — workspace, lighting, quiet, freedom from distractions, and effects
of technology (browser, monitor size, touch screen). At minimum, an environment responsive to
candidate concerns and dignity; inform test takers of conditions that may affect performance.
- Scoring instructions & interpretation guidelines — how to score and interpret; keep scoring
keys out of manuals/technical reports (release only to those who score). Document any
administrator role in interpreting scores; ensure accuracy in scoring/recording (responsibility
can't be delegated away). For computer-based scoring/automated feedback, give detailed
decision-use instructions.
- Reporting and using scores — how scores are reported and used, who has access; caution against
using scores for unintended purposes (e.g., later retention decisions) without evidence; if
scores drive differential job assignments, require evidence linking scores to those jobs.
Security
Public disclosure of content/scoring is a serious threat to reliability, validity, and subsequent
use. Retain all materials at a security level allowing access only to those with a need to
know. In UIT/RPIT, adopt mechanisms to reduce content compromise and cheating and to confirm
identity. Communicate and enforce protections (verification codes, content rotation, supervised
destruction of confidential documents); document a retention period for confidential information;
consult evolving international data-privacy law.
Candidate-facing communications
- Information provided to candidates — clear, pertinent, timely: intended use, procedures, format/
interface, completion strategies (e.g., can they skip/return), time parameters, feedback/score
access, confidentiality, accommodation request process, retesting policy, conduct expectations.
Standardize content and delivery for consistent treatment.
- Candidate feedback — if feasible/appropriate, give clear, understandable feedback without
compromising test security or scoring.
Special situations
- Nonstandard administrations — document how to handle accommodations and disruptions (power
failures, emergencies, illness), errors (mismatched booklet/sheet, timing mistakes). Establish a
clear process to document and explain any deviation in administration, scoring, or handling.
Where reporting a nonstandard event (e.g., dropped internet connection) is left to the test taker,
explain how and when to report.
- Reassessing candidates — generally provide reassessment opportunities when feasible; state
clearly whether and how candidates may be reassessed (required time intervals, prerequisite
developmental activities). Some one-time examinations may preclude it.
- Corrective reassessment — define when it's appropriate (administrator errors like timing
mistakes/nonmatching materials; extraordinary disturbances). Document procedures, required timing,
and documentation; inform remote test takers of the conditions under which they may request it.
Data retention & test-score databases
Decide what administration records/scores to keep; document record-keeping and retention policies
(duration, security, accessibility) consistent with federal/state/local law, industry best practice,
and recent court rulings. Retain raw item data and scores (derived scales limit future research).
Secure personally identifying data to need-to-know access. Maintain databases long enough to support
periodic audits and ongoing evaluation of the selection system.
Review & updating
Periodically review operational use — including timeliness of normative data — to decide whether more
research is needed. The greater the changes to test format, mode of administration, instructions, or
language, the more likely score interpretations no longer hold. Revise documentation (or add an
addendum) whenever changes make any statement or instruction incorrect or misleading. Changes in
organizational demands (new assessments, cutoff adjustments) require further study of the
procedure.
Pitfalls
- Putting scoring keys or live items in the manual.
- One-size communications that aren't standardized → inconsistent candidate treatment.
- No documented process for deviations/disruptions.
- Vague or absent retest/reassessment rules.
- Letting the manual drift out of date after format/mode/language changes.
- Using operational scores for purposes never validated.
Checklist
See also
technical-validation-report (keep separate) · candidate-accommodations (nonstandard
administrations) · fairness-and-bias-analysis (equitable treatment) ·
selection-decisions-and-scoring (how scores are used)
Source: Principles (5th ed., 2018), "Operational Considerations → Administration Information and
Other Circumstances Regarding the Validation Effort and Use of Selection Procedures."
