faa-ams-vv

name

faa-ams-vv

description

Knowledge base from the FAA AMS Lifecycle V&V Guidelines (verification and validation), Version 3.0, dated April 2017. Use for applying verification and validation across the FAA Acquisition Management System (AMS) lifecycle: the verify-vs-validate distinction (built right vs. right thing), the work-product/component/product tiers, the nine V&V planning elements, the four method catalogs (Appendix C), decision-point-gated V&V (SASP, CRD, Investment Analysis, Solution Implementation, In-Service Management), the Appendix B document checklists (PRD/APB/ISPD/PMP), and the DT/OT/IOA test triad. Scope limits: this is an FAA acquisition-V&V doctrine, NOT a general SE process model, requirements-engineering manual, or test-design textbook. It borrows its core V&V definitions from CMMI v1.2 (SEI, paraphrased) and presumes the surrounding AMS lifecycle (see the FAA SEM pack); its phase tables are policy snapshots as of April 2017 and it is thin on detailed test methods, tool specifics, and anything outside FAA acquisition governance.

FAA AMS Lifecycle V&V Guidelines (Verification and Validation) — Version 3.0, April 2017

Source: Federal Aviation Administration (FAA) — US Government work, public domain | Chapters: 7

When to use

Use this skill when you need to apply verification and validation across the FAA Acquisition Management System (AMS) lifecycle, as defined in the FAA AMS Lifecycle V&V Guidelines Version 3.0 (April 2017). This pack covers the verify-vs-validate distinction (built right vs. right thing), the work-product/component/product tiers, the nine V&V planning elements, the four method catalogs (Appendix C), decision-point-gated V&V across AMS phases (SASP, CRD, Investment Analysis, Solution Implementation, In-Service Management), the Appendix B document checklists (PRD/APB/ISPD/PMP), and the DT/OT/IOA test triad. Pair it with the FAA SEM pack for the surrounding AMS lifecycle context.

Prerequisites: none, plain Markdown; no MCP server, API key, or licence tier needed at runtime.

How to Use This Skill

• Without arguments — load the Core Frameworks below for the verify-vs-validate split, the three V&V tiers, the four method catalogs, and how V&V gates each AMS decision point.
• With a topic — ask about verifying vs. validating, choosing a V&V method, the nine planning elements, the Appendix B checklists, the DT/OT/IOA triad, or what gets V&V'd in a given phase.
• With a phase — ask for SASP/CRD (ch03), Investment Analysis (ch04), Solution Implementation (ch05), or In-Service Management (ch06).
• With a chapter — request ch01–ch07 directly (index below).

Supporting files: glossary.md (terms + acronyms), patterns.md (11 how-to techniques), cheatsheet.md (decision rules, phase/gate/method tables, tells & smells).

Prerequisites: none — plain Markdown; no MCP server, API key, or licence tier needed at runtime. Pairs with the FAA SEM pack, which defines the surrounding AMS lifecycle this V&V doctrine overlays.

Core Frameworks & Mental Models

Verification vs. Validation — the two questions

The spine of the entire guideline. Verification confirms the thing was built right — that selected work products meet the requirements specified for them (templates, standards, specifications, policy). It is inherently incremental, running from initial requirements through every change to the finished end product. Validation confirms the right thing was built — that the finished product, a component, or a selected work product will deliver its intended purpose and satisfy user needs once in its operating environment. Both definitions are drawn from, and here paraphrased from, CMMI for Development v1.2 (SEI, 2006); the guideline cites CMMI and notes GAO relies on it for quality audits. The relative emphasis of the two shifts with program maturity: validation-heavy while needs and concepts are still forming, verification-heavy as design and product harden. It is a sliding balance, not a fixed split.

A worked example crystallizes it: a system specification is verified against the standards/templates that govern its content and format, and validated against the needs captured in the higher-level requirements document. Verification looks down at the spec; validation looks up at the need.

The three V&V tiers (and the upward flow)

V&V applies to a deliberate hierarchy: work product → product component → product.

• Work product — anything that represents, defines, or directs the product (designs, specs, plans, requirements, models, prototypes, contracts).
• Product component — a lower-level element integrated upward (possibly nested).
• Product — the delivered system, service, facility, or operational change.

V&V flows up the hierarchy: work products first, components before integration, the product before operational acceptance. Any item may undergo V&V more than once if modified. Both government and contractor organizations are required to perform V&V, on both parties' work. V&V ≠ QA: V&V targets requirements/mission/risk; QA targets whether the established processes were followed.

The four method catalogs (Appendix C)

Cross "verification vs. validation" with "work product vs. product/component" to get four method lists:

• Work-product verification — checklists, audits, peer reviews, inspections.
• Work-product validation — broader and user-facing: surveys and discussions with users, functional presentations, dry-run walk-throughs, storyboarding and modeling, demonstrations and testing, plus the verification set (analyses, audits, peer reviews, inspections, checklists).
• Product/component verification — analyses, audits, peer reviews, inspections, checklists, simulations, testing, demonstrations, and accreditation.
• Product/component validation — user-evaluation questionnaires and discussions with users, functional presentations, simulations and testing, demonstrations, analyses, audits, peer reviews, inspections, and checklists.

Validation catalogs are user-facing and broader; product-tier catalogs add analysis, simulation, and (for verification) accreditation. Definitions live in Appendix C (Ch 7).

Plan → Perform → Report → Decide

V&V is a four-stage pipeline, not an end-of-line test. Plan: V&V planning is mandatory for every AMS program and embedded in the program's documents (ISPD, TEMP, PMP, SEMP, QA Plan, phase plans), each carrying the relevant subset of the nine planning elements (methods × 2, items to V&V, processes/standards/criteria, tracking/reports, roles, independent reviewers, tools/labs, training). Perform: independent reviewers (not the item's developers) run the matched methods, confirming prerequisite V&V was done and still holds. Report: a controlled, archived report covering Introduction, Purpose, Criteria, Approach, Participants, Results, Recommendations. Decide: results become entrance criteria for AMS decision points. A gap in any earlier stage starves the decision.

Decision-point-gated, table-driven V&V across the AMS lifecycle

The guideline organizes V&V by the AMS lifecycle (SASP → CRD → Investment Analysis → Solution Implementation → In-Service Management), not a generic V-model — with the non-phase activity RSA feeding the front end. For each phase, a table (Tables 1–6) anchors on decision points and fixes four columns: the decision point, the minimum work products to V&V there, the criteria (validation criteria = substantive upstream content; verification criteria = templates/policy), and the responsible stakeholder. To plan V&V for any artifact, find its row — criteria and owner are pre-bound. Critically, the criteria are themselves V&V'd (yardsticks are earned, not assumed), and the tables are policy snapshots as of April 2017 — current phase guidance supersedes them. Figure 3 shows emphasis only; the tables define the required minimum.

Foundations, chaining, and front-loaded safety/security

Foundational documents (NAS ConOps, NAS Requirements Document, NSIP, NARP, NextGen Implementation Plan, Enterprise Architecture) are V&V'd against FAA Strategic Initiatives before downstream use. V&V is a chain: each verified-and-validated artifact becomes the baseline for the next, and a maturing product's earlier V&V results feed in as validation criteria for its later version — so under-validating early work products propagates the wrong product forward. Information security enters at the very first phase (ISS assessments validated against FIPS-199, verified against ISGSA) and safety (FAA SMS, Safety Risk Management documentation) joins as a gated obligation toward the In-Service Decision. The EA is both a source criterion and a recipient of V&V feedback — a closed loop.

The product-V&V engine: DT / OT / IOA

Product and component V&V lives mainly in Test and Evaluation, concentrated in Solution Implementation and In-Service Management: Development Test (DT) serves verification (developer-run, FAA witnessing); Operational Test (OT) serves validation (ANG-E OT Test Director and team); Independent Operational Assessment (IOA) gives an independent operational-readiness read for designated products. PDR verifies against the allocated baseline, CDR against the product baseline. The surrounding test plans/procedures/reports are themselves V&V'd first — bad test artifacts mean a decision built on bad data.

---

Chapter Index

#	Section	Key content
ch01	Introduction: Terms, Definitions, Philosophy	Verify vs. validate (paraphrased from CMMI v1.2), the work-product/component/product hierarchy, who performs V&V, V&V vs. QA, the EA as source criterion + feedback loop, V&V as a decision-and-risk instrument
ch02	Implementation Guidance: Plan/Perform/Report/Decide	The nine planning elements, document-embedded planning, independence, the validity chain, the four method catalogs, T&E (DT/OT/IOA), report minimum content, decision support
ch03	Approach + Early Lifecycle (SASP, CRD, RSA)	The five AMS phases, RSA as a non-phase activity, the five early management documents (APB/PRD/BC/ISPD/PMP), Appendix B checklists, the IER track, Tables 1–2, front-loaded security (FIPS-199/ISGSA)
ch04	Investment Analysis (Initial & Final)	Two-stage gating (which alternative? / is the baseline ready?), per-product validation vs. verification criteria, prior V&V results as forward criteria, per-product stakeholder accountability, Tables 3–4
ch05	Solution Implementation	The IBR baseline revalidation, the SI gate spine (Contract Award → ISD), DT/OT/IOA, PDR/CDR baseline staircase, safety (SMS) and security (System Security Authorization/POA&M) gates, Table 5
ch06	In-Service Management	"Prove the change, re-prove the purpose," the implement/deploy gates, configuration management (FAA Form 1800-2, Orders 1800.66 & 1320.58A), ISM DT/OT/IOA, EA-change control authority, Table 6
ch07

Topic Index

• Accreditation (method) → ch02, ch07
• Acquisition Management System (AMS) lifecycle → ch01, ch03
• Acquisition Program Baseline (APB) → ch03, ch04, ch07
• Analysis (verification method) → ch07
• Appendix B checklists (PRD/APB/ISPD/PMP) → ch03, ch07
• Artifact (evidence record) → ch07
• Audit (method) → ch02, ch07
• Business Case + IER track → ch03, ch04
• Concept & Requirements Definition (CRD) → ch03
• Concept of Operations / Concept of Use → ch03, ch05
• Configuration management (Form 1800-2, Order 1800.66) → ch06
• Critical Design Review (CDR) → ch05
• Critical Operational Issue (COI) → ch05
• Critical Performance Requirement (CPR) → ch07
• Critical technical documents → ch02
• CMMI v1.2 (paraphrased source of definitions) → ch01, ch03, ch07
• Decision points / gates (CRDRD, IARD, IID, FID, ISD) → ch02, ch03, ch04, ch05, ch07
• Demonstration (method) → ch02, ch07
• Development Test (DT) → ch02, ch05, ch06
• Enterprise Architecture (EA) → ch01, ch03, ch04, ch06
• EA Control Board → ch03, ch04, ch06
• FIPS-199 (security categorization) → ch03
• Independence of reviewers → ch02
• Independent Evaluation Review (IER) → ch03
• Independent Operational Assessment (IOA) → ch02, ch05, ch06
• Information Security Guidance for System Acquisition (ISGSA) → ch03, ch04
• Information System Security (ISS) assessment → ch03, ch04
• In-Service Decision (ISD) → ch05, ch07
• In-Service Management (ISM) → ch06
• Integrated Baseline Review (IBR) → ch05
• Inspection (method) → ch02, ch07
• Investment Analysis (Initial & Final) → ch04
• Method catalogs (four) → ch02, ch07
• → ch07

Supporting Files

• glossary.md — alphabetical terms and acronyms (ACAT through V&V) with chapter refs.
• patterns.md — 11 reusable patterns (When / How / Trade-offs): split verify/validate, embed planning, gate on decision points, find-your-row, V&V the foundations & criteria, chain forward, match method to tier, demand method+artifact, two governance tracks, front-load safety/security, prove-the-change-and-re-prove-the-purpose.
• cheatsheet.md — decision rules, the four-catalog method matrix, phase/gate tables, the nine planning elements, the four checklist documents, requirement-quality bar, method codes & tells, the DT/OT/IOA triad, and anti-pattern tells & smells.

Scope & Limits

• Source version: the FAA AMS Lifecycle V&V Guidelines (verification and validation), Version 3.0, dated April 2017. A living document; the FAA states it is updated as better practices emerge, so check FAST for a current version.
• This is FAA acquisition-V&V doctrine, not general SE. It assumes the surrounding AMS lifecycle and the FAA Acquisition Management Policy as scaffolding — it does not define them (see the FAA SEM pack). It is not a general systems-engineering process model, a requirements-engineering manual, or a test-design textbook.
• Definitions are paraphrased, not canonical. The core verification/validation definitions trace to CMMI for Development v1.2 (SEI) — a copyrighted model the guideline cites; this pack paraphrases and never quotes it. For the authoritative process-area treatment, go to CMMI directly.
• Tables 1–6 are policy snapshots as of April 2017. The source repeatedly cautions that detailed, current phase guidelines and templates exist and take precedence over the published tables. Treat the table contents (criteria, stakeholders, owning offices, form/order numbers) as illustrative of the model, not as live authority.
• Thin on: detailed test-method design and statistics; specific tools, labs, and simulators (named only as planning elements); cost and schedule estimating; and anything outside FAA acquisition governance. Office names and org structures (e.g., ANG-E, ATO) reflect the 2017 FAA organization and may have changed.
• No source link published (per the repo's no-source-link policy); the source is identified textually by title, publisher (FAA), and version.