job-provenance

name	job-provenance
description	Retrieve W3C PROV provenance metadata tracking the complete execution lineage and data derivation. Includes information about inputs, outputs, processes, agents, and temporal relationships for reproducibility and data lineage tracking.
license	Apache-2.0
compatibility	Requires Weaver API access with provenance feature enabled (weaver.cwl_prov=true).
metadata	{"author":"fmigneault"}

Get Job Provenance

Retrieve W3C PROV provenance metadata for tracking execution lineage and data derivation.

When to Use

• Tracking data lineage and derivation
• Ensuring reproducibility of results
• Auditing and compliance requirements
• Understanding workflow execution paths
• Documenting research workflows
• Publishing scientific results with provenance

Parameters

Required

• job_id (string): Job identifier

Optional

• format (string): Provenance format
  • json: JSON-LD format (default)
  • xml: PROV-XML format
  • turtle: RDF Turtle format
  • rdf: RDF/XML format

CLI Usage

# Get provenance in JSON-LD
weaver provenance -u $WEAVER_URL -j a1b2c3d4-e5f6-7890-abcd-ef1234567890

# Get provenance in RDF Turtle
weaver provenance -u $WEAVER_URL -j a1b2c3d4-e5f6-7890-abcd-ef1234567890 -f turtle

# Save to file
weaver provenance -u $WEAVER_URL -j a1b2c3d4-e5f6-7890-abcd-ef1234567890 -o provenance.jsonld

Python Usage

from weaver.cli import WeaverClient

client = WeaverClient(url="https://weaver.example.com")

# Get provenance
prov = client.provenance(job_id="a1b2c3d4-e5f6-7890-abcd-ef1234567890")

# Access provenance entities
for entity in prov.body.get("entities", []):
    print(f"Entity: {entity['id']}")

# Get provenance in specific format
prov_turtle = client.provenance(
    job_id="a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    format="turtle"
)

API Request

GET /jobs/a1b2c3d4-e5f6-7890-abcd-ef1234567890/prov
Accept: application/ld+json

Returns

W3C PROV document with:

{
  "@context": "https://www.w3.org/ns/prov",
  "entity": [
    {
      "@id": "input-file-1",
      "@type": "prov:Entity",
      "prov:atLocation": "https://example.com/input.nc",
      "prov:generatedAtTime": "2026-02-19T10:00:00Z"
    }
  ],
  "activity": [
    {
      "@id": "execution-1",
      "@type": "prov:Activity",
      "prov:startedAtTime": "2026-02-19T10:00:05Z",
      "prov:endedAtTime": "2026-02-19T10:05:20Z",
      "prov:used": {"@id": "input-file-1"}
    }
  ],
  "wasGeneratedBy": [
    {
      "prov:entity": {"@id": "output-file-1"},
      "prov:activity": {"@id": "execution-1"}
    }
  ]
}

Provenance Elements

Entities

• Input files and data
• Output files and results
• Intermediate data products
• Configuration files

Activities

• Process executions
• Workflow steps
• Data transformations
• Service invocations

Agents

• Software tools and versions
• Computing infrastructure
• Users and organizations

Relationships

• wasGeneratedBy: Output generated by activity
• used: Activity used entity as input
• wasAssociatedWith: Activity associated with agent
• wasDerivedFrom: Entity derived from another entity
• wasInformedBy: Activity informed by another activity

Use Cases

Scientific Reproducibility

# Extract complete workflow lineage
prov = client.provenance(job_id="a1b2c3d4-e5f6-7890-abcd-ef1234567890")

# Document software versions used
for agent in prov.body.get("agents", []):
    print(f"Software: {agent['label']} v{agent['version']}")

Data Lineage Tracking

# Trace data derivation chain
for derivation in prov.body.get("wasDerivedFrom", []):
    print(f"{derivation['entity']} derived from {derivation['source']}")

Compliance and Auditing

# Export provenance for archival
weaver provenance -u $WEAVER_URL -j $JOB_ID -f xml -o compliance/job-$JOB_ID-prov.xml

Related Skills

• job-status - Check job status
• job-results - Retrieve outputs
• job-execute - Run processes with provenance tracking

Documentation

• Provenance Tracking
• W3C PROV Overview
• Configuration
• CLI Reference

Note

Provenance tracking must be enabled in Weaver configuration (weaver.cwl_prov=true). Jobs executed without this setting will not have provenance data available.