| name | nf-bids-architecture |
| description | nf-bids plugin architecture: runtime flow of Channel.fromBIDS, set handler routing, suffix mapping design, flat output shaping, cross-modal broadcasting. Use when navigating the codebase, tracing data flow, or modifying handlers/output structure. |
nf-bids Architecture
Package map
Source dirs live under src/main/groovy/nfneuro/<sub>/, but package declarations are
nfneuro.plugin.<sub> — except the plugin entry points, whose package is plain nfneuro.plugin
(dir src/main/groovy/nfneuro/plugin/).
| Package | Contents |
|---|
nfneuro.plugin | BidsPlugin (pf4j entry), BidsExtension (@Factory fromBIDS + operators), BidsFactory/BidsObserver (trace observer, banner only) |
nfneuro.plugin.channel | BidsChannelFactory (pre-flight checks), BidsHandler (orchestrator), operations/ (GroupTupleByOp, JoinByOp, CombineByOp, keys/) |
nfneuro.plugin.config | BidsConfigLoader, BidsConfigAnalyzer (set-type flags, loopOverEntities), BidsConfigValidator |
nfneuro.plugin.parser | BidsParser, LibBidsShWrapper (runs bundled libBIDS.sh), BidsValidator |
nfneuro.plugin.grouping | BaseSetHandler + PlainSetHandler, NamedSetHandler, SequentialSetHandler, MixedSetHandler |
nfneuro.plugin.model | BidsEntity, BidsFile, BidsDataset, BidsChannelData |
nfneuro.plugin.util | BidsLogger, BidsErrorHandler, BidsCsvParser, BidsEntityUtils, SuffixMapper, ParticipantsMetadataMerger |
Runtime flow of Channel.fromBIDS()
BidsHandler is the orchestrator (fluent builder: withConfig/withBidsDir/withOpts/withParser/ignite).
Its execute() runs these stages:
- Config load & analyze —
loadConfiguration(): BidsConfigLoader.load() (or loadDefaults()),
then SuffixMapper.suffixMapping(config) and BidsConfigAnalyzer →
analyzeConfiguration() (flags like hasNamedSets) and getLoopOverEntities().
- Parse dataset —
BidsParser.parseToDataset() via libBIDS.sh →
BidsDataset/BidsFile objects; dataset.loadParticipants() collects participants metadata.
- Route to set handlers —
processDatasets() instantiates one handler per set type present
in the analysis (hasNamedSets → NamedSetHandler, likewise sequential/mixed/plain) and
merges all handler queues into one DataflowQueue.
- Unify —
unifyResults(): group items by loop-over groupingKey, then bucket by
data-key fingerprint (set of configKey names). Same fingerprint = alternatives for one
slot; different fingerprints = independent streams, combined by cross-product
(computeOuterJoin) — outer-join behaviour.
- Cross-modal broadcasting —
applyCrossModalBroadcasting(): pools available data per
looping key, applyIncludeCrossModal() injects include_cross_modal requests, and
shouldKeepChannel() drops groups whose data was only requested by others.
- Flatten & emit —
validateAndEmitChannel(): unless flatten_output: false,
flattenTupleToMap() builds [meta: {...}, <configKey>: {...}, ...] — meta gets loop-over
entity values plus participants metadata; configKey maps sit at the top level (not under
data). All file paths become java.nio.file.Path via FileHelper.asPath() (never
java.io.File); optional unpack_json_sidecar parses JSON sidecars into maps. Fails if
zero groups were emitted; channel closed only by poison pill Channel.STOP.
Full trace: documentation/modules/ROOT/pages/source-model/runtime-flow.adoc.
Suffix mapping design (suffix_maps_to)
SuffixMapper.suffixMapping(config) builds setType → configKey → targetSuffix
(e.g. named_set → dwi_fullreverse → dwi). The direction is deliberate: configKey is unique
(YAML top-level key), fileSuffix is not — an inverted suffix → configKey map would lose
entries when several configKeys map to one suffix.
resolveConfigKeys(setType, suffix, mapping) searches the map values and returns all
candidate configKeys for a file suffix (falls back to [suffix] when unmapped).BaseSetHandler iterates the candidates and validates each against the set config's
filters, exclude_entities, and required_entities before assigning the file.- configKey is used for OUTPUT map keys; targetSuffix for INPUT file matching.
Async model
ignite(session) creates the output channel via CH.create(), then (DSL2) defers work with
session.addIgniter { perform(true) } — the channel is consumable before parsing starts.perform(async=true) runs execute() in a CompletableFuture; failures abort the session
via handlerException.- Intermediate results move through unbound GPars
DataflowQueues between stages; only
validateAndEmitChannel() binds to the target channel and sends Channel.STOP.
Diagrams
PlantUML sources in documentation/diagrams:
class-core.puml,
package-dependencies.puml,
libbids-integration.puml,
sequence-frombids.puml.
Rendered gallery: documentation/modules/ROOT/pages/diagrams.adoc.
Deep-dive docs
Output shapes per handler
The top-level data key is always the configKey, never the BIDS file suffix. All file values are absolute java.nio.file.Path objects.
PlainSetHandler
[meta: [subject: 'sub-01', ...], T1w: [nii: Path(...), json: Path(...)]]
// Access: item.T1w.nii
NamedSetHandler (e.g. configKey dwi_ap with suffix_maps_to: "dwi")
[meta: [...], dwi_ap: [ap: [nii: Path, bval: Path, bvec: Path], pa: [...]]]
// Access: item.dwi_ap.ap.nii item.dwi_ap.pa.bval
SequentialSetHandler (echoes as arrays)
[meta: [...], mese: [nii: [Path1, Path2, ...], json: [Path1, Path2, ...]]]
// Access: item.mese.nii[0] item.mese.nii.size()
MixedSetHandler (named groups each containing sequences)
[meta: [...], mpm: [MTw: [nii: [Path, Path], json: [Path, Path]], PDw: [...]]]
// Access: item.mpm.MTw.nii[0]
Cross-modal broadcasting
include_cross_modal: [T1w] inside a set body copies T1w data into this group's emitted item.
A configKey whose data is only requested by others via include_cross_modal is dropped from output — it exists solely to provide data to the requesting group.
Config analysis flags
BidsConfigAnalyzer extracts four booleans used by BidsHandler.processDatasets to route files:
hasNamedSets, hasSequentialSets, hasMixedSets, hasPlainSets.
Legacy format (deprecated)
flatten_output: false option emits [groupingKey, enrichedMap] tuples with relative string paths instead of flat maps with Path objects. This format was deprecated and removed in a later version.
