Apache NiFi: FlowFiles, Processors & Connections

Apache NiFi automates and manages the flow of data between systems using a visual, drag-and-drop interface -- routing, transforming, and moving data without requiring custom code for most common data-movement tasks.

FlowFiles: The Unit of Data

Every piece of data moving through NiFi is a FlowFile -- content (the actual data) plus attributes (key-value metadata) that processors inspect and modify as it moves through the flow.

Processors & Connections

Processors are the building blocks (GetFile, RouteOnAttribute, ConvertRecord) dragged onto the canvas, each performing a discrete operation. Connections wire processors together, defining how FlowFiles flow from one processor's output relationship (success, failure, retry) to the next processor's input.

Back Pressure

If an upstream processor produces FlowFiles faster than a downstream one consumes them, back pressure limits how much can queue on that connection -- protecting the system from unbounded growth and giving operators visibility into the bottleneck.

Process Groups

A complex flow can be broken into smaller, named, collapsible process groups -- similar to functions or modules in code, improving readability and letting related logic be reused or managed as one unit.