Maven Projects for Extensions
Overview
Apache NiFi extensions are packaged in NARs (NiFi archives). A NAR allows several components and their dependencies to be packaged together into a single package. The NAR package is then provided ClassLoader isolation from other NAR packages. See the Developer Guide for more detailed information on NARs.
Processor Projects
Apache NiFi processors are typically organized in processor bundles. A processor bundle is generally composed of the following:
- A Maven project that produces a jar of processors
- A Maven project that packages the processors into a NAR
- Parent pom for the bundle that builds the processor and NAR projects
An example processor bundle from the NiFi distribution is the nifi-kafka-bundle, containing the sub-projects nifi-kafka-processors and nifi-kafka-nar.
Maven Processor Archetype
NiFi provides a Maven archetype for easily creating the processor bundle project structure. To use the archetype, run the following Maven command:
mvn archetype:generate -DarchetypeGroupId=org.apache.nifi -DarchetypeArtifactId=nifi-processor-bundle-archetype -DarchetypeVersion=1.11.4 -DnifiVersion=1.11.4
The archetypeVersion corresponds to the version of the nifi-processor-bundle-archetype that is being used, currently this version is the same as the top-level NiFi version (i.e. for the NiFi 0.7.0 release, the archetypeVersion would be 0.7.0, and so on). The nifiVersion is the version of NiFi that the new processors will depend on.
NOTE: The latest released version of the archetype can always be found by checking the Maven central repositories. If you build NiFi from source you will also have access to the latest version of the archetype through a SNAPSHOT dependency.
Running the above command will prompt you for the following properties:
| Property | Description |
|---|---|
| groupId | The Maven groupId for the new bundle. |
| artifactId | The Maven artifactId for the new bundle. Generally something like "nifi-basename-bundle", where basename is specific to your bundle. |
| version | The Maven version for the new bundle. |
| artifactBaseName | The base name from the artifactId. For example, to create a helloworld bundle, the artifactBaseName would be helloworld, and the artifactId would be nifi-helloworld-bundle. |
| package | The Java package for the processors. |
Example Processor Bundle Structure
A processor bundle created with the archetype will create the following project structure:
nifi-basename-bundle├── nifi-basename-nar│ └── pom.xml├── nifi-basename-processors│ ├── pom.xml│ └── src│ ├── main│ │ ├── java│ │ │ └── org.apache.nifi.processors.basename│ │ │ └── MyProcessor.java│ │ └── resources│ │ ├── META-INF│ │ │ └── services│ │ └── org.apache.nifi.processor.Processor│ └── test│ └── java│ └── org.apache.nifi.processors.basename│ └── MyProcessorTest.java└── pom.xml
The default processor generated by the archetype is called MyProcessor. If you decide to rename this processor, make sure to also update the org.apache.nifi.processor.Processor file. This file is used by NiFi to locate available processors, and the fully qualified class name of the processor must be listed here in order for it to be available from the user interface.
Controller Service Projects
NiFi provides a mechanism to share common functionality through the concept of ControllerServices. A ControllerService is typically composed of the following:
- A Maven project providing the API (i.e. the Java interface that extends the ControllerService interface)
- A Maven project that packages the API in NAR
- A Maven project providing an implementation of the above API
- A Maven project that packages the implementation in a NAR
Maven Controller Service Archetype
NiFi provides a Maven archetype for easily creating the controller service project structure. To use the archetype, run the following Maven command:
mvn archetype:generate -DarchetypeGroupId=org.apache.nifi -DarchetypeArtifactId=nifi-service-bundle-archetype -DarchetypeVersion=1.11.4 -DnifiVersion=1.11.4
The archetypeVersion corresponds to the version of the nifi-service-bundle-archetype that is being used, currently this version is the same as the top-level NiFi version (i.e. for the NiFi 0.7.0 release, the archetypeVersion would be 0.7.0, and so on). The nifiVersion is the version of NiFi that the new controller service will depend on.
Running the above command will prompt you for the following properties:
| Property | Description |
|---|---|
| groupId | The Maven groupId for the new bundle. |
| artifactId | The Maven artifactId for the new bundle. Generally something like "nifi-something-services", where "something" is specific to your bundle. |
| version | The Maven version for the new bundle. |
| artifactBaseName | The base name from the artifactId. For example, to create a nifi-helloworld-services bundle, the artifactBaseName would be helloworld-service, and the artifactId would be nifi-helloworld-services. |
| package | The Java package for the services and API. |
Example Controller Service Structure
Running the archetype generate command with an artifactId of nifi-example-services and an artifactBaseName of example-service produces the following structure:
nifi-example-services├── nifi-example-service│ ├── pom.xml│ └── src│ ├── main/java│ │ │ └── org.apache.nifi.example.service│ │ │ └── StandardMyService.java│ │ └── resources│ │ └── META-INF│ │ └── services│ │ └── org.apache.nifi.controller.ControllerService│ └── test/java│ └── org.apache.nifi.example.service│ ├── TestProcessor.java│ └── TestStandardMyService.java├── nifi-example-service-api│ ├── pom.xml│ └── src│ └── main/java│ └── org.apache.nifi.example.service│ └── MyService.java├── nifi-example-service-api-nar│ └── pom.xml├── nifi-example-service-nar│ └── pom.xml└── pom.xml
The dependency relationship between these projects is the following:
nifi-example-service-nar -> nifi-example-service-api-nar -> nifi-standard-services-api-nar
These dependencies are all of type NAR which allows a NAR to access classes from another NAR at runtime. A NAR can only have one NAR dependency.
Linking Processors and Controller Services
To leverage a ControllerService, processor projects will need a provided dependency on the given API in order to compile, and the NAR will need a NAR dependency on the given API NAR in order for the API to be available at runtime.
As an example, to leverage the SSLContextService from a new processor bundle generated with the archetype, the following dependency would be added to the pom of the processors jar project:
<dependency><groupId>org.apache.nifi</groupId><artifactId>nifi-ssl-context-service-api</artifactId><version>1.11.4</version><scope>provided</scope> </dependency>
Then the following NAR dependency would be added to pom of the processor's NAR project:
<dependency><groupId>org.apache.nifi</groupId><artifactId>nifi-standard-services-api-nar</artifactId><version>1.11.4</version><type>nar</type></dependency>
A processor can then define the SSLContextService with a property as follows:
public static final PropertyDescriptor SSL_CONTEXT_SERVICE = new PropertyDescriptor.Builder().name("SSL Context Service").description("The Controller Service to use in order to obtain an SSL Context").required(false).identifiesControllerService(SSLContextService.class).build();
An instance of the service can then be obtained as follows:
final SSLContextService sslContextService = context.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextService.class);
The service returned from the context will correspond with the implementation that was configured in the NiFi user interface. Configuring ControllerServices through the user interface is described in detail in the user guide.
In addition to the ControllerServices that come with Apache NiFi, alternate implementations, or additional services, can be developed by providing a custom NAR with an implementation of the given service. After deploying the NAR and restarting NiFi, the new implementation would be available from the NiFi user interface.
The nifi-standard-services module contains the shared APIs that can be leveraged. Browsing this module can provide the groupId, artifactId, and version of the various APIs.
Inheritance
By default, projects generated from the archetypes will inherit from the nifi-nar-bundles artifact from Apache NiFi, for the given nifiVersion that was specified. This is well suited for bundles that plan to be included with Apache NiFi, or plan to be Apache licensed, but it may not be desired in all cases. If it is preferred to not inherit from the nifi-nar-bundles artifact, then the newly generated bundle can be modified in the following ways:
- Remove the <parent>...</parent> from the bundle's root pom, or replace it with another desired parent
-
Add the nar plugin to bundle's root pom (ensure the latest version of the nar plugin is being used):
<build><plugins><plugin><groupId>org.apache.nifi</groupId><artifactId>nifi-nar-maven-plugin</artifactId><version>1.3.1</version><extensions>true</extensions></plugin></plugins></build> - Specify the necessary dependency versions in the bundle's processor or service pom. This is required because the default behavior is for the dependencies to inherit their versions through the nifi-nar-bundle parent, so with a different parent these versions will have to be explicitly set. In addition, the nifi-nar-bundle parent normally sets the nifi-api dependency to provided, so this should be explicitly set in the processors pom.
Deployment
Any bundle can be deployed by copying the NAR to the NiFi lib directory and restarting NiFi. If you increment the version of your NAR, be sure to remove the older version from the NiFi lib directory to avoid running two versions of the same artifact.