105 – Apache NiFi Controller Services

This course assumes you are familiar with the basic of Apache NiFi – Read here

You can follow along using our Auto-Launching NiFi – Learn How Here

Overview

Apache NiFi has the ability to control large groups of processors with “Controller Services”. Controller Services are extremely useful for centralizing information and configurations such as service credentials or data schemas. In one of our previous blogs when NiFi was accessing an AWS S3 bucket we put the key/secret in the processor directly. This approach can be problematic at scale due to each individual developer needing access to the key/secret or at least the keyfile. A more secure way to achieve this would be to use the “AWS Credentials Provider Service”

Setting up A Controller Service

There are three main ways to access the controller service menu

From the NiFi Canvas – Right click > Configure > Select the Controller Services Tab
From a Processor Group – Right click > Configure > Select the Controller Services Tab
From a processor – detailed blow

A Better S3 Connection

In the previously mentioned blog we accessed S3 on individual processor configurations. Let us go through Configuring the same flow using a controller service. If you right click to configure the PutS3Object processor you can click on the field for “AWS Credentials Provider Service” and select the dropdown to “Create new Service”

You will have the option to create many different Controller Services but the correct AWS Credentials one will already be selected so simply press “Create”. Congrats, you have now created a Controller Service.

You can follow along using our Auto-Launching NiFi – Learn How Here

Configuring the Controller Service

To input your credentials you need to configure the service. A shortcut to this would be to click the arrow that appears to the right of the field and click “yes” if prompted to save changes. This will bring you to the Controller Services menu. From here you can configure(gear), activate(lightning), or delete(trash) any of the services.

Clicking the gear will bring up a menu very similar to a processor’s in which you can enter properties for the NiFi key/secret or file. When done click apply and the lightning bolt to start the service. On the service start screen you will be prompted if you want to start all processors associated with the service or just the service itself.

Referencing the Service

Any processor in the same root canvas or nested processor group can reference the service. If a service exists in a processor group anything not nested inside of that group will be unable to reference it. This enables teams and administrators to have both inherited credentials along with isolated ones for security.