Apache Nifi 简明教程

Apache NiFi - Data Provenance

Apache NiFi 会记录和存储有关流中已摄取数据上发生的事件的每个信息。数据来源存储库会存储此信息,并提供界面来搜索此事件信息。既可以访问针对整个 NiFi 层级的数据来源,也可以访问针对处理程序层级的数据来源。

Apache NiFi logs and store every information about the events occur on the ingested data in the flow. Data provenance repository stores this information and provides UI to search this event information. Data provenance can be accessed for full NiFi level and processor level also.

data provenance

下表列出了 NiFi 数据来源事件列表中各个字段,这些字段包括:

The following table lists down the different fields in the NiFi Data Provenance event list have following fields −

S.No.

Field Name

Description

1

Date/Time

Date and time of event.

2

Type

Type of Event like ‘CREATE’.

3

FlowFileUuid

UUID of the flowfile on which the event is performed.

4

Size

Size of the flowfile.

5

Component Name

Name of the component which  performed the event.

6

Component Type

Type of the component.

7

Show lineage

Last column has the show lineage icon, which is used to see the flowfile lineage as shown in the below image.

lineage icon

要获取有关该事件的更多信息,用户可以单击 NiFi Data Provenance UI 第一列中显示的信息图标。

To get more information about the event, a user can click on the information icon present in the first column of the NiFi Data Provenance UI.

nifi.properties 文件中有一些属性用于管理 NiFi Data Provenance 存储库。

There are some properties in nifi.properties file, which are used to manage NiFi Data Provenance repository.

S.No.

Property Name

Default Value

Description

1

nifi.provenance.repository.directory.default

./provenance_repository

To specify the default path of NiFi data provenance .

2

nifi.provenance.repository.max.storage.time

24 hours

To specify the maximum retention time of NiFi data provenance.

3

nifi.provenance.repository.max.storage.size

1 GB

To specify the maximum storage of NiFi data provenance.

4

nifi.provenance.repository.rollover.time

30 secs

To specify the rollover time of NiFi data provenance.

5

nifi.provenance.repository.rollover.size

100 MB

To specify the rollover size of NiFi data provenance.

6

nifi.provenance.repository.indexed.fields

EventType, FlowFileUUID, Filename, ProcessorID, Relationship

To specify the fields used to search and index NiFi data provenance.