Apache Nifi 简明教程
Apache NiFi - Monitoring
在 Apache NiFi 中，有多种方式可以监控系统中的不同统计信息，例如错误、内存使用情况、CPU 使用情况、数据流统计信息等。我们将在本教程中讨论最流行的统计信息。
In Apache NiFi, there are multiple ways to monitor the different statistics of the system like errors, memory usage, CPU usage, Data Flow statistics, etc. We will discuss the most popular ones in this tutorial.
In built Monitoring
在本节中，我们将进一步了解 Apache NiFi 中内置的监控功能。
In this section, we will learn more about in built monitoring in Apache NiFi.
Bulletin Board
公告栏以实时方式显示 NiFi 处理器生成的最新 ERROR 和 WARNING。要访问公告栏，用户必须转到右边的下拉菜单，然后选择“公告栏”选项。它会自动刷新，用户还可以禁用它。用户还可以通过双击错误导航到实际的处理器。用户也可以通过以下方法过滤公告：
The bulletin board shows the latest ERROR and WARNING getting generated by NiFi processors in real time. To access the bulletin board, a user will have to go the right hand drop down menu and select the Bulletin Board option. It refreshes automatically and a user can disable it also. A user can also navigate to the actual processor by double-clicking the error. A user can also filter the bulletins by working out with the following −
-
by message
-
by name
-
by id
-
by group id
Data provenance UI
要监控在任何特定处理器或整个 NiFi 中发生的事件，用户可以从公告栏的同一菜单访问数据来源。用户还可以通过以下字段过滤数据来源存储库中的事件：
To monitor the Events occurring on any specific processor or throughout NiFi, a user can access the Data provenance from the same menu as the bulletin board. A user can also filter the events in data provenance repository by working out with the following fields −
-
by component name
-
by component type
-
by type
NiFi Summary UI
Apache NiFi 摘要也可以从公告栏的同一菜单访问。此 UI 包含有关该特定 NiFi 实例或集群的所有组件的信息。它们可以按名称、类型或 URI 进行筛选。不同的组件类型有不同的选项卡。以下是在 NiFi 摘要 UI 中可以监控的组件：
Apache NiFi summary also can be accessed from the same menu as the bulletin board. This UI contains information about all the components of that particular NiFi instance or cluster. They can be filtered by name, by type or by URI. There are different tabs for different component types. Following are the components, which can be monitored in the NiFi summary UI −
-
Processors
-
Input ports
-
Output ports
-
Remote process groups
-
Connections
-
Process groups
在此 UI 中，在右下角有一个名为系统诊断的链接，用于检查 JVM 统计信息。
In this UI, there is a link at the bottom right hand side named system diagnostics to check the JVM statistics.
Reporting Tasks
Apache NiFi 提供多项报告任务，以支持外部监控系统，如 Ambari、Grafana 等。开发人员可以创建一个自定义报告任务或配置内置的任务，以便将 NiFi 的指标发送到外部监控系统。下表列出了 NiFi 1.7.1 提供的报告任务。
Apache NiFi provides multiple reporting tasks to support external monitoring systems like Ambari, Grafana, etc. A developer can create a custom reporting task or can configure the inbuilt ones to send the metrics of NiFi to the externals monitoring systems. The following table lists down the reporting tasks offered by NiFi 1.7.1.
S.No. |
Reporting Task Name |
Description |
1 |
AmbariReportingTask |
To setup Ambari Metrics Service for NiFi. |
2 |
ControllerStatusReportingTask |
To report the information from the NiFi summary UI for the last 5 minute. |
3 |
MonitorDiskUsage |
To report and warn about the disk usage of a specific directory. |
4 |
MonitorMemory |
To monitor the amount of Java Heap used in a Java Memory pool of JVM. |
5 |
SiteToSiteBulletinReportingTask |
To report the errors and warning in bulletins using Site to Site protocol. |
6 |
SiteToSiteProvenanceReportingTask |
To report the NiFi Data Provenance events using Site to Site protocol. |
NiFi API
有一个名为系统诊断的 API，可用于在任何自定义开发应用程序中监视 NiFi 统计信息。让我们在 Postman 中检查 API。
There is an API named system diagnostics, which can be used to monitor the NiFI stats in any custom developed application. Let us check the API in postman.
Response
{
"systemDiagnostics": {
"aggregateSnapshot": {
"totalNonHeap": "183.89 MB",
"totalNonHeapBytes": 192819200,
"usedNonHeap": "173.47 MB",
"usedNonHeapBytes": 181894560,
"freeNonHeap": "10.42 MB",
"freeNonHeapBytes": 10924640,
"maxNonHeap": "-1 bytes",
"maxNonHeapBytes": -1,
"totalHeap": "512 MB",
"totalHeapBytes": 536870912,
"usedHeap": "273.37 MB",
"usedHeapBytes": 286652264,
"freeHeap": "238.63 MB",
"freeHeapBytes": 250218648,
"maxHeap": "512 MB",
"maxHeapBytes": 536870912,
"heapUtilization": "53.0%",
"availableProcessors": 4,
"processorLoadAverage": -1,
"totalThreads": 71,
"daemonThreads": 31,
"uptime": "17:30:35.277",
"flowFileRepositoryStorageUsage": {
"freeSpace": "286.93 GB",
"totalSpace": "464.78 GB",
"usedSpace": "177.85 GB",
"freeSpaceBytes": 308090789888,
"totalSpaceBytes": 499057160192,
"usedSpaceBytes": 190966370304,
"utilization": "38.0%"
},
"contentRepositoryStorageUsage": [
{
"identifier": "default",
"freeSpace": "286.93 GB",
"totalSpace": "464.78 GB",
"usedSpace": "177.85 GB",
"freeSpaceBytes": 308090789888,
"totalSpaceBytes": 499057160192,
"usedSpaceBytes": 190966370304,
"utilization": "38.0%"
}
],
"provenanceRepositoryStorageUsage": [
{
"identifier": "default",
"freeSpace": "286.93 GB",
"totalSpace": "464.78 GB",
"usedSpace": "177.85 GB",
"freeSpaceBytes": 308090789888,
"totalSpaceBytes": 499057160192,
"usedSpaceBytes": 190966370304,
"utilization": "38.0%"
}
],
"garbageCollection": [
{
"name": "G1 Young Generation",
"collectionCount": 344,
"collectionTime": "00:00:06.239",
"collectionMillis": 6239
},
{
"name": "G1 Old Generation",
"collectionCount": 0,
"collectionTime": "00:00:00.000",
"collectionMillis": 0
}
],
"statsLastRefreshed": "09:30:20 SGT",
"versionInfo": {
"niFiVersion": "1.7.1",
"javaVendor": "Oracle Corporation",
"javaVersion": "1.8.0_151",
"osName": "Windows 7",
"osVersion": "6.1",
"osArchitecture": "amd64",
"buildTag": "nifi-1.7.1-RC1",
"buildTimestamp": "07/12/2018 12:54:43 SGT"
}
}
}
}