Logstash 简明教程

Logstash - Collecting Logs

使用 shippers 从不同的服务器或数据源中收集日志。Shipper 是 Logstash 中安装的一个实例，它可以访问服务器日志并将其发送到特定的输出位置。

Logs from different servers or data sources are collected using shippers. A shipper is an instance of Logstash installed in the server, which accesses the server logs and sends to specific output location.

它主要向 Elasticsearch 发送输出以进行存储。Logstash 从以下来源获取输入——

It mainly sends the output to the Elasticsearch for storage. Logstash takes input from the following sources −

  1. STDIN

  2. Syslog

  3. Files

  4. TCP/UDP

  5. Microsoft windows Eventlogs

  6. Websocket

  7. Zeromq

  8. Customized extensions

Collecting Logs Using Apache Tomcat 7 Server

在这个示例中，我们使用 file 输入插件收集了安装在 Windows 中的 Apache Tomcat 7 服务器的日志并将其发送到另一个日志。

In this example, we are collecting logs of Apache Tomcat 7 Server installed in windows using the file input plugin and sending them to the other log.

logstash.conf

在此处，Logstash 配置为访问本地安装的 Apache Tomcat 7 的访问日志。file 插件的路径设置中使用了正则表达式模式来从日志文件中获取数据。它在其名称中包含“access”，它添加了一个 apache 类型，这样有助于在集中目的地源中区分 apache 事件和其他事件。最后，输出事件将显示在 output.log 中。

Here, Logstash is configured to access the access log of Apache Tomcat 7 installed locally. A regex pattern is used in path setting of the file plugin to get the data from the log file. This contains “access” in its name and it adds an apache type, which helps in differentiating the apache events from the other in a centralized destination source. Finally, the output events will be shown in the output.log.

input {
   file {
      path => "C:/Program Files/Apache Software Foundation/Tomcat 7.0/logs/*access*"
      type => "apache"
   }
}
output {
   file {
      path => "C:/tpwork/logstash/bin/log/output.log"
   }
}

Run Logstash

我们可以使用以下命令运行 Logstash。

We can run Logstash by using the following command.

C:\logstash\bin> logstash –f  Logstash.conf

Apache Tomcat Log

访问 Apache Tomcat 服务器及其 Web 应用 ( http://localhost:8080 ) 以生成日志。Logstash 实时读取日志中更新的数据，并按配置文件中指定的格式将其存储到 output.log 中。

Access the Apache Tomcat Server and its web apps (http://localhost:8080) to generate logs. The updated data in the logs are read by Logstash in real time and stashed in output.log as specified in configuration file.

Apache Tomcat 根据日期生成一个新的访问日志文件，并将访问事件记录到其中。在本例中，它是在 Apache Tomcat 的 logs 目录中的 localhost_access_log.2016-12-24.txt。

Apache Tomcat generates a new access log file according to date and logs the access events there. In our case, it was localhost_access_log.2016-12-24.txt in the logs directory of Apache Tomcat.

0:0:0:0:0:0:0:1 - - [
   25/Dec/2016:18:37:00 +0800] "GET / HTTP/1.1" 200 11418
0:0:0:0:0:0:0:1 - munish [
   25/Dec/2016:18:37:02 +0800] "GET /manager/html HTTP/1.1" 200 17472
0:0:0:0:0:0:0:1 - - [
   25/Dec/2016:18:37:08 +0800] "GET /docs/ HTTP/1.1" 200 19373
0:0:0:0:0:0:0:1 - - [
   25/Dec/2016:18:37:10 +0800] "GET /docs/introduction.html HTTP/1.1" 200 15399

output.log

你可以在输出事件中看到，添加了一个类型字段，事件显示在消息字段中。

You can see in the output events, a type field is added and the event is present in the message field.

{
   "path":"C:/Program Files/Apache Software Foundation/Tomcat 7.0/logs/
   localhost_access_log.2016-12-25.txt",
   "@timestamp":"2016-12-25T10:37:00.363Z","@version":"1","host":"Dell-PC",
   "message":"0:0:0:0:0:0:0:1 - - [25/Dec/2016:18:37:00 +0800] \"GET /
   HTTP/1.1\" 200 11418\r","type":"apache","tags":[]
}
{
   "path":"C:/Program Files/Apache Software Foundation/Tomcat 7.0/logs/
   localhost_access_log.2016-12-25.txt","@timestamp":"2016-12-25T10:37:10.407Z",
   "@version":"1","host":"Dell-PC",
   "message":"0:0:0:0:0:0:0:1 - munish [25/Dec/2016:18:37:02 +0800] \"GET /
   manager/html HTTP/1.1\" 200 17472\r","type":"apache","tags":[]
}
{
   "path":"C:/Program Files/Apache Software Foundation/Tomcat 7.0/logs/
   localhost_access_log.2016-12-25.txt","@timestamp":"2016-12-25T10:37:10.407Z",
   "@version":"1","host":"Dell-PC",
   "message":"0:0:0:0:0:0:0:1 - - [25/Dec/2016:18:37:08 +0800] \"GET /docs/
   HTTP/1.1\" 200 19373\r","type":"apache","tags":[]
}
{
   "path":"C:/Program Files/Apache Software Foundation/Tomcat 7.0/logs/
   localhost_access_log.2016-12-25.txt","@timestamp":"2016-12-25T10:37:20.436Z",
   "@version":"1","host":"Dell-PC",
   "message":"0:0:0:0:0:0:0:1 - - [25/Dec/2016:18:37:10 +0800] \"GET /docs/
   introduction.html HTTP/1.1\" 200 15399\r","type":"apache","tags":[]
}

Collecting Logs Using STDIN Plugin

在本部分中，我们将讨论使用 STDIN Plugin 收集日志的另一个示例。

In this section, we will discuss another example of collecting logs using the STDIN Plugin.

logstash.conf

这是一个非常简单的示例，在其中 Logstash 正在读取用户在标准输入中输入的事件。在本例中，它是命令提示符，它存储在 output.log 文件中的事件。

It is a very simple example, where Logstash is reading the events entered by the user in a standard input. In our case, it is the command prompt, which stores the events in the output.log file.

input {
   stdin{}
}
output {
   file {
      path => "C:/tpwork/logstash/bin/log/output.log"
   }
}

Run Logstash

我们可以使用以下命令运行 Logstash。

We can run Logstash by using the following command.

C:\logstash\bin> logstash –f  Logstash.conf

在命令提示符中写入以下文本：

Write the following text in the command prompt −

用户输入了以下两行。Logstash 以分隔符设置分隔事件，其默认值为“\n”。用户可以通过更改 file 插件中的分隔符的值来进行更改。

The user entered the following two lines. Logstash separates the events by the delimiter setting and its value by default is ‘\n’. The user can change by changing the value of the delimiter in the file plugin.

Tutorialspoint.com welcomes you
Simply easy learning

output.log

以下代码块显示了输出日志数据。

The following code block shows the output log data.

{
   "@timestamp":"2016-12-25T11:41:16.518Z","@version":"1","host":"Dell-PC",
   "message":"tutrialspoint.com welcomes you\r","tags":[]
}
{
   "@timestamp":"2016-12-25T11:41:53.396Z","@version":"1","host":"Dell-PC",
   "message":"simply easy learning\r","tags":[]
}