Apache Pig 简明教程

Apache Pig - Diagnostic Operators

load 语句将简单地把数据加载到 Apache Pig 中指定的关系中。要验证 Load 语句的执行，你必须使用 Diagnostic Operators 。Pig Latin 提供四种不同的诊断操作符 −

  1. Dump operator

  2. Describe operator

  3. Explanation operator

  4. Illustration operator

在本章中，我们将讨论 Pig Latin 的 Dump 操作符。

Dump Operator

Dump 操作符用于运行 Pig Latin 语句并在屏幕上显示结果。它通常用于调试目的。

Syntax

下面给出 Dump 操作符的语法。

grunt> Dump Relation_Name

Example

假设我们在 HDFS 中有一个文件 student_data.txt ，内容如下。

001,Rajiv,Reddy,9848022337,Hyderabad
002,siddarth,Battacharya,9848022338,Kolkata
003,Rajesh,Khanna,9848022339,Delhi
004,Preethi,Agarwal,9848022330,Pune
005,Trupthi,Mohanthy,9848022336,Bhuwaneshwar
006,Archana,Mishra,9848022335,Chennai.

并且我们已使用 LOAD 运算符，将它读入到一个关系 student 中，如下所示。

grunt> student = LOAD 'hdfs://localhost:9000/pig_data/student_data.txt'
   USING PigStorage(',')
   as ( id:int, firstname:chararray, lastname:chararray, phone:chararray,
   city:chararray );

现在，我们使用 Dump operator 打印关系的内容，如下所示。

grunt> Dump student

一旦执行了上述 Pig Latin 语句，它将启动 MapReduce 作业以从 HDFS 读取数据。它将生成以下输出。

2015-10-01 15:05:27,642 [main]
INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher -
100% complete
2015-10-01 15:05:27,652 [main]
INFO  org.apache.pig.tools.pigstats.mapreduce.SimplePigStats - Script Statistics:
HadoopVersion  PigVersion  UserId    StartedAt             FinishedAt       Features
2.6.0          0.15.0      Hadoop  2015-10-01 15:03:11  2015-10-01 05:27     UNKNOWN

Success!
Job Stats (time in seconds):

JobId           job_14459_0004
Maps                 1
Reduces              0
MaxMapTime          n/a
MinMapTime          n/a
AvgMapTime          n/a
MedianMapTime       n/a
MaxReduceTime        0
MinReduceTime        0
AvgReduceTime        0
MedianReducetime     0
Alias             student
Feature           MAP_ONLY
Outputs           hdfs://localhost:9000/tmp/temp580182027/tmp757878456,

Input(s): Successfully read 0 records from: "hdfs://localhost:9000/pig_data/
student_data.txt"

Output(s): Successfully stored 0 records in: "hdfs://localhost:9000/tmp/temp580182027/
tmp757878456"

Counters: Total records written : 0 Total bytes written : 0 Spillable Memory Manager
spill count : 0Total bags proactively spilled: 0 Total records proactively spilled: 0

Job DAG: job_1443519499159_0004

2015-10-01 15:06:28,403 [main]
INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLau ncher - Success!
2015-10-01 15:06:28,441 [main] INFO  org.apache.pig.data.SchemaTupleBackend -
Key [pig.schematuple] was not set... will not generate code.
2015-10-01 15:06:28,485 [main]
INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths
to process : 1
2015-10-01 15:06:28,485 [main]
INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths
to process : 1

(1,Rajiv,Reddy,9848022337,Hyderabad)
(2,siddarth,Battacharya,9848022338,Kolkata)
(3,Rajesh,Khanna,9848022339,Delhi)
(4,Preethi,Agarwal,9848022330,Pune)
(5,Trupthi,Mohanthy,9848022336,Bhuwaneshwar)
(6,Archana,Mishra,9848022335,Chennai)