Overview:

Cloudera University’s Data Analyst Training course focuses on Apache Pig, Apache Hive, and Apache Impala. You will learn how to apply traditional data analytics and business intelligence skills to big data. Cloudera presents the tools data professionals need to access, manipulate, transform, and analyze complex data sets using SQL and familiar scripting languages.

Delivery Method and Course Duration:

Classroom: 4 days

Objectives:

Through instructor-led discussion and interactive, hands-on exercises, participants will navigate the Hadoop ecosystem, learning topics such as:

• The features that Pig, Hive, and Impala offer for data acquisition, storage, and analysis
• The fundamentals of Apache Hadoop and data ETL (extract, transform, load), ingestion, and processing with Hadoop
• How Pig, Hive, and Impala improve productivity for typical analysis tasks
• Joining diverse datasets to gain valuable business insight
• Performing real-time, complex queries on datasets

Intended Audience & Prerequisites:

This course is designed for data analysts, business intelligence specialists, developers, system architects, and database administrators. Knowledge of SQL is assumed, as is basic Linux command-line familiarity. Knowledge of at least one scripting language (e.g., Bash scripting, Perl, Python, Ruby) would be helpful but is not essential. Prior knowledge of Apache Hadoop is not required

Advance your ecosystem expertise:

Apache Pig applies the fundamentals of familiar scripting languages to the Hadoop cluster. Apache Hive makes transformation and analysis of complex, multi-structured data scalable in Hadoop. Cloudera Impala enables real-time interactive analysis of the data stored in Hadoop via a native SQL environment. Together, Pig, Hive, and Impala make multi-structured data accessible to analysts, database administrators, and others without Java programming expertise

Course outlines:

1.      Introduction

2.      Apache Hadoop Fundamentals

• The Motivation for Hadoop
• Hadoop Overview
• Data Storage: HDFS
• Distributed Data Processing: YARN, MapReduce, and Spark
• Data Processing and Analysis: Pig, Hive, and Impala
• Database Integration: Sqoop
• Other Hadoop Data Tools
• Exercise Scenarios

3.      Introduction to Apache Pig

• What is Pig?
• Pig’s Features
• Pig Use Cases
• Interacting with Pig

4.      Basic Data Analysis with Apache Pig

• Pig Latin Syntax
• Loading Data
• Simple Data Types
• Field Definitions
• Data Output
• Viewing the Schema
• Filtering and Sorting Data
• Commonly Used Functions

5.      Processing Complex Data with Apache Pig

• Storage Formats
• Complex/Nested Data Types
• Grouping
• Built-In Functions for Complex Data
• Iterating Grouped Data

6.      Multi-Dataset Operations with Apache Pig

• Techniques for Combining Datasets
• Joining Datasets in Pig
• Set Operations
• Splitting Datasets

7.      Apache Pig Troubleshooting and Optimization

• Troubleshooting Pig
• Logging
• Using Hadoop’s Web UI
• Data Sampling and Debugging
• Performance Overview
• Understanding the Execution Plan
• Tips for Improving the Performance of Pig Jobs

8.      Introduction to Apache Hive and Impala

• What is Hive?
• What is Impala?
• Why Use Hive and Impala?
• Schema and Data Storage
• Comparing Hive and Impala to Traditional Databases
• Use Cases

9.      Querying with Apache Hive and Impala

• Databases and Tables
• Basic Hive and Impala Query Language Syntax
• Data Types
• Using Hue to Execute Queries
• Using Beeline (Hive’s Shell)
• Using the Impala Shell

10.  Apache Hive and Impala Data Management

• Data Storage
• Creating Databases and Tables
• Loading Data
• Altering Databases and Tables
• Simplifying Queries with Views
• Storing Query Results

11.  Data Storage and Performance

• Partitioning Tables
• Loading Data into Partitioned Tables
• When to Use Partitioning
• Choosing a File Format
• Using Avro and Parquet File Formats

12.  Relational Data Analysis with Apache Hive and Impala

• Joining Datasets
• Common Built-In Functions
• Aggregation and Windowing

13.  Complex Data with Apache Hive and Impala

• Complex Data with Hive
• Complex Data with Impala

14.  Analyzing Text with Apache Hive and Impala

• Using Regular Expressions with Hive and Impala
• Processing Text Data with SerDes in Hive
• Sentiment Analysis and n-grams in Hive

15.  Apache Hive Optimization

• Understanding Query Performance
• Bucketing
• Indexing Data
• Hive on Spark

16.  Apache Impala Optimization

• How Impala Executes Queries
• Improving Impala Performance

17.  Extending Apache Hive and Impala

• Custom SerDes and File Formats in Hive
• Data Transformation with
• Custom Scripts in Hive
• User-Defined Functions
• Parameterized Queries

18.  Choosing the Best Tool for the Job

• Comparing Pig, Hive, Impala, and Relational Databases
• Which to Choose?

19.   Conclusion