Master Big Data - Apache Spark/Hadoop/Sqoop/Hive/Flume/Mongo

Focused View

Navdeep Kaur

10:57:25

39 View

01 - Big Data Introduction

001 Course Intro.mp4

01:26

002 Big Data Intro.mp4

05:24

003 Understanding Big Data Ecosystem.mp4

10:28

02 - Environment Setup

001 GCP Cluster Fixes.html

002 Cluster Setup on Google Cloud.mp4

21:20

002 gcp-cluster.txt

002 resources.zip

002 retail-db.zip

003 Environment Update.mp4

00:42

03 - Hadoop And Yarn

001 HDFS and Hadoop Commands.mp4

09:16

002 Yarn Cluster Overview.mp4

07:41

04 - Sqoop Import

001 Sqoop Introduction.mp4

15:48

001 Sqoop-Commands.docx

001 Sqoop-Import-pdf.pdf

002 Managing Target Directories.mp4

07:26

002 Sqoop-Import.docx

003 Working with Parquet File Format.mp4

08:24

003 parquet-tools-1.10.0.zip

004 Working with Avro File Format.mp4

11:35

004 avro-tools-1.8.2.zip

005 Working with Different Compressions.mp4

10:08

006 Conditional Imports.mp4

04:26

007 Split-by and Boundary Queries.mp4

08:27

008 Field delimeters.mp4

03:18

009 Incremental Appends.mp4

11:38

010 Sqoop-Hive Cluster Fix.html

011 Sqoop Hive Import.mp4

03:31

012 Sqoop List TablesDatabase.mp4

04:13

013 Sqoop Import Practice1.mp4

04:57

013 Sqoop-Import-Practice.docx

013 Sqoop-Import-Practice-Sol.docx

014 Sqoop Import Practice2.mp4

03:32

05 - Sqoop Export

001 Export from Hdfs to Mysql.mp4

03:39

001 sqoop-export.zip

002 Export from Hive to Mysql.mp4

02:30

003 Export Avro Compressed to Mysql.mp4

07:30

004 Bonus Lecture Sqoop with Airflow.mp4

02:57

06 - Apache Flume

001 Flume Introduction And Architecture.mp4

10:07

001 flume-config-resources-20200505T024933Z-001.zip

002 Exec Source and Logger Sink.mp4

03:41

003 Moving data from Twitter to HDFS.mp4

09:25

004 Moving data from NetCat to HDFS.mp4

04:39

005 Flume Interceptors.mp4

01:56

006 Flume Interceptor Example.mp4

04:53

007 Flume Multi-Agent Flow.mp4

06:49

008 Flume Consolidation.mp4

06:11

07 - Apache Hive

001 Hive Introduction.mp4

03:41

001 Hive-pdf.pdf

002 Hive Database.mp4

08:30

002 Hive-Practice-pdf.pdf

003 Hive Managed Tables.mp4

06:23

004 Hive External Tables.mp4

02:26

005 Hive Inserts.mp4

05:30

006 Hive Analytics.mp4

04:21

007 Working with Parquet.mp4

03:30

008 Compressing Parquet.mp4

04:27

009 Working with Fixed File Format.mp4

03:04

010 Alter Command.mp4

06:12

011 Hive String Functions.mp4

06:22

012 Hive Date Functions.mp4

05:40

013 Hive Partitioning.mp4

07:16

014 Hive Bucketing.mp4

03:45

08 - Spark with Yarn And HDFS

001 What is Apache Spark.mp4

02:47

002 Understanding Cluster Manager (Yarn).mp4

04:25

003 Understanding Distributed Storage (HDFS).mp4

03:38

004 Running Spark on YarnHDFS.mp4

08:31

005 Understanding Deploy Modes.mp4

01:23

09 - GCS Cluster

001 Spark on GCS Cluster.mp4

01:48

10 - Spark Internals

001 Drivers And Executors.mp4

02:12

002 RDDs And Dataframes.mp4

04:28

003 Transformation And Actions.mp4

06:12

004 Wide And Narrow Transformations.mp4

05:22

005 Understanding Execution Plan.mp4

04:57

006 Different Plans by Driver.mp4

02:30

11 - Spark RDD Transformation And Actions

001 MapFlatMap Transformation.mp4

04:28

002 FilterIntersection.mp4

04:00

003 UnionDistinct Transformation.mp4

02:23

004 GroupByKey Group people based on Birthday months.mp4

05:54

005 ReduceByKey Total Number of students in each Subject.mp4

06:44

006 SortByKey Sort students based on their rollno.mp4

06:03

007 MapPartition MapPartitionWithIndex.mp4

06:20

008 Change number of Partitions.mp4

03:34

009 Join join email address based on customer name.mp4

03:06

010 Spark Actions.mp4

06:05

12 - Spark RDD Practice

001 Scala Tuples.mp4

03:05

001 spark-dataset-20200505T025156Z-001.zip

002 Filter Error Logs.mp4

10:22

003 Frequency of word in Text File.mp4

08:35

004 Population of each city.mp4

03:53

005 Orders placed by Customers.mp4

09:20

006 average rating of movie.mp4

07:04

13 - Spark Dataframes And Spark SQL

001 Dataframe Intro.mp4

02:16

001 Spark-Dataframe.pdf

001 dataframe-dataset-20200505T025651Z-001.zip

002 Dafaframe from Json Files.mp4

08:42

003 Dataframe from Parquet Files.mp4

07:26

004 Dataframe from CSV Files.mp4

05:14

005 Dataframe from Avro File.mp4

07:14

006 Working with XML.mp4

03:22

007 Working with Columns.mp4

05:23

008 Working with String.mp4

04:05

009 Working with Dates.mp4

03:47

010 Dataframe Filter API.mp4

02:50

011 DataFrame API Part1.mp4

04:51

012 DataFrame API Part2.mp4

06:25

013 Spark SQL.mp4

01:41

014 Working with Hive Tables in Spark.mp4

02:35

015 Datasets versus Dataframe.mp4

03:28

016 User Defined Functions (UDFS).mp4

03:38

14 - Using Intellij IDE

001 Intellij Setup.mp4

02:24

002 Project Setup.mp4

03:43

003 Writing first Spark program on IDE.mp4

07:55

004 Understanding spark configuration.mp4

07:00

005 Adding ActionsTransformations.mp4

07:55

006 Understanding Execution Plan.mp4

07:43

15 - Running Spark on EMR (AWS Cloud)

001 EMR Cluster Overview.mp4

02:02

002 Cluster Setup.mp4

07:56

003 Setting Spark Code for EMR.mp4

06:31

004 Using Spark-submit.mp4

05:42

005 Running Spark on EMR Cluster.mp4

04:54

16 - Spark with Cassandra

001 Cassandra Course.html

002 Creating Spark RDD from Cassandra Table.mp4

09:13

003 Processing Cassandra data in Spark.mp4

08:18

004 Cassandra Rows to Case Class.mp4

02:33

005 Saving Spark RDD to Cassandra.mp4

02:58

17 - Getting Started with MongoDB

001 MongoDB Intro.mp4

04:18

001 mongo-commads3.docx

001 mongo-commands1.zip

001 movies.zip

002 MongoDB Usecase And Limitations.mp4

04:18

003 MongoDB Installation.mp4

08:03

18 - CRUD Operations

001 Find.mp4

03:37

002 Find With Filter.mp4

02:09

003 Insert.mp4

04:20

004 Update.mp4

05:55

005 Update Continues.mp4

05:30

006 Projections.mp4

02:29

007 Delete.mp4

04:14

19 - Working with Operators

001 In not in Operators.mp4

02:39

002 gte lte Operators.mp4

02:16

003 and or operators.mp4

03:03

004 regex operator.mp4

02:47

20 - MongoDB Compass

001 Working with GUI.mp4

04:51

21 - Advanced Mongo

001 ValidationSchema.mp4

03:41

002 Working with Indexes.mp4

05:18

22 - Spark with Mongo

001 Spark Mongo Integration.html

Description

In-depth course on Big Data - Apache Spark , Hadoop , Sqoop , Flume & Apache Hive, MongoDB & Big Data Cluster setup

What You'll Learn?

Hadoop distributed File system and commands. Lifecycle of sqoop command. Sqoop import command to migrate data from Mysql to HDFS. Sqoop import command to migrate data from Mysql to Hive. Working with various file formats, compressions, file delimeter,where clause and queries while importing the data. Understand split-by and boundary queries. Use incremental mode to migrate the data from Mysql to HDFS. Using sqoop export, migrate data from HDFS to Mysql. Using sqoop export, migrate data from Hive to Mysql. Understand Flume Architecture. Using flume, Ingest data from Twitter and save to HDFS. Using flume, Ingest data from netcat and save to HDFS. Using flume, Ingest data from exec and show on console. Flume Interceptors.

Who is this for?

Who want to learn big data in detail

What You Need to Know?

More details

Description
In this course, you will start by learning what is hadoop distributed file system and most common hadoop commands required to work with Hadoop File system.

Then you will be introduced to Sqoop Import
Understand lifecycle of sqoop command.
Use sqoop import command to migrate data from Mysql to HDFS.
Use sqoop import command to migrate data from Mysql to Hive.
Use various file formats, compressions, file delimeter,where clause and queries while importing the data.
Understand split-by and boundary queries.
Use incremental mode to migrate the data from Mysql to HDFS.

Further, you will learn Sqoop Export to migrate data.
What is sqoop export
Using sqoop export, migrate data from HDFS to Mysql.
Using sqoop export, migrate data from Hive to Mysql.

Further, you will learn about Apache Flume
Understand Flume Architecture.
Using flume, Ingest data from Twitter and save to HDFS.
Using flume, Ingest data from netcat and save to HDFS.
Using flume, Ingest data from exec and show on console.
Describe flume interceptors and see examples of using interceptors.
Flume multiple agents
Flume Consolidation.

In the next section, we will learn about Apache Hive
Hive Intro
External &Â Managed Tables
Working with Different Files - Parquet,Avro
Compressions
Hive Analysis
Hive String Functions
Hive Date Functions
Partitioning
Bucketing

You will learn about Apache Spark
Spark Intro
Cluster Overview
RDD
DAG/Stages/Tasks
Actions &Â Transformations
Transformation &Â Action Examples
Spark Data frames
Spark Data frames - working with diff File Formats & Compression
Dataframes API's
Spark SQL
Dataframe Examples
Spark with Cassandra Integration
Running Spark on Intellij IDE
Running Spark on EMR

Who this course is for:
Who want to learn big data in detail

User Reviews

Rating

average 0

Total votes0

Focused display

Navdeep is one of the renowned Premium Instructor at Udemy. Navdeep has 12 years of industry experience in different technologies and domains. With 9+ courses and 40,000+ students and rating of 4.5*, she is one of the leading instructors in the field of Big Data & Cloud.Happy Learning!

Udemy

View courses Udemy

Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.