PYSPARK End to End Developer Course (Spark with Python)

Focused View

Kedar Nanda

2:40:26

210 View

1. Introduction

1.1 PySpark Slides v1.pdf

1.2 retail db.pptx

1.3 RetailDB SalesData.zip

1. Download Course Slides and Data Files.html

2. Spark Installation and Set Up Standalone (Windows)

1.1 01 Resources PySpark Set Up on Windows.pdf

1. Resources.html

2. Minimum Supported VersionsPrerequisites.mp4

02:04

3. Java Installation.mp4

10:56

4. Python Installation.mp4

04:23

5. Spark Installation.mp4

10:14

6. Winutils Set up.mp4

06:12

7. PyCharm Instalaltion.mp4

05:14

8. PyCharm Basics.mp4

09:09

9. PyCharm run time arguments.mp4

04:35

10. Integrate Python and PySpark.mp4

06:13

11.1 11 debug.zip

11. How to debug Python Applications using PyCharm.mp4

13:00

3. HDFS Course

1.1 02 01 hdfs slides.zip

1. Download Slides.html

2. What is HDFS and Why HDFS.mp4

03:43

3. HDFS Components and Metadata.mp4

03:59

4. HDFS Block and Replication.mp4

05:12

5. Rack Awareness.mp4

01:53

6. HDFS Read Mechanism Architecture.mp4

02:40

7.1 07 HDFS+Help+Commands.txt

7. Exercise HDFS CLI Help Commands.mp4

02:18

8.1 08 Get+Data+From+GitHub+to+Local+to+HDFS.txt

8. Exercise - Bring Data from GitHub to Local to HDFS.mp4

02:37

9.1 09 List+and+Sort+Files+and+Directories+in+HDFS.txt

9. Exercise - Listing and Sorting Files and Directories.mp4

02:59

10.1 10 Create+or+Remove+Directories+in+HDFS.txt

10. Exercise - Create or Remove Directories in HDFS.mp4

09:08

11.1 11 Copy+Data+from+HDFS+to+Local+System.txt

11. Exercise - Copy Data from HDFS to Local.mp4

05:12

12.1 12 Copy+Data+from+Local+to+HDFS.txt

12. Exercise - Copy data from Local to HDFS.mp4

06:58

13.1 13 Showing+Data+in+HDFS.txt

13. Exercise - Preview Data in HDFS.mp4

03:56

14.1 14 Knowing+Statistics+in+HDFS.txt

14. Exercise - Knowing Statistics in HDFS.mp4

03:47

15.1 15 Knowing+Storage+in+HDFS+File+System.txt

15. Exercise - Knowing Storage in HDFS File System.mp4

03:11

16.1 16 HDFS+MetaData.txt

16. Exercise - Metadata in HDFS.mp4

04:59

17.1 17 File+Permission+in+HDFS.txt

17. File Permission in HDFS.mp4

05:28

18.1 18 Override+Properties+in+HDFS.txt

18. Exercise - Update Properties in HDFS.mp4

08:18

4. Introduction to Spark

1. Why Spark was developed.mp4

06:44

2. What is Spark and its features.mp4

04:01

3. Spark Main Components.mp4

01:23

Description

Learn PySpark end to end features and functionalities. Course also includes a Python course and HDFS Commands Course.

What You'll Learn?

Complete Development Functionalities and Features of PySpark
Spark Cluster Execution Architecture
Spark SQL Architecture
Spark Performance and Optimization
Python Course
HDFS Course

Who is this for?

Data Engineers

Data Scientists

Data Analysts

Database Developers

More details

Description
Introduction to Spark.
HDFS Commands
Python Course.
Why Spark was developed.
What is Spark and its features.
Spark Main Components.
Introduction to Spark.
HDFS Commands
Introduction to SparkSession
RDD Fundamentals
What is RDD
RDD Properties
When to use RDD
RDD Problems
Create RDD
Different Ways to Create RDDs
RDD Operations
Transformations -Â Low Level
Transformations - Join Types
Actions -Â Total Aggregations
Shuffle and Combiner
Transformations -Â Key Aggregations
Transformations -Â Sorting
Transformations -Â Ranking
Transformations -Â Set
Transformations -Â Sampling
Transformations -Â Partition
Transformations -Â Repartition
Transformations -Â Repartition and Sort
Transformations -Â Coalesce
Transformations -Â Repartition Vs Coalesce
Extraction
Spark Cluster Execution Architecture_Full Architecture
Spark Cluster Execution Architecture_YARN As Spark Cluster Manager
Spark Cluster Execution Architecture_JVMs across Clusters
Spark Cluster Execution Architecture- Commonly Used Terms in Execution Framework
Spark Cluster Execution Architecture - Narrow and Wide Transformations
Spark Cluster Execution Architecture - DAG Scheduler
Spark Cluster Execution Architecture - Task Scheduler
RDD Persistence
Spark Shared Variables
SparkSQL Architecture
Detailed SparkSession Features
DataFrame Fundamentals
Datatypes
DataFrame Rows
DataFrame Columns
DataFrame ETL
DataFrame ETL_Introduction to Transformations and Extraction
DataFrame ETL_DataFrame APIs Introduction Extraction
DataFrame ETL_DataFrame APIs Selection
DataFrame ETL_DataFrame APIs Filter or Where
DataFrame ETL_DataFrame APIs Sorting
DataFrame ETL_DataFrame APIs Set
DataFrame ETL_DataFrame APIs Join
DataFrame ETL_DataFrame APIs Aggregations
DataFrame ETL_DataFrame APIs GroupBy
DataFrame ETL_DataFrame APIs Windows
DataFrame ETL_DataFrame Built-in Functions Introduction
Performance and Optimization

Who this course is for:
Data Engineers
Data Scientists
Data Analysts
Database Developers

User Reviews

Rating

average 0

Total votes0

Focused display

I have been teaching Spark and Other IT Courses like Snowflake, AWS, relational databases for over 10 Years now. Professionally, I work as a Senior Data Engineer with 14 plus years of experience in various data engineering functions like ETL Solutions, cloud solutions and data-warehousing solutions. I love teaching in a very easy and structured manner. Teaching is my hobby and passion.

Udemy

View courses Udemy

Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.