Apache Spark - PySpark

Focused View

Blismos Academy

19:58:51

89 View

1. THE FUNDAMENTALS

1. Data VS Information.mp4

04:20

2. Data Storage and Processing.mp4

08:50

3. Data Sources.mp4

07:59

4. Big Data Introduction.mp4

12:12

2. THE FOUNDATIONS OF BIG DATA

1. Emergence of Big Data.mp4

05:52

2. Basic Terminologies.mp4

04:50

3. Central theme of Big Data.mp4

08:06

4. Requirements of Programming Model.mp4

13:20

5. Understand Distributed Processing through a Story.mp4

06:05

3. ENVIRONMENT AND INSTALLATION

1. Oracle VirtualMachine_Installation.mp4

03:18

2. How to install Ubuntu operating system on Virtual Box.mp4

07:18

3. How to install PySpark on Ubuntu with Java and Python 3.mp4

10:33

4. How to configure Pyspark with Pycharm with_Installation.mp4

07:06

5. Google Cloud Platform Setup.mp4

13:07

4. HADOOP ECOSYSTEM

1. Introduction to Hadoop Ecosystem.mp4

12:57

5. PYTHON FOR PYSPARK

1. INTRODUCTION TO PROGRAMMING.mp4

16:37

2. Introduction to Python.mp4

07:03

3. Environment for Python.mp4

04:24

4. Executing Python Code.mp4

06:15

5. Syntax, Indentation and Comments.mp4

06:43

6. Syntax, Indentation and Comments - Practical.mp4

05:14

7. Variables.mp4

12:12

8. Variable Practicals.mp4

11:46

9. Python Datatypes.mp4

15:32

10. Python Datatypes Practicals.mp4

10:15

11. Python Operator Concepts.mp4

13:47

12. Python Operator Practicals.mp4

09:01

13. Control Flows in Python.mp4

04:44

14. Control Flows - IF ELSE Concepts.mp4

06:41

15. If Else Practical.mp4

04:57

16. Loops Theory.mp4

10:53

17. Loops Practical.mp4

08:02

18. Python Function Concepts.mp4

11:37

19. Python Function Hands-on.mp4

09:31

6. APACHE SPARK

1. Why Spark.mp4

06:55

2. Advantages of Spark.mp4

07:12

3. What is Spark.mp4

06:21

4. Components of Spark.mp4

02:00

5. History of Spark.mp4

05:37

7. OVERVIEW OF SPARK

1. Architecture of Spark.mp4

09:15

2. Spark Session.mp4

06:35

3. Spark Session Terminal And Jupyter notebook Hands-On.mp4

04:22

4. Spark Language API.mp4

04:45

5. Dataframes and Partitions.mp4

08:02

6. Spark Transformations.mp4

09:47

7. Spark Actions.mp4

08:22

8. STRUCTURED API OVERVIEW

1. Structured APIs - Dataframes and Datasets.mp4

07:29

2. Schema Definition.mp4

04:41

3. Spark Types.mp4

05:06

4. Structured API Execution.mp4

06:06

9. OPERATIONS ON DATAFRAMES

1. Dataframe Columns.mp4

11:17

2. Columns as Expression.mp4

05:42

3. Dataframe Rows.mp4

06:09

4. Ways of Creating Dataframe.mp4

16:48

5. Methods to Manipulate Columns.mp4

20:17

6. DataFrame Transformations.mp4

02:29

7. Dataframe Transformation - Columns.mp4

15:08

8. Dataframe Transformations - Rows Part1.mp4

14:24

9. Dataframe Transformation - Rows Part2.mp4

19:54

10. WORKING WITH DIFFERENT TYPES OF DATABASE

1. Introduction to working with Different Types of Data.mp4

02:58

2. Working with Booleans.mp4

15:54

3. Working with Strings.mp4

10:18

4. Working with Strings Practical1.mp4

08:05

5. Working with Strings Practical2.mp4

07:52

6. Working with Date and Time Stamps.mp4

17:11

7. Working with Null Concepts.mp4

07:36

8. Working with Nulls Practicals.mp4

15:40

9. Working with Complex Types.mp4

09:51

10. Working with Complex types practical.mp4

11:45

11. User Defined Functions - Concepts.mp4

12:48

12. Working with Complex types practical.mp4

08:40

11. CREATING DATAFRAMES FROM DIFFERENT SOURCES

1. Data Sources Introduction.mp4

04:27

2. Read-API- Data Sources.mp4

04:07

3. Read-API-Practical.mp4

12:19

4. Write-API-Data Sources.mp4

03:38

5. Write-API-Practical.mp4

13:02

6. Reading from CSV Files.mp4

10:51

7. Writing into CSV Files.mp4

05:09

8. Reading from JSON Files and Writing into JSON.mp4

09:51

9. Reading from Parquet and writing into Parquet.mp4

11:52

10. Reading from ORC and writing into ORC.mp4

09:16

11. Unstructured Data - Text File - Reading and Writing.mp4

12:26

12. Introduction to reading data from structured sources.mp4

06:28

13. Reading data from structured sources - Database - Concepts.mp4

09:13

14. Reading data from structured sources - Database - Practicals.mp4

15:20

15. Query Pushdown Concepts.mp4

08:40

16. Query Pushdown Praticals.mp4

07:13

17. Writing into structured sources - Database - Concepts.mp4

05:40

18. Writing into structured sources - Database - Practicals.mp4

11:42

12. AGGREGATIONS

1. Introduction to Aggregations.mp4

09:18

2. Aggregataion Concepts - Count.mp4

07:45

3. Aggregation Practical-1-Count.mp4

15:43

4. Aggregation Concepts - First, Sum and Average.mp4

04:58

5. Aggregation - Practical 2 - First Last Average.mp4

12:07

6. Aggregation-Practical-3-StatisticalFunctions.mp4

11:42

7. Aggregation Concepts - Grouping.mp4

05:18

8. Aggregation-Practical-4-GroupBy.mp4

09:50

9. Aggregation Concepts - Window Functions.mp4

08:16

10. Aggregation-Practical-5-WindowFunctions.mp4

15:53

11. Aggregation Concepts - RollUp and Cube.mp4

06:16

12. Aggregation-Practical-6-RollupandCube.mp4

12:09

13. SPARK JOINS

1. Spark Joins Theory-1-Introduction.mp4

05:53

2. Spark Joins Theory-2-How Joins Work.mp4

05:35

3. Spark Joins-Theory-3-Inner Joins.mp4

02:47

4. Spark Joins -Practical -1-Innerjoins.mp4

07:54

5. Saprk Joins - Theory-4 - Outer Joins.mp4

05:43

6. Spark Joins -Practical - Outer Joins.mp4

06:47

7. Spark Joins -Theory - 5-Left Semi And Anti Joins.mp4

06:34

8. Spark Joins - Practical - Left Semi And Anti Joins.mp4

04:24

9. Spark Joins -Theory -6-CrossJoin.mp4

04:26

10. Spark Joins - Practical- Cross Joins.mp4

03:48

11. Spark Joins -Theory -7-Challenges In Joins.mp4

07:01

12. Spark Joins-5-Practical-Tackling the Challenges in Joins.mp4

16:25

13. Spark Joins -Theory -8-Communication Strategies.mp4

17:40

14. RESILIENT DISTRIBUTED DATASETS- RDDs

1. What is an RDD .mp4

06:56

2. Introduction to Low Level APIs.mp4

07:51

3. Properties Of RDD.mp4

03:09

4. When to use RDDs.mp4

03:59

5. Creating RDDs.mp4

12:03

6. RDD Practical-1-Creating RDDs.mp4

10:52

7. RDD Lineage.mp4

06:08

8. RDD Transformations.mp4

13:28

9. RDD - Transformations Practical.mp4

10:14

10. RDD Actions.mp4

12:38

11. RDD Actions - Practical.mp4

11:07

12. RDDT Saving To File.mp4

03:46

13. RDD Saving to a File - Practical.mp4

03:52

15. DISTRIBUTED VARIABLES

1. Distributed Variables - Introduction.mp4

03:47

2. Broadcast Variables.mp4

13:34

3. Broadcast Variables - Practical.mp4

06:13

4. Accumulators.mp4

10:59

5. Accumulators - Practical.mp4

06:37

16. HOW SPARK WORKS ON A CLUSTER

1. Introduction.mp4

03:18

2. How Spark runs on a Cluster - Cluster Manager.mp4

03:25

3. How Spark runs on a Cluster - Execution Modes.mp4

04:54

4. Life Cycle a Spark Application - Outside Spark.mp4

07:45

5. Life Cycle of a Spark Application - Inside Spark.mp4

12:22

Description

PySpark

What You'll Learn?

Learners will understand the Apache Spark Foundation and Spark Architecture
How Apache Spark can be used in Data Engineering and Data Processing
Working with different Data Sources and types of Datasets
Working with Data Frames and PySpark
Use Python and Spark together to analyze Big Data
Learner will understand about PySpark RDD
PySpark DataFrames Actions and Transformation
Use of different file formats such as Parquet, JSON, CSV etc in building Data Engineering Pipelines

Who is this for?

Computer Science or IT Students or other graduates with passion to get into IT

Data Warehouse Developers or Testers who want to transition to Data Engineering roles

Someone who is very familiar with another programming language and needs to learn Spark

Data Engineers,Data Scientists,Data Analysts, Database Developers

What You Need to Know?

Basic Knowledge of Python and SQL are necessary

Having a reliable internet connection and a strong desire to learn are essential prerequisites.

More details

Description
Learn the latest Big Data technology, Apache Spark, and its collaboration with Python, one of the most popular programming languages. This comprehensive course covers everything from the basics to advanced levels of data analysis.
Apache Spark is a highly sought-after technology in the Big Data analytics industry, with top companies like Google, Facebook, Netflix, Airbnb, Amazon, and NASA utilizing it to solve their data challenges. Its superior performance, up to 100 times faster than Hadoop MapReduce, has led to a surge in demand for professionals skilled in Spark.
By mastering Spark and its DataFrame framework, which is relatively new and in high demand, you'll position yourself as a highly knowledgeable candidate in the job market.

Throughout the course, you'll work with PySpark for data analysis, exploring Spark RDDs, DataFrames, and the various transformations and actions you can perform on data using them.
In addition, the course covers essential topics such as Spark architecture, the Data Sources API, and the DataFrame API. You'll learn how to efficiently ingest CSV files, as well as simple and complex JSON files, into the data lake as parquet files or tables.
The course also delves into important PySpark transformations, including filtering, joining, simple aggregations, groupBy operations. These transformations enable you to manipulate and analyze data effectively within PySpark.
Furthermore, you'll gain expertise in creating local and temporary views, allowing you to organize and work with data more efficiently in PySpark.
With a comprehensive coverage of topics ranging from Spark architecture to transformations, and view creation, this course equips you with the necessary skills to become a proficient PySpark Developer.
With over 150 concise tutorial videos, this course provides a comprehensive understanding of the concepts and methodologies of PySpark. Whether you're aiming to become a PySpark Developer or enhance your Big Data skills, this course is a must-have.
Who this course is for:
Computer Science or IT Students or other graduates with passion to get into IT
Data Warehouse Developers or Testers who want to transition to Data Engineering roles
Someone who is very familiar with another programming language and needs to learn Spark
Data Engineers,Data Scientists,Data Analysts, Database Developers

User Reviews

Rating

average 0

Total votes0

Focused display

Practitioners of Big Data and related technologiesTeam has over two decades of experience in the industryPassionate in dealing with data and providing IT solutionsWe believe in continuous learningEnjoy spreading the knowledge through Training, Workshops, Internships and Projects assignmentsOur Solution provides support and expertise advice that is presented for consideration and decision-making in Big Data Technologies

Udemy

View courses Udemy

Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.