Learning Apache Spark | Master Spark for Big Data Processing

Focused View

7:11:08

0 View

1 - Getting Started

1 -Why Should You Learn Apache Spark.mp4

06:13

2 -What Does This Course Offer on Apache Spark.mp4

05:37

2 - All about Apache Spark

1 -Lets understand WordCount.mp4

04:23

2 -Lets understand Map and Reduce.mp4

03:07

3 -Programming with Map and Reduce.mp4

03:24

3 -section02 03 map reduce.zip

4 -Lets understand Hadoop.mp4

03:54

5 -Apache Hadoop Architecture.mp4

04:44

6 -Apache Hadoop and Apache Spark.mp4

05:01

7 -Apache Spark Architecture.mp4

04:06

8 -What is PySpark.mp4

03:21

3 - Installations for Apache Spark

1 -Install JAVA JDK.mp4

05:19

1 -Section03 - 00 - Resources.zip

2 -Install Python.mp4

02:50

3 -Install JupyterLab.mp4

04:05

4 -Install PySpark.mp4

01:57

5 -Spark Session by Initialization.mp4

03:06

6 -Running PySpark on AWS EC2 Instances P1.mp4

07:27

7 -Running PySpark on AWS EC2 Instance P2.mp4

08:02

4 - Using Databricks Community Edition

1 -Why Use Databricks Community Edition.mp4

03:42

2 -Register for Databricks Community Edition.mp4

05:25

3 -When to use Databricks Community Edition.mp4

04:36

4 -Running Magic Commands in Databricks P1.mp4

06:43

5 -Running Magic Commands in Databricks P2.mp4

01:51

5 - Spark DataFrames

1 -Apache Spark DataFrame.mp4

03:09

2 -Create DataFrames from CSV Files P1.mp4

06:42

2 -data.zip

2 -section05 03 create dataframes using csv files.zip

3 -Create DataFrames from CSV Files P2.mp4

06:00

4 -Create DataFrames from Parquet Files.mp4

04:53

4 -section05 05 create dataframes using parquet files.zip

6 - Spark Data Transformations

1 -Using SELECT.mp4

06:29

1 -data.zip

1 -section06 01 using select.zip

2 -Using FILTER.mp4

03:37

2 -section06 02 using filter.zip

3 -Using ORDER BY.mp4

03:12

3 -section06 03 using order by.zip

4 -Using GROUP BY.mp4

04:11

4 -section06 04 using group by.zip

5 -Using AGGREGATE Functions.mp4

05:18

5 -section06 05 using aggregate functions.zip

6 -Using INNER JOIN.mp4

05:25

6 -section06 06 using inner joins.zip

7 - Spark SQL Catalog

1 -Spark SQL Catalogs.mp4

01:48

2 -Access Spark SQL Catalogs.mp4

02:48

2 -data.zip

2 -section07 02 access spark sql catalogs.zip

3 -List Databases from Catalogs.mp4

01:59

3 -section07 03 list databases from catalogs.zip

4 -List Tables from Current Database.mp4

01:57

4 -section07 04 list tables from current databases.zip

5 -Create Spark Temp View.mp4

05:01

5 -section07 05 create spark temp view.zip

6 -Run SQL Queries on Temp Views.mp4

04:16

6 -section07 06 run sql queries.zip

7 -Drop Temp Views.mp4

02:25

7 -section07 07 drop temp views.zip

8 - Databricks Utility FileSystem for Apache Spark

1 -Using Databricks Utilities.mp4

03:22

1 -data.zip

1 -section08 01 databricks file system for apache spark.zip

2 -Using dbfs - Databricks Utility FileSystem.mp4

07:06

3 -Using dbfs - Make Directory.mp4

02:33

4 -Using dbfs - Copy Files.mp4

02:27

5 -Using dbfs - Delete Files.mp4

01:28

9 - Pandas API on Spark

1 -Introduction to Pandas.mp4

03:15

2 -Pandas API on Spark.mp4

02:57

3 -Reading and Writing Data with Pandas P1.mp4

07:36

3 -data.zip

3 -section09 03 reading and writing data with pandas.zip

4 -Reading and Writing Data with Pandas P2.mp4

05:34

5 -Data Manipulation with PySpark Pandas.mp4

07:13

5 -section09 05 data manipulation with pyspark pandas.zip

6 -Merging and Joining in PySpark Pandas.mp4

04:57

6 -section09 06 merging and joining dataframes with pyspark pandas.zip

7 -Grouping and Aggregation with PySpark Pandas.mp4

04:34

7 -section09 07 grouping and aggregation with pyspark pandas.zip

8 -Visualizing Data in PySpark Pandas.mp4

03:46

8 -section09 08 visualizing with pyspark pandas.zip

10 - Structured Streaming Using Apache Spark

1 -What is Apache Spark Structure Streaming.mp4

04:10

2 -How Apache Spark handles Structured Streaming.mp4

03:32

3 -Handling Programmatically Streaming Data.mp4

03:26

4 -Programmatic Modes by Apache Spark.mp4

07:09

5 -DataFrames for Streaming.mp4

05:41

6 -Section10 - 00 - Resources.zip

6 -readStream API.mp4

08:48

7 -writeStream API.mp4

08:20

8 -Querying Data.mp4

02:18

9 -StreamingQuery - stop.mp4

07:35

10 -Structured Streaming with Kafka and Spark P1.mp4

04:02

11 -Structured Streaming with Kafka and Spark P2.mp4

06:55

12 -Structured Streaming with Kafka and Spark P3.mp4

10:00

13 -Terminate the Kafka Environment.mp4

02:19

14 -Handling Late Data Arrivals and Water Marking P1.mp4

08:29

15 -Handling Late Data Arrivals and Water Marking P2.mp4

08:47

11 - Machine Learning with Spark

1 -About this section.mp4

02:13

2 -Learning about Machine Learning.mp4

06:44

3 -How to build a Machine Learning Model.mp4

03:49

4 -Apache Spark MLLib Overview.mp4

04:05

5 -Learning about ML Pipelines using Spark MLlib.mp4

03:52

6 -Data Sources by Spark MLlib to Build ML Models.mp4

03:18

7 -Create DataFrames from Data Sources.mp4

03:06

7 -Section10 - 00 - Resources.zip

8 -Learning about Featurization using Spark MLlib.mp4

03:12

9 -Using Apache Spark MLlibs - Feature Transformers.mp4

03:11

10 -Using Tokenizer.mp4

06:01

11 -Using StringIndexer.mp4

04:00

12 -Using Pipelines.mp4

07:45

13 -Using VectorAssembler.mp4

04:22

14 -Using VectorIndexer.mp4

05:34

15 -Using MLlib Estimator - Linear Regression.mp4

08:30

16 -Using MLlib Estimator - Logisitic Regression.mp4

06:29

17 -Measure ML Effiecny using Spark MLlib Evaluators.mp4

08:43

18 -Using ML for Solving Real World Problem.mp4

07:18

19 -Building ML Model P1 - Using Local Host.mp4

07:47

20 -Building ML Model P2 - Using Databricks Community Edition.mp4

11:30

21 -Using Apache Spark MLFlow with Databricks Community Edition.mp4

03:17

Description

Embark on a comprehensive journey to Master Apache Spark from Data Manipulation to Machine Learning!

What You'll Learn?

Understand the fundamentals of Spark’s architecture and its distributed computing capabilities
Learn to write and optimize Spark SQL queries for efficient data processing
Master the creation and manipulation of DataFrames, a core component of Spark
Learn to read data from different file formats such as CSV and Parquet
Develop skills in filtering, sorting, and aggregating data to extract meaningful insights
Learn to process and analyze streaming data for real-time insights
Explore the capabilities of Spark’s MLlib for machine learning
Learn to create and fine-tune models using pipelines and transformers for predictive analytics

Who is this for?

IT professionals interested in big data and analytics

Aspiring Data Scientists

Aspiring Data Analysts

Aspiring Machine Learning Engineers

Business Analysts

Software Engineers

Students and Academics

Researchers

Anyone Interested in Big Data

What You Need to Know?

You should know how to write and run Python code

Basic understanding of Python syntax and concepts is necessary

Understanding SQL (Structured Query Language) is important

You should know how to create and manage tables, transform data, and run queries

More details

Description
Unlock the power of big data with Apache Spark!
In this course, youâ€™ll learn how to use Apache Spark with Python to work with data.
Weâ€™ll start with the basics and move up to advanced projects and machine learning.
Whether youâ€™re just starting or already know some Python, this course will teach you step-by-step how to process and analyze big data.
What Youâ€™ll Learn:
Use PySparkâ€™s DataFrame: Learn to organize and work with data.
Store Data Efficiently: Use formats like Parquet to store data quickly.
Use SQL in PySpark: Work with data using SQL, just like with DataFrames.
Connect PySpark with Python Tools: Dig deeper into data with Pythonâ€™s data tools.
Machine Learning with PySparkâ€™s MLlib: Work on big projects using machine learning.
Real-World Examples: Learn by doing with practical examples.
Handle Large Data Sets: Understand how to manage big data easily.
Solve Real-World Problems: Apply Spark to real-life data challenges.
Build Confidence in PySpark: Get better at big data processing.
Manage and Analyze Data: Gain skills for both work and personal projects.
Prepare for Data Jobs: Build skills for jobs in tech, finance, and healthcare.
By the end of this course, youâ€™ll have a solid foundation in Spark, ready to tackle real-world data challenges.
Who this course is for:
IT professionals interested in big data and analytics
Aspiring Data Scientists
Aspiring Data Analysts
Aspiring Machine Learning Engineers
Business Analysts
Software Engineers
Students and Academics
Researchers
Anyone Interested in Big Data

User Reviews

Rating

average 0

Total votes0

Focused display

Apache Spark

Udemy

View courses Udemy

Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.