Big-Data with PySpark, Spark Streaming, Spark ML and Kafka

Focused View

Implied Concepts AI

3:27:45

12 View

1. Introduction

1. PySpark Course Introduction and Outline.mp4

02:51

2. Important Notice.html

3. Why Spark.mp4

04:07

4. Spark Logical Architecture.mp4

03:48

2. PySpark Installation

1. Spark Installation Intro and Outline.mp4

01:18

2. Anaconda Installation.mp4

01:36

3. Java Run Time Installation.mp4

01:43

4. Spark-Hadoop Libraries.mp4

01:51

5. Environment Variables.mp4

05:37

6. Final Setup and Permissions.mp4

01:33

3. Cloud based Libraries Setup - Azure, AWS, GCP

1. Cloud File Systems Installation - Azure,AWS,GCP - 1.mp4

03:16

2. Cloud File Systems Installation - Azure,AWS,GCP - 2.mp4

14:05

4. PySpark Hands On - First Job

1.1 FirstSparkJob.zip

1. First Spark Job.mp4

08:56

2. Spark Job with Could Systems.mp4

09:25

5. Spark Core

1. Spark Context Vs Spark Sessions.mp4

02:19

2. Spark Data-Frames.mp4

02:17

6. PySpark - Transformation and Actions

1.1 2008 1.csv

1.2 RDD-Transformations and Actions.zip

1.3 trip data 1.csv

1.4 trip fare 1.csv

1. RDD Transformations and Actions - 1.mp4

07:36

2.1 RDD - Transformations & Actions - 2.zip

2. RDD Transformations and Actions - 2.mp4

11:43

3.1 DataFrames .zip

3. DataFrames - Transformations and Actions -1.mp4

07:39

4. DataFrames - Transformations and Actions - 2.mp4

11:08

7. PySpark Clusters on Cloud - Azure, AWS, GCP

1. Azure HDInsights Intro.mp4

03:15

2. Azure HDInsights - 2.mp4

02:06

3. Cloud Cluster Demo - 1.mp4

04:26

4. Cloud Cluster Demo - 2 - RDD Operations.mp4

03:55

5. Cloud Cluster Demo - 3 - DataFrames Operations.mp4

05:05

6. GCP DataProc Clusters.mp4

03:04

7. AWS EMR Cluster.mp4

03:54

8. BigData Examples

1.1 BIgData Data ingestion & cleanup.zip

1.2 Full Dataset - 2013 NYC taxi trip and fare dataset.html

1. BigData Ingestion and Preprocessing.mp4

05:00

2.1 Full Dataset - 2013 NYC taxi trip and fare dataset.html

2.2 SparkSQL.zip

2. PySpark - SQL.mp4

09:04

9. Spark Streaming with Kafka IoT Pipeline

1. Spark Streaming Introduction.mp4

03:02

2. Popular Big Data Streaming Processing Frameworks.mp4

02:53

3. Spark Streaming Vs Spark Structured Streaming.mp4

03:32

4. Spark Streaming Hands-on Demo Introduction.mp4

01:15

5.1 ex-steps-wordscoutspy.txt

5.2 wordcount.zip

5. Spark Streaming Hands-on - Socket Streaming.mp4

05:35

6.1 ex-steps-wordscoutsFilepy.txt

6.2 wordcount files.zip

6. Spark Streaming Hands-on - File Streaming.mp4

02:09

7.1 packages.zip

7. Kafka, Zookeeper, Pyspark Startup Scripts.mp4

06:14

8.1 ex-example1.txt

8.2 kafka-pyspark-1.zip

8. Kafka - PySpark Ex-1.mp4

06:16

9.1 ex-example2.txt

9.2 kafka-pyspark-2.zip

9. Kafka - PySpark Ex-2.mp4

05:19

10.1 ex-example3.txt

10.2 kafka-pyspark-3.zip

10. Kafka - PySpark Ex-3.mp4

02:14

11.1 ex-example4.txt

11.2 kafka-pyspark-4.zip

11. Kafka - PySpark Ex-4.mp4

07:22

12.1 kafka-pyspark-5.zip

12. Kafka - PySpark Ex-5.mp4

03:29

13. Spark Streaming - Tumbling and Sliding Window.mp4

04:50

14.1 kafka-pyspark-6.zip

14.2 kafka-pyspark-7.zip

14. Kafka - PySpark Ex- 6 & 7.mp4

05:06

15.1 kafka-pyspark-8.zip

15. Spark Stream - WaterMarking & Ex - 8.mp4

05:52

Description

BigData with PySpark, Spark Streaming, Spark ML, Kafka, IoT, Architecture, 40+ RDDs & Dataframes Functions, ML

What You'll Learn?

Complete Pyspark for Big Data Engineers and Architects
Spark ML - Learn how to design Spark ML models with pipelines
Spark Streaming- Learn all about spark streaming
Learn how to do Big Data ETL jobs with 40+ data transformation and actions
Learn to be Big Data Architect with Kafka, Spark Architecture for real time and batch processing

Who is this for?

Big-Data, Spark, Spark Stream, Kafka, Python, Clusters,Spark Machine Learning at scale

What You Need to Know?

Basics of Python

More details

Description
In the 'Mastering PySpark with Cloud Computing' course, you will:
Learn to navigate the rapidly evolving digital industry: As more businesses migrate to cloud services, understanding and deploying Big Data Frameworks on the cloud has become an essential skill set.
Gain proficiency in PySpark and Cloud Computing: You will master PySpark, one of the most potent Big Data Frameworks, and learn cluster-based cloud computing on platforms such as Azure HDInsight, AWS EMR, and GCP DataProc.
Expand your knowledge beyond basic PySpark: The course covers advanced topics including Spark Streaming, Spark Machine Learning, Kafka, IoT, and more.
Understand Spark Architecture and RDD: You will learn about Spark Architecture, Dataframe, and Resilient Distributed Datasets (RDD) by exploring 40+ Spark Transformation and Action methods.
Benefit from high-quality visual aids: Complex concepts will be simplified through visual aids. This includes a detailed analysis of Tumbling Window, Sliding Window, and Watermarking in Spark Structured Streaming.
Get hands-on with Spark Streaming: The course provides eight distinct examples to solidify your understanding of Spark Streaming.
Explore Big Data Ingestion and Pre-processing: You'll delve into the intricacies of Big Data Ingestion and Pre-processing, SQL in PySpark.
Learn Spark Machine Learning with real-world examples: Gain practical knowledge with real-world examples of Regression, Clustering, Decision Trees, and Topic Modeling using Pipelines.
Join our 'Mastering PySpark with Cloud Computing' course today to close any gaps in your knowledge of data architecture and cloud computing, and move forward in becoming a Big Data expert.

Who this course is for:
Big-Data, Spark, Spark Stream, Kafka, Python, Clusters,Spark Machine Learning at scale

User Reviews

Rating

average 0

Total votes0

Focused display

Implied Concepts is Academy and Consultancy in Deep Learning, Machine Learning, Big-Data, Communication Systems, Cloud Computing, Embedded Systems, Computer Vision, Predictive Maintenance, Advance Process Control. We offer the most up to date and bleeding edge learning Course in Deep Learning, Machine Learning with Digestive Math series and Computation series that not only helps you understand the implied ideas, but its underline math to help you build new products.We guarantee after completion of the course, you will know the implied math, concepts and program solutions yourself.This is a unique initiative that bridges the gap for professionals, students, enthusiast who need to understand the implied concepts.In today's fast changing research, there is need to have exposure to cross domains to create out of box solution, to be relevant in industry.A statistical background pure play Data Scientist would not know how to write production code on IoT DeviceComputer Science student would not know statistics or augmented reality. Signal Processing is exclusive domain that is difficult for Data Scientist or Computer Scientist to understand.Our trainers are have more than 10 years of industry experience in product development from worlds top universities and are exposed to cross domain development.Welcome to the world of implied ideas!

Udemy

View courses Udemy

Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.