AWS Glue - The Complete Masterclass

Focused View

Data Soup

4:11:19

81 View

1. Introduction

1. Introduction.mp4

03:52

2. Course Overview.mp4

04:59

3. Glue Pipeline Resources (Section 2,3 and 5) Overview.mp4

03:01

2. Glue Resources Setup Part 1 - IAM, KMS, SNS

1. Section Overview.mp4

00:58

2. IAM 101 - Authentication, Authorization and Identities.mp4

02:38

3. IAM Lab - Setting Up Users and User Group.mp4

02:59

4. IAM Lab - Setting Up IAM Role.mp4

03:13

5. I AM 101 - Policies.mp4

04:45

6. KMS 101 And KMS Lab - Setting Up KMS Key.mp4

03:03

7. AWS SNS 101.mp4

02:38

8. Recap.mp4

01:13

9. Create GlueJobRole.html

3. Glue Resources Setup Part 2 - S3, AWS CLI , CloudFormation and CloudWatch

1. Section Overview.mp4

00:47

2.1 bucketpolicy_access_gluejobrole1.zip

2.2 bucketpolicy_example_noaccess.zip

2. AWS S3 101.mp4

03:43

3. AWS CLI 101.mp4

03:15

4. Configuring AWS CLI using IAM User Credentials.mp4

03:22

5.1 Sample CloudFormation Template.html

5. AWS Cloudformation 101.mp4

04:41

6. Create S3 Bucket - awsglueudemycourse-datasoup-gluejob2-source.html

7. Optional Assignment - Create S3 Bucket for GlueJob1 target -.html

4. Creating Bucket And Uploading Data For the Course

1. Section Overview.mp4

00:39

2.1 Glue-MasterClass.zip

2. Course Materials.mp4

01:09

3. Creating S3 Buckets.mp4

01:58

4. Uploading Data to S3 Buckets.mp4

02:38

5. Upload sample.csv file to bucket.html

5. Glue Resources SetUp Part 3 - Glue Catalog, Crawler

1. Section Overview.mp4

01:38

2.1 DataCatalog AWS Document.html

2. AWS Glue Catalog 101.mp4

03:28

3. AWS Glue Crawler 101.mp4

06:03

4.1 Crawler Classifier AWS Document.html

4. AWS Glue Crawler Classifier 101.mp4

03:16

5. Crawler Lab - First Glue Crawler Creation.mp4

04:17

6. First Glue Crawler Running.mp4

04:19

7. Crawler Lab - Second Glue Crawelr Creation.mp4

05:24

8. Crawler Lab - Third Glue Crawelr Creation.mp4

02:26

9. Crawler Lab - Forth Glue Crawler Creation.mp4

07:04

10. Crawler Lab - Fifth Glue Crawler Creation And Running.mp4

02:54

11.1 Glue Job Libraries.html

11. AWS Glue Job 101.mp4

07:21

12. AWS Glue Trigger 101.mp4

04:18

13. AWS Glue Workflow 101.mp4

02:52

14. Recap.mp4

02:13

6. CloudFormation Templates

1. Section Overview.mp4

01:20

2. CloudFormation Templates 101.mp4

00:59

3. First Glue Pipeline - CFN Templates.mp4

03:34

4. Second Glue Pipeline - CFN Templates.mp4

03:16

5. Glue Job 345 - CFN Template.mp4

02:05

6. Recap CFN Template Update.mp4

01:05

7. Upload CFN Templates to S3.mp4

02:13

7. Frist AWS Glue Pipeline Creation

1. Section Overview.mp4

00:57

2. Getting Ready For Glue Pipeline Creation.mp4

02:41

3. Deploying Glue Pipeline Stack Using CloudFormation.mp4

05:46

4. CloudFormation Template Deployment Debugging.mp4

06:41

5. Analyzing Glue Job Script And Running The Job.mp4

05:17

6. Going Through The Log And Verifying Job Output.mp4

04:06

8. AWS Glue Job Debugging

1. Section Overview.mp4

01:21

2. Section Prerequisite.mp4

01:59

3. Fix Error Retrieving The Script.mp4

05:40

4. Fix Launch Error And Glue Argument Error.mp4

05:40

5.1 bucketpolicy_access_gluejobrole1.zip

5. Fix Resource Policy Error - Error Reading From Source Bucket.mp4

03:33

6. Fix Identity Policy Error - Error Reading The Key.mp4

03:00

7. Workflow Running GlueJob2.mp4

01:34

8. Recap.mp4

04:26

9. Glue Streaming Job

1. Section Oveerview.mp4

02:22

2. Getting Ready For Glue Streaming Pipeline.mp4

02:05

3. Deploying Glue Streaming Job Infrastructure.mp4

04:04

4. Lab - Creating Python Shell Glue Job For Stream Generation.mp4

02:13

5. Lab - Creating Glue Streaming Loading Job.mp4

03:14

6. Lab- Creating Glue Streaming Transforming Job.mp4

03:30

7. Recap Before Running All Three Glue Streaming Jobs.mp4

02:25

8. Running Glue Streaming Generator Job.mp4

01:35

9. Running Glue Streaming Transformation Job.mp4

03:49

10. Section Recap.mp4

02:25

10. Glue Data Quality

1. Section Overview.mp4

01:43

2. Data Quality 101.mp4

02:43

3. Setting Up Data Quality Rule Set.mp4

04:02

4. Glue Job With Data Quality Check.mp4

03:38

5. Running the Glue Job.mp4

02:56

6. Setting Up Glue Data Quality CloudWatch Metrics.mp4

04:08

7. Receiving Alerts for Data Quality Issues.mp4

01:49

11. Glue Data Brew

1. Section Overview.mp4

01:01

2. Data Brew 101.mp4

03:33

3. Create DataSource and Profile Data.mp4

03:57

4. Create Project and Review Data Profile Output.mp4

04:27

5. Create and Publish Recipe.mp4

04:33

6. Create Job by Using Published Recipe.mp4

04:50

Description

Master building complete AWS Glue ETL Pipelines, Glue Data Quality, Glue Data Brew along with other AWS resources

What You'll Learn?

Understanding of AWS Glue Data Catalog and creating AWS Glue Database, Glue Tables and Crawlers
Using AWS Glue Studio, creating the ETL pipeline along with scheduled triggers, conditional triggers and glue workflow
KMS, IAM Role, SNS, S3 and other associated AWS resources associated with Glue. Understanding and creation of all the resources
Understanding of AWS Glue Data Quality and creating the associated Glue ETL pipeline
Understanding AWS Glue Data Brew , creating the recipe, project and job to curate the dataset
Understanding the AWS Glue streaming, creating the stream using the Python shell job and load the stream using the Spark streaming
Different ways AWS Glue job can fail and debugging the failure and fix
Creating the AWS resources for AWS Glue Pipeline using the AWS console and cloudformation

Who is this for?

Data Engineer, ETL Developer, Data warehouse developer or BI Develper who is moving from on premised to AWS cloud for ETL

Data Scientists who want to understand the Glue ETL concepts and curate the data

Software Development Engineer who wants to do ETL in the AWS cloud

More details

Description
Learn the latest in AWSÂ Glue - And learn to use it with other AWSÂ resources.
In this growing world of data and growing cloud computing, it is necessary to have the core competency in cloud ETLÂ tool also.Â AWSÂ Glue come with the in built Spark support, Data Quality and data curation using Data brew. The top technology, finance and insurance companies like JPMC, Vanguard, BCBS, Amazon, Capital One, Capgemini, FINRAÂ and more are all using AWSÂ GlueÂ to run their ETLÂ on PetaBytes scale of data everyday.
AWSÂ Glue provides server less and scalable ETLÂ solution where scripts can be written in Python, Spark and currently using Ray. It also provides the visual drag and drop options to create the ETL pipelines. As now more and more companies are migrating to cloud it has caused an explosion in demand for this skill! With the mastery of AWSÂ Glue, you now have the ability to quickly become one of the most knowledgeable people in the job market!
This course will teach the basics in AWS Glue Data Catalog, AWSÂ Glue Studio, AWSÂ resources such as IAM, SNS, KMS, CloudFormation, CloudWatch and continuing on to learning how to use AWSÂ Glue to build ETLÂ solution for the organization! Once we've done that we'll go through how to use the Glue Data Quality, Glue Streaming and Glue Data Brew ETLÂ pipelines. All along the way you'll have multiple labs to create all the resources and ETLÂ pipelines using AWSÂ console and CloudFormation templates that you put you right into a real world situation where you need to use your new skills to solve a real problem!
If you're ready to jump into the data engineering world of AWSÂ Glue, this is the course for you!

Who this course is for:
Data Engineer, ETL Developer, Data warehouse developer or BI Develper who is moving from on premised to AWS cloud for ETL
Data Scientists who want to understand the Glue ETL concepts and curate the data
Software Development Engineer who wants to do ETL in the AWS cloud

User Reviews

Rating

average 0

Total votes0

Focused display

Professional Data Engineers, Data Warehouse Developers and ETL developers with expertise in different ETL tools.We have years of experience in building complex ETL processes and Data Acquisition processes. We started our cloud journey long time back and would like share our learnings and experience with fellow learning enthusiast who is looking forward to start the cloud journey or want to use our cloud experience to learn something out of it. We believe in learning by doing it so we provide lot of practical assignment and lab in the course so students can learn by doing it.

Udemy

View courses Udemy

Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.