Learn Azure data factory and AWS GLUE (ETL)

Focused View

manish tiwari

9:48:01

119 View

1. Introduction

1. Introduction.mp4

00:48

2. Course Curriculum.mp4

02:11

2. DEMO PROJECT

1. AWS GLUE PROJECT.mp4

21:44

2.1 source.zip

2. ADF PROJECT.mp4

46:24

3. AWS GLUE

1. Introduction ETL.mp4

01:59

2. WHAT IS GLUE.mp4

02:23

3. GLUE BENEFITS.mp4

01:25

4. GLUE USE CASE.mp4

01:24

5. GLUE TERMINOLOGY.mp4

02:48

6. GLUE ARCHITECTURE.mp4

03:14

7. GLUE DEMO LAB.mp4

20:11

8. GLUE TRANSFORMATION LAB.mp4

22:49

9. GLUE TANSFORMATION LAB - MULTIPLE SOURCE.mp4

17:06

10. PROJECT -1 END TO END REAL TIME PROJECT (GLUE+S3+LAMBDA).mp4

17:57

11. PROJECT -2 END TO END REAL TIME PROJECT PARTITIONING.mp4

13:46

4. AWS (ATHENA)

1. Athena Introduction.mp4

02:35

2. how athena works.mp4

01:18

3. who is athena for and prereqisite.mp4

01:45

4. difference between sql server and athena.mp4

01:51

5. LAB-1 athena create table by crawler.mp4

11:34

6. Lab -2 create table without crawler and directly from s3.mp4

03:39

7. Project -1 superstore data analysis by using athena.mp4

11:22

8. Project -2 Partitioning using athena.mp4

07:16

5. ADF (COPY & DELETE activities)

1. Introduction- s3 to azure blob data pipeline.mp4

01:17

2. Setup- s3 to azure blob data pipeline.mp4

01:06

3. setup lab - s3 to azure blob data pipeline.mp4

08:44

4. pipeline- s3 to azure blob data pipeline.mp4

12:59

5. Introduction - blob to blob (copy & delete).mp4

01:24

6. Setup - blob to blob (copy & delete).mp4

00:59

7. Setup Lab - blob to blob (copy & delete).mp4

05:30

8. pipeline blob to blob (copy & delete).mp4

08:54

6. ADF ( SQL TO BLOB)

1. Introduction.mp4

01:31

2. SETUP.mp4

00:45

3. SETUP LAB.mp4

09:41

4. DATA PIPELINE.mp4

05:10

5. SQL table specific columns transfer to azure blob.mp4

01:39

6. Data Pipeline.mp4

10:36

7. ADF (CSV file to SQL table data pipeline)

1. Introduction and business use case.mp4

01:05

2. project setup lab.mp4

10:13

3. project create data pipeline.mp4

05:12

8. ADF (TRANSFORMATION)

1. Introduction- 2 azure blob storage to output.mp4

01:46

2. SETUP- 2 azure blob storage to output.mp4

04:41

3. Data pipeline.mp4

16:24

4. Introduction - Union.mp4

01:07

5. create data pipeline.mp4

12:38

6. Alter & filter.mp4

01:27

7. create data pipeline.mp4

13:18

9. ADF (INCREMENTAL LOAD)

1. introduction and business use case.mp4

01:48

2. setup lab.mp4

08:11

3. create data pipeline.mp4

07:46

10. ADF (Parameterization )

1. introduction and business use case.mp4

01:44

2. setup theory.mp4

01:00

3. setup lab.mp4

10:13

4. create data pipeline.mp4

12:39

11. ADF( SCD TYPE-1 & SCD TYPE 2)

1. Introduction -SCD TYPE-1.mp4

03:47

2. SETUP -SCD TYPE-1.mp4

10:13

3. PIPELINE -SCD TYPE-1.mp4

13:44

4. Introduction -SCD TYPE-2.mp4

05:57

5. SETUP -SCD TYPE-2.mp4

10:13

6. PIPELINE -SCD TYPE-2.mp4

20:27

12. ADF (FAILURE)

1. INTRODUCTION.mp4

01:00

2. PIPELINE.mp4

07:59

13. SQL

1. Introduction.mp4

07:02

2. sql commands.mp4

04:45

3. create table.mp4

10:39

4. alter.mp4

04:44

5. filter.mp4

05:59

6. delete drop.mp4

03:42

7. group by & having.mp4

05:45

8. aggregate funcion.mp4

02:27

9. update delete.mp4

02:32

10. sorting.mp4

03:10

11. between & in.mp4

02:47

12. like.mp4

02:18

13. distinct.mp4

01:24

14. union vs union all.mp4

02:59

15. join in sql.mp4

13:13

16. case satement.mp4

03:34

17. view.mp4

02:41

18. store procedure -1.mp4

02:56

19. store procedure -2.mp4

05:05

20. constrain in sql.mp4

08:04

21. constrain lab.mp4

11:41

22. row number, rank ,dense rank.mp4

08:18

Description

Master ETL in cloud with AWS glue & Azure data factory lab

What You'll Learn?

Confidently work with AWS Serverless services to develop Data Catalogue, ETL, Analytics on a Data Lake
Build a serverless data lake on AWS using structured and unstructured data
By end of this course you will be able to develop pipeline in azure data factory
how to perform transformation in azure data factory
you will learn how to develop incremental load pipeline
you will be able to understand about databricks architecture and how to use
Industry level SCD type-1,type-2 projects
Failure handling and debug pipeline and learn about email alerts and all

Who is this for?

Any one who wants to learn and get hands on azure data factory and databricks

Any one looking for job change in the data engineering field

Anyone who wants to learn AWS Serverless technologies for data and analytics should take this course

What You Need to Know?

Good to have Azure and AWS Account for the practical lab

Good internet and laptop/desktop

Willing to learn

More details

Description
In this course, we would learn the following:
1) We will start with Basics on Serverless Computing .
2) We will learn Schema Discovery, ETL, Scheduling, and Tools integration using Serverless AWS Glue Engine built on Spark environment.
3) We will learn to develop a centralized Data Catalogue too using Serverless AWS Glue Engine.
Businesses have always wanted to manage less infrastructure and more solutions. Big data challenges are continuously challenging the infrastructure boundaries. Having Serverless Storage, Serverless ETL, Serverless Analytics, and Serverless Reporting, all on one cloud platform had sounded too good to be true for a very long time. But now its a reality on AWS platform. AWS is the only cloud provider that has all the native serverless components for a true Serverless Data Lake Analytics solution.
This course understands your time is important, and so the course is designed to be laser-sharp on lecture timings, where all the trivial details are kept at a minimum and focus is kept on core content for experienced AWS Developers / Architects / Administrators. By the end of this course, you can feel assured and confident that you are future-proof for the next change and disruption sweeping the cloud industry.
I am very passionate about AWS Serverless computing on Data and Analytics platform, and am covering A-to-Z of all the topics discussed in this course.

This course will target anyone who likes to learn azure data factory and databricks . This course will cover all ADF components from the Model and View layer. In this course I have selected 15+ real time industry level project learning which will cover all the topic which is necessary to learn azure data factoryÂ and databricks and will be able to work on industry . This is completely hands on learning tutorial where we will do practical's based learning and by end of this course we will develop a complete ADF application together (step by step) andÂ By the end of this course, you should be able to develop a complete ADF application by yourself.
If you will be able to practical's with me you will be able to get complete picture of data factory
I have also included lessons on the storage blob Storage, amazon s3 , Azure SQL Database etc. Also, there are lessons onÂ Azure Databricks. I have even included lessons on building reports using Power BI on the data processed by the Azure Data Factory data pipelines.
Who this course is for:
Any one who wants to learn and get hands on azure data factory and databricks
Any one looking for job change in the data engineering field
Anyone who wants to learn AWS Serverless technologies for data and analytics should take this course

User Reviews

Rating

average 0

Total votes0

Focused display

Hi I am manish and I am a full time data engineer and having 5+ years of IT experiences delivering some of the large data projects for industries ranging from technology, security, finance, retail . The projects I delivered were both on cloud platforms such as Azure and AWS as well as On-premises.I have a passion for teaching and I take great pride in the success of my students. I have a different style of teaching than that of a standard I.T. trainer. My courses are based on real world projects. My courses will, not only explain the concepts, but also make them stick by using real world projects and examples. Throughout the course I give guidance on good practices and guide you towards building a production ready application. I strongly believe that once you have completed my courses you will have sufficient experience and knowledge required to start a real time project on that technology. Of course you may require further learning to progress in your career, but I will give you the required foundation and put you in the right direction to gain additional knowledge.I value your time as much as I do mine. So, I keep my courses to the point and my courses have been taught in simple English without any jargons.

Udemy

View courses Udemy

Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.