Azure Synapse Analytics For Data Engineers -Hands On Project

Focused View

Ramesh Retnasamy

13:27:10

9 View

01 - Introduction

001 Course Introduction.mp4

08:19

002 Azure-Synapse-Course-Presentation.pdf

002 Course Materials Download.html

002 sql-spark-power-bi-code.zip

003 Useful Links.html

02 - Azure Subscription (Optional)

001 Creating Azure Account.mp4

05:52

002 Azure Portal Overview.mp4

03:52

03 - Azure Synapse Analytics Overview

001 Introduction to Azure Synapse Analytics.mp4

02:45

002 History of Data Warehouse Data Lakes.mp4

04:36

003 Emergence of Azure Synapse Analytics.mp4

09:00

004 Create Azure Synapse Analytics Workspace.mp4

10:18

005 Azure Synapse Analytics Workspace Overview.mp4

05:32

006 Azure Synapse Studio Overview.mp4

08:11

007 Data Hub Overview.mp4

08:03

007 taxi-zone.csv

008 Develop Hub Overview.mp4

06:11

009 Integrate Hub Overview.mp4

02:40

010 Monitor Hub Overview.mp4

02:28

011 Manage Hub Overview.mp4

05:12

04 - NYC Taxi Project Overview

001 Section Overview.mp4

01:04

002 NYC Taxi Data Source Overview.mp4

05:36

003 NYC Taxi Data Files Overview.mp4

02:45

004 Upload NYC Taxi Data to Data Lake.mp4

07:23

004 nyc-green-taxi-data.zip

005 Project Requirements Overview.mp4

03:29

006 Solution Architecture Overview.mp4

05:43

05 - Serverless SQL Pool - Overview

001 Section Overview.mp4

00:58

002 Introduction to Serverless SQL Pool.mp4

08:31

003 Serverless SQL Pool Cost Control.mp4

11:36

004 Connect from Azure Data Studio to Serverless SQL Pool (Optional).mp4

09:39

06 - Serverless SQL Pool - Query CSV

001 Section Overview.mp4

02:11

002 OPENROWSET Function Overview.mp4

04:21

003 Query Taxi Zone File (CSV File).mp4

08:21

004 Specify Data Types.mp4

08:25

005 Specify Collation.mp4

06:04

006 Query Subset of Columns.mp4

09:07

007 Debugging & Identifying Errors.mp4

02:30

008 Use External Data Source.mp4

09:24

009 Query Calendar File (CSV File) - Assignment.mp4

09:33

010 Query Vendor File (Quoted and Escaped Columns).mp4

06:32

011 Query Trip Type File (TSV File) - Assignment.mp4

02:43

07 - Serverless SQL Pool - Query JSON

001 Section Overview.mp4

03:34

002 Query Payment Type (Single Line JSON) - JSON VALUE Function.mp4

08:46

003 Query Payment Type (Single Line JSON) - OPENJSON Function.mp4

06:14

004 Query JSON Array.mp4

09:17

005 Query Standard JSON.mp4

05:07

006 Query Multi Line JSON (Assignment).mp4

02:24

08 - Serverless SQL Pool - Query Folders & Multiple Files

001 Query Folders and Subfolders.mp4

08:20

002 File Metadata Functions.mp4

10:06

09 - Serverless SQL Pool - Query Columnar Formats

001 Query Single Parquet File.mp4

08:04

002 Query Folders and Sub Folders (Assignment).mp4

03:36

003 Query Delta files.mp4

13:33

10 - Serverless SQL Pool - Data Discovery

001 Data Discovery Overview.mp4

03:21

002 Identify Duplicates.mp4

05:20

003 Data Quality Checks.mp4

09:04

004 Joining Files.mp4

08:29

005 Transform Data.mp4

06:11

006 Data Discovery Assignment.mp4

06:54

11 - Serverless SQL Pool - Data Virtualisation

001 Data Virtualisation Overview.mp4

04:09

002 Introduction to External Tables.mp4

09:50

003 Create External Table - CSV.mp4

19:37

004 Handling Rejections.mp4

15:49

005 Create External Table - CSV (Assignment).mp4

07:05

006 Create External Table - Parquet.mp4

04:18

007 Create External Table - Delta (Assignment).mp4

03:15

008 Views - Introduction.mp4

03:05

009 Create View for JSON files.mp4

05:43

010 Create View for JSON files (Assignment).mp4

02:12

011 Partition Pruning.mp4

08:47

12 - Serverless SQL Pool - Data Ingestion

001 Section Overview.mp4

01:01

002 CREATE EXTERNAL TABLE AS (CETAS).mp4

05:39

003 Transform to Parquet.mp4

08:14

004 Transform to Parquet (Assignment).mp4

01:40

005 Transform JSON to Parquet.mp4

03:02

006 Transform JSON to Parquet (Assignment).mp4

01:59

007 Transform Partitioned Data to Parquet.mp4

04:49

008 Introduction to Stored Procedure.mp4

03:20

009 Transform Partitioned Data - Solution.mp4

04:31

010 Transform Partitioned Data - Solution Lab.mp4

14:33

011 Create View (assignment).mp4

03:54

13 - Serverless SQL Pool - Data Transformation

001 Section Overview.mp4

00:43

002 Project Requirements.mp4

06:10

003 Select Statement.mp4

11:38

004 Stored Procedure.mp4

05:46

005 Create View.mp4

04:15

006 Assignment.mp4

07:31

14 - Synapse Pipelines & Triggers

001 Section Overview.mp4

01:13

002 Synapse Pipelines Overview.mp4

05:54

003 Synapse Pipeline Components.mp4

03:34

004 Transformation Pipeline Design - Taxi Zone.mp4

03:30

005 Create Linked Service & Dataset.mp4

09:44

006 Create Pipeline - Delete Activity.mp4

07:14

007 Create Pipeline - Script Activity.mp4

09:23

008 Create Pipeline - Stored Procedure Activity.mp4

05:53

009 Parameters & Variables Overview.mp4

04:08

010 Parameters & Variables (Demo).mp4

08:59

011 Dynamic Pipeline.mp4

10:20

012 Pipeline Design - Partitioned File.mp4

02:54

013 Create Pipeline (Partitioned File) - Delete Activity.mp4

11:25

014 Create Pipeline (Partitioned File) - Stored Procedure Script Activity.mp4

06:54

015 Create Pipeline (Partitioned File) - Assignment (Gold Table).mp4

03:02

016 Pipeline Dependencies.mp4

06:18

017 Manual Trigger.mp4

04:58

018 Automated Trigger.mp4

08:29

15 - Spark Pool

001 Section Overview.mp4

00:55

002 Spark Pool Overview.mp4

07:38

003 Create Spark Pool.mp4

09:52

004 Notebooks Overview.mp4

11:44

005 Integration between spark pool and serverless sql pool.mp4

04:10

006 Trip Data Aggregation.mp4

11:19

007 Create Synapse Pipeline.mp4

06:29

16 - Power BI Integration

001 Section Overview.mp4

00:46

002 Power BI Integration Overview.mp4

03:40

003 Connecting from PowerBI Desktop.mp4

08:03

004 Publish to PowerBI Workspace.mp4

09:28

005 Access to Power BI.mp4

08:10

006 Synapse Studio Power BI Integration.mp4

04:51

007 Report from Synapse Studio.mp4

11:09

008 Campaign Report.mp4

08:10

009 Demand Report.mp4

03:38

17 - Synapse Link HTAP

001 Section Overview.mp4

00:54

002 Synapse Link Overview.mp4

07:23

003 Project Overview.mp4

03:09

004 Create Cosmos DB Service.mp4

06:21

005 Data Preparation.mp4

08:59

006 Query using Serverless SQL Pool.mp4

08:15

007 Query using Spark Pool.mp4

05:36

18 - Dedicated SQL Pool

001 Section Overview.mp4

00:57

002 Dedicated SQL Pool Overview.mp4

07:52

003 Create dedicated SQL Pool.mp4

06:09

004 Project Requirement.mp4

04:03

005 Copy to Dedicated Pool - Polybase.mp4

12:06

006 Copy to Dedicated Pool - Copy Command.mp4

06:53

007 Connection to Azure Data Studio & PowerBI.mp4

03:59

19 - Next Steps

001 Good Luck.mp4

00:51

002 Bonus Lecture.html

Description

Hands on Project for Data Engineers using all the services available in Azure Synapse Analytics [DP-203, DP-500]

What You'll Learn?

You will learn how to build a real world project using Azure Synapse Analytics. This course has been taught using real world data from NYC Taxi Trips data
You will acquire professional level data engineering skills in Azure Synapse Analytics
You will learn how to create SQL scripts and Spark notebooks in Azure Synapse Analytics
You will learn how to create dedicated SQL pools and spark pools in Azure Synapse Analytics
You will learn how to enable synapse link and enable analytic store in Cosmos DB
You will learn how to ingest and transform data Serverless SQL Pool and Spark Pool
You will learn how to load data into dedicated SQL Pool
You will learn how to serve data to Power BI from Serverless SQL Pool and Dedicated SQL Pool
You will learn how to execute scripts and notebooks using Synapse Pipelines and Triggers
You will learn how to do operational reporting from the data stored in Cosmos DB using Azure Synapse Analytics
You will learn how to build reports in Power BI for the data stored in Azure Synapse Analytics

Who is this for?

University students looking for a career in Data Engineering

IT developers working on other disciplines trying to move to Data Engineering

Data Engineers/ Data Warehouse Developers currently working on on-premises technologies, or other cloud platforms such as AWS or GCP who want to learn Azure Data Technologies

Data Architects looking to gain an understanding about Azure Data Engineering stack

What You Need to Know?

All the code and step-by-step instructions are provided, but the skills below will greatly benefit your journey

Basic SQL knowledge will be required

Basic Python programming experience will be required

Knowledge of cloud fundamentals will be beneficial, but not necessary

Azure subscription will be required, If you don't have one we will create a free account in the course

More details

Description
Welcome!
I am looking forward to helping you with learning one of the in-demand data engineering tools in the cloud, Azure Synapse Analytics! This course has been taught with implementing a data engineering solution using Azure Synapse Analytics for a real world project of analysing and reporting on NYCÂ Taxi trips data.
This is like no other course in Udemy for Azure Synapse Analytics. Once you have completed the course including all the assignments, I strongly believe that you will be in a position to start a real world data engineering project on your own and also proficient on Azure Synapse Analytics.Â The primary focus of the course is Azure Synapse Analytics, but it also covers the relevant concepts and connectivity to the other technologies mentioned.
The course follows a logical progression of a real world project implementation with technical concepts being explained and the scripts and notebooks being built at the same time. Even though this course is not specifically designed to teach you the skills required for passing the exams Azure Data Engineer Associate Certification [DP-203] or Designing and Implementing Enterprise-Scale Analytics Solutions Using Microsoft Azure and Microsoft Power BI [DP-500], it can greatly help you get most of the necessary skills required for the exams.
I value your time as much as I do mine. So, I have designed this course to be fast-paced and to the point. Also, the course has been taught with simple English and no jargons. I start the course from basics and by the end of the course you will be proficient in the technologies used.
Currently the course teaches you the following
Azure Synapse Analytics Architecture
Serverless SQLÂ Pool
Spark Pool
Dedicated SQLÂ Pool
Synapse Pipelines
Synapse Link for Cosmos DB / Hybrid Transactional and Analytical Processing (HTAP) capability
PowerÂ BIÂ Integration with Azure Synapse Analytics
Azure Data Lake Storage Gen2 integration with Azure Synapse Analytics
Project using NYCÂ Taxi Trips data using the above technologies
Please note that the following are not currently covered
Data Flows
Advanced concepts around Dedicated SQLÂ Pool
Spark Programming
SQLÂ Fundamentals
Who this course is for:
University students looking for a career in Data Engineering
IT developers working on other disciplines trying to move to Data Engineering
Data Engineers/ Data Warehouse Developers currently working on on-premises technologies, or other cloud platforms such as AWS or GCP who want to learn Azure Data Technologies
Data Architects looking to gain an understanding about Azure Data Engineering stack

User Reviews

Rating

average 0

Total votes0

Focused display

Hello! I am a full time senior data engineer/ architect. I have over 20 years of experience delivering some of the large data projects for industries ranging from technology, gaming, finance, retail and government. The projects I delivered were both on cloud platforms such as Azure and AWS as well as On-premises. I am also a Microsoft certified Azure Data Engineer Associate.I have a passion for teaching and I take great pride in the success of my students. I have a different style of teaching than that of a standard I.T. trainer. My courses are based on real world projects. My courses will, not only explain the concepts, but also make them stick by using real world projects and examples. Throughout the course I give guidance on good practices and guide you towards building a production ready application. I strongly believe that once you have completed my courses you will have sufficient experience and knowledge required to start a real time project on that technology. Of course you may require further learning to progress in your career, but I will give you the required foundation and put you in the right direction to gain additional knowledge.I value your time as much as I do mine. So, I keep my courses to the point and my courses have been taught in simple English without any jargons.

Udemy

View courses Udemy

Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.