Build a Secure Data Lake in AWS using AWS Lake Formation

Focused View

Yomi Owoyemi

3:18:29

90 View

1 - Introduction

1 - Introduction to the course.mp4

11:42

2 - Setting up the Data Lake with AWS Lake formation

2 - Configuring S3 lake formation.mp4

08:41

2 - datalakediagram.zip

3 - Simple file ingestion into the data lake.mp4

13:05

3 - customers.csv

4 - Use blueprints and workflows in Lake formation for ingesting data from MySQL RDS.mp4

17:09

5 - Ingest realtime data using Kinesis firehose into the data lake.mp4

10:07

6 - Security and governance of our data lake with governed tables.mp4

14:21

3 - Preparation and analysis of data in our data lake using AWS Glue DataBrew

7 - Introduction to AWS Glue DataBrew.mp4

19:05

8 - Analysis and transformation of data in our data lake with Glue DataBrew.mp4

14:53

9 - Create DataBrew recipes and applying them to a larger datasets.mp4

10:19

4 - Author run and monitor ETL jobs using AWS Glue Studio

10 - Introduction to AWS Glue Studio.mp4

16:33

11 - Author ETL jobs for moving data between the different zones in our data lake.mp4

07:42

12 - Ingest data from DynamoDB into the data lake using AWS Glue and catalog it.mp4

09:50

5 - Prepare our data for analytics and reporting

13 - Introduction to Amazon Redshift and setting up our Amazon Redshfit cluster.mp4

15:38

14 - Author ETL job for moving data from our data lake into the Redshift warehouse.mp4

11:06

15 - Using Redshift Spectrum for querying data located in our data lake.mp4

04:33

6 - Bonus

16 - Introduction to Amazon Macie for managing data security and privacy in our lake.mp4

13:45

Description

Step by step guide for setting up a data lake in AWS using Lake formation, Glue, DataBrew, Athena, Redshift, Macie etc.

What You'll Learn?

How to quickly setup a data lake in AWS using AWS Lake formation
You will learn to build real-world data pipeline using AWS glue studio and ingest data from sources such as RDS, Kinesis Firehose and DynamoDB
You will learn how to transform data using AWS Glue Studio and AWS Glue DataBrew
You will acquire good data engineering skills in AWS using AWS lake formation, Glue Studio and, blueprints and workflows in lake formation

Who is this for?

Data Architects looking to architect data integration solutions in AWS cloud

Data Engineers

Anyone looking to start a career as an AWS Data Engineer

Data Scientist, Data Analyts and Database Administrators

IT professionals looking to move into the Data Engineering space

What You Need to Know?

Basic undestanding of cloud computing

Basic understanding of what a data lake and data warehouse are is essential but not required

An active AWS account is required to be able to follow along

More details

Description
In this course, we will be creating a data lake using AWS Lake Formation and bring data warehouse capabilites to the data lake to form the lakehouse architecture using Amazon Redshift. Using Lake Formation, we also collect and catalog data from different data sources, move the data into our S3 data lake, and then clean and classify them.
The course will follow a logical progression of a real world project implementation with hands on experience of setting upÂ a data lake,Â creating data pipelinesÂ for ingestion and transforming your data in preparation for analytics and reporting.

Chapter 1
Setup the data lake using lake formation
Create different data sources (MySQL RDS and Kinesis)
Ingest data from the MYSQL RDS data source into the data lake by setting up blueprint and workflow jobs in lake formation
Catalog our Database using crawlers
Use governed tables for managing access control and security
Query our data lake using Athena

Chapter 2,
Explore the use of AWS Gluw DataBrew for profiling and understanding our data before we starting performing complex ETL jobs.
Create Recipes for manipulating the data in our data lake using different transformations
Clean and normalise data
Run jobs to apply the recipes on all new data or larger datasets

Chapter 3
Introduce Glue Studio
Author and monitor ETL jobs for tranforming our data and moving themÂ between different zone of our data lake
Create a DynamoDB source and ingest data into our data lake using AWSÂ Glue

Chapter 4
Introduce and create a redshift cluster to bring datawarehouse capabilities to our data lake to form the lakehouse architecture
Create ETL jobs for moving data from our lake into the warehouse for analytics
Use redshift spectrum to query against data in our S3 data lake without the need for duplicating data or infrastructure

Chapter 5
Introduce Amazon Macie for managing data security and data privacy and ensure we can continue to identify sensitive data at scale as our data lake grows

Who this course is for:
Data Architects looking to architect data integration solutions in AWS cloud
Data Engineers
Anyone looking to start a career as an AWS Data Engineer
Data Scientist, Data Analyts and Database Administrators
IT professionals looking to move into the Data Engineering space

User Reviews

Rating

average 0

Total votes0

Focused display

Hi, I am a Technical Architect with over 10 years of industry and consulting exeprience in spearheading the management, design, development and implementation of large transformation projects in the cloud and on premise. I have experience working across a variety of industries including finance, technology, publishing and government ranging from SMEs to Fortune 500 companies.I am committed to establishing a warm, supportive, and inclusive learning experience to optimise student learning

Udemy

View courses Udemy

Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.