Data Lake, Firehose, Glue, Athena, S3 and AWS SDK for .NET

Focused View

Darren Cox

1:54:49

97 View

1 - Introduction

1 - ConsoleFirehoseTest.zip

1 - Introduction.mp4

01:18

2 - Overview of AWS Web Services in a Data Lake

2 - AWS Serverless pieces to our Data Lake jigsaw puzzle.mp4

04:54

3 - AWS RDS and S3

3 - S3 Bucket creation.mp4

01:56

4 - RDS creation.mp4

05:10

4 - AWS Kinesis Firehose C and the AWS SDK

5 - Creating Kinesis Data Firehose delivery streams.mp4

08:05

6 - Console NET core C entityframework project.mp4

10:54

7 - Kinesis Firehose C API class.mp4

06:23

8 - Kinesis Firehose C API send data example.mp4

06:30

5 - AWS Glue Athena ETL jobs Triggers and using Hangfire

9 - Creating Glue Crawlers.mp4

07:09

10 - Query Data in Athena using SQL.mp4

14:56

11 - Creating AWS ETL jobs.mp4

16:45

12 - Hangfire Automation.mp4

09:25

6 - AWS Lake Formation Cognito and Parquet NET

13 - AWS Lake Formation.mp4

02:24

14 - AWS Cognito.mp4

10:20

15 - Parquet NET Alternative to using Firehose.mp4

05:26

7 - Final thoughts and advice

16 - Conclusion.mp4

03:14

Description

Leverage AWS Kinesis Data Firehose, AWS Glue, S3, Athena and the AWS SDK to build a Data Lake.

What You'll Learn?

How to build a Data Lake using Firehose API for .NET, S3, AWS Glue and Athena

Who is this for?

AWS developers interested in learning about building a data lake using AWS services and the AWS SDK.

What You Need to Know?

You must have an active AWS Account to utilized the included code.

Prior knowledge of AWS Web Services would be helpful, but not needed.

More details

Description
The purpose of this class is to demonstrate a proof of concept using a series of lab exercise's (in the AWS Console using AWS Kinesis Data Firehose, AWS Glue, S3, Athena and the AWS SDK, with C# code using the AWS SDK) of building a Data Lake in the AWS ecosystem.
In this class, we will be sending data from a local SQL Server database to AWS RDS securely, and automatically.
I am utilizing the .NET SDK for AWS, however, this could easily be migrated to your language of choice once you understand the concepts that IÂ am teaching.
I will be providing the full working source code of this proof of concept.
I will also provide a working example of creating Parquet files and sending to S3 (without using Firehose).
Who this course is for:
AWS developers interested in learning about building a data lake using AWS services and the AWS SDK.

User Reviews

Rating

average 0

Total votes0

Focused display

Amazon AWS

Darren Cox

Instructor's Courses

I have been a .NET software developer for the past 20 years, creating many different applications, which include service applications, web applications, and client server applications.I have achieved two different AWS certifications: AWS Certified Solution Architect and AWS Certified Developer.I also love music, and I play the 5-string banjo (Scruggs style).

Udemy

View courses Udemy

Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.