Data Engineering 101: The Beginner's Guide

Focused View

Seungchan Lee,Nami Kim

3:47:08

311 View

1. Introduction to data engineering

1. Introduction.mp4

05:05

2. What is data engineering and why is it important.mp4

07:35

3. Overall archiecture of end-to-end data pipeline.mp4

04:22

4. Section 1 - Quiz 1.html

5. Some historical context.mp4

05:40

6. Data maturity.mp4

02:51

7. Data engineers place within a data team.mp4

04:37

8. Section 1 - Quiz 2.html

2. End-to-end data pipeline in-depth

1. Generation (source systems) Intro.mp4

02:27

2. Generation (source systems) Types of databases.mp4

05:06

3. Section 2 - Quiz 1.html

4. Generation (source systems) Third-party systems and event streams.mp4

06:41

5. Storage Intro.mp4

04:03

6. Storage Serialization (storage file formats), compression, and caching.mp4

03:27

7. Storage Distributed storage.mp4

06:03

8. Storage Types of storage systems.mp4

04:34

9. Storage Row-based vs columnar storage (or OLTP vs OLAP).mp4

05:36

10. Storage Data warehouse, data lake, and data lakehouse.mp4

07:26

11. Section 2 - Quiz 2.html

12. Ingestion Batchmicro-batchstreaming ingestion and ETL vs ELT.mp4

04:10

13. Ingestion Streaming ingestion.mp4

03:03

14. Ingestion Scalability and upstream schema changes.mp4

02:02

15. Ingestion Different ways to ingest data.mp4

05:24

16. Transformation Queries.mp4

07:25

17. Transformation Data modeling.mp4

10:53

18. Transformation Queries and transformations.mp4

03:53

19. Transformation Events vs states and batch vs streaming.mp4

02:21

20. Serving.mp4

08:47

21. Section 2 - Quiz 3.html

3. Undercurrents

1. DataOps.mp4

03:59

2. Orchestration.mp4

06:15

3. Security, privacy, data quality.mp4

06:40

4. Development workflow.mp4

03:33

5. Section 3 - Quiz 1.html

4. Data pipeline architecture examples

1. What is a good data architecture.mp4

06:01

2. What factors should you consider when designing your data architecture.mp4

03:39

3. BI stack (data warehouse) Storage.mp4

06:42

4. BI stack (data warehouse) DuckDB.mp4

01:36

5. BI stack (data warehouse) Orchestrator.mp4

02:21

6. BI stack (data warehouse) Ingestion.mp4

02:51

7. BI stack (data warehouse) Transformation and visualization.mp4

02:14

8. BI stack (data lakehouse) Storage - Apache Iceberg.mp4

04:27

9. BI stack (data lakehouse) Compute engine - Trino.mp4

02:43

10. Section 4 - Quiz 1.html

11. Streaming stack Event ingestion.mp4

03:30

12. Streaming stack Event processing.mp4

03:48

13. ML stack The basics of ML and AI.mp4

08:20

14. ML stack Feature store.mp4

02:47

15. ML stack Model training.mp4

03:43

16. ML stack Model serving (a.k.a. model inference).mp4

04:26

17. Deep learning stack Intro.mp4

02:19

18. Deep learning stack Labeling, training, and inference.mp4

04:55

19. Other considerations to building a data pipeline.mp4

05:11

20. Section 4 - Quiz 2.html

5. Whats next

1. Future of data engineering Our opinion.mp4

08:01

2. Whats next.mp4

03:36

Description

Master Modern Data Engineering Fundamentals

What You'll Learn?

Get an intuitive understanding of data engineering
Understand the most critical concepts in data engineering
Learn what the end-to-end data pipeline is comprised of and what each part of the pipeline does
Solidify your understanding with example end-to-end data pipeline architectures for the most popular use cases

Who is this for?

Beginners curious about building a career in data engineering.

Data professionals like Data Scientists or Data Analysts seeking to understand the fundamentals of modern data engineering.

Anyone who wants to learn about what role Data Engineering plays in data-driven businesses.

What You Need to Know?

No prior knowledge of data engineering required. You'll learn everything you need to know.

No programming experience required.

More details

Description
Master Modern Data Engineering Fundamentals
Are you curious about data engineering but unsure where to start?
Are you a software engineer, data scientist, or data analyst who wants to learn more about modern data engineering?
Dive into our intro course where we demystify the complexities of the field and give you a solid foundation in modern data engineering.
This course is tailored specifically for those who are new to the field, providing a clear and concise introduction to the essential concepts and tools used in modern data engineering today.

What You'll Learn:
What Exactly Is Data Engineering? Understand the core concept of data pipelines and the role of data engineering in a wider data team.
End-to-end data pipeline: Explore each part of the end-to-end data pipeline from data generation, storage, ingestion, transformation, and serving. Learn how data flows from creation to consumption.
Critical Data Engineering Concepts: Learn the most important concepts in data engineering, such as data warehouse vs data lakehouse, row-based vs column-based data stores, ELT vs ETL and more.
Introduction to Modern Data Engineering Tools: See how modern data tools like Dagster, Trino, DBT, and Apache Iceberg are used to build out the end-to-end data pipelines.
Modern Data Stack Architecture Examples: Get a solid introduction to the tools and technologies that data engineers use to architect end-to-end modern data pipelines for the four most common use cases: 1) Business analytics, 2) Streaming, 3) ML, and 4) Deep Learning.

Who Should Take This Course:
Beginners curious about building a career in data engineering.
Data professionals like Data Scientists or Data Analysts seeking to understand the fundamentals of modern data engineering.
Anyone who wants to learn about what role Data Engineering plays in data-driven businesses.

Why Take This Course?
This course is designed as a gentle introduction to the field of data engineering. It breaks down data engineering into clear sections within a data pipeline so you can see what data engineering is all about.
You will leave with a clear understanding of what data engineering is and a solid foundation to further explore more advanced topics or actually build out the modern data stack.

What This Course Is NOT:
Before you take this course, you should know what this course is NOT. This course is NOT a tutorial. So if youâ€™re expecting to learn about how to set up and use tools like Spark or Airflow, or write SQL statements, this is not the course for you.
Thereâ€™s no docker setup, thereâ€™s no tool installations, and thereâ€™s no coding.
If youâ€™re looking for that kind of tutorial course, take a look at our more advanced data engineering courses where we walk you through how to actually build end-to-end data pipelines using cutting-edge open-source tools.

Join us on this exciting journey to discover the fundamentals of data engineering. Whether you're planning a career shift or just looking to broaden your technological horizons, this course will provide you with the knowledge and tools you need to succeed.
Ready to transform the way you see data? Join the course and letâ€™s get started!
Who this course is for:
Beginners curious about building a career in data engineering.
Data professionals like Data Scientists or Data Analysts seeking to understand the fundamentals of modern data engineering.
Anyone who wants to learn about what role Data Engineering plays in data-driven businesses.

User Reviews

Rating

average 0

Total votes0

Focused display

Data Science

Seungchan Lee

Instructor's Courses

I'm currently working on two projects. First, DeepIntuitions is my attempt at bringing more talent into the field of AI. I feel not enough courses out there are kind enough to newcomers in this field. DeepIntuitions course focuses heavily on an intuitive understanding of Deep Learning and aims to make the subject a lot more approachable to people just starting out.My other project is Sidetrek - we help data/ML/AI teams build a modern data/ML/AI pipeline 10x easier and faster using open-source tools.

Nami Kim

Instructor's Courses

My current passion is to learn AI/ML and share my knowledge with many people. I co-created a deep learning class with Seungchan, helping him with course content and materials.My other focus is Sidetrek - we help data/ML/AI teams build a modern data/ML/AI pipeline 10x easier and faster using open-source tools.Prior to Sidetrek, I worked as a product manager at Adobe and a management consultant at McKinsey & Company. I believe in multidisciplinary approach in learning and applying AI.

Udemy

View courses Udemy

Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.