Data Engineering Foundations

Focused View

Harshit Tyagi

1:03:51

125 View

01 - Introduction

01 - What is data engineering.mp4

01:15

02 - Introduction to Data Engineering

01 - Introduction to data engineering.mp4

02:50

02 - Data engineer vs. data scientist.mp4

02:02

03 - Essential tools for data engineering.mp4

04:06

03 - Databases and Dataframes

01 - Intro to databases and their types.mp4

04:04

02 - Understanding database schema.mp4

02:48

03 - Distributive computing.mp4

05:10

04 - Data Engineering Tools

01 - MapReduce and Hadoop.mp4

02:03

02 - Hive.mp4

01:41

03 - Spark.mp4

02:49

04 - Airflow.mp4

04:21

05 - ETL Pipelines

01 - Sources of data extraction.mp4

04:46

02 - Data extraction from a PostgreSQL database.mp4

04:51

03 - Challenge Data extraction.mp4

00:40

04 - Solution Data extraction.mp4

00:51

05 - Transforming data.mp4

02:03

06 - Challenge Transforming data.mp4

00:42

07 - Solution Transforming data.mp4

00:58

08 - Loading data into a DB.mp4

04:11

09 - Challenge Loading data.mp4

00:59

10 - Solution Loading data.mp4

01:00

11 - Scheduling ETL pipeline using Airflow.mp4

09:03

06 - Conclusion

01 - Next steps.mp4

00:38

Exercise Files

Ex_Files_Data_Engineering_Foundations.zip

Description

Data science can be generally defined as the process of making data useful, and data engineering is a key part of how and why. If you think of data science like a race car, the data engineers are the pit crew. They’re not driving the car, but they make the car much easier to drive. Data engineers make sure the data flow is running smoothly, monitor systems, anticipate problems, and repair the data pipeline whenever problems arise. They extract and gather data from multiple sources and load it into a single, easy-to-query database. In short, data engineers make data scientists’ lives easier.

In this course, Harshit Tyagi explains the fundamentals of data engineering. He covers key topics like data wrangling, database schema, and developing ETL pipelines. He also details several data engineering tools like Hive, Hadoop, Spark, and Airflow. By the end of this course, it should be abundantly clear why the data engineer is one of the most valuable people in a data-driven organization.

More details

User Reviews

Rating

average 0

Total votes0

Focused display

Data Science

Data Architecture

Data Modeling

Harshit Tyagi

Instructor's Courses

- An engineer with amalgamated experience in web technologies and data science(aka full-stack data science). - Mentored over 10K AI/Web/Data Science aspirants. - Designing data science, data engineering and ML engineering learning tracks. - Previously, developed data processing algorithms with research scientists at Yale, MIT, and UCLA.

Linkedin Learning

View courses Linkedin Learning

LinkedIn Learning is an American online learning provider. It provides video courses taught by industry experts in software, creative, and business skills. It is a subsidiary of LinkedIn. All the courses on LinkedIn fall into four categories: Business, Creative, Technology and Certifications. It was founded in 1995 by Lynda Weinman as Lynda.com before being acquired by LinkedIn in 2015. Microsoft acquired LinkedIn in December 2016.