Learn Data Engineering with Databricks on AWS Cloud

Focused View

SKILL CURB

4:50:26

26 View

1 - Getting Startted with Databricks on AWS

1 - Introduction to Getting Started with Databricks on AWS.mp4

01:22

2 - Signing up for aws free account.mp4

03:36

3 - Logging in into AWS Management Console.mp4

02:42

4 - Setting up Databricks workspace on AWS using Quickstart.mp4

07:53

5 - Logging in into Databricks Workspace on AWS.mp4

03:07

6 - Cleaning up the workspace and resources.mp4

03:20

7 - Quick Walkthrough of Databricks UI on AWS.mp4

02:05

8 - Creating SingleNode Databricks cluster on AWS.mp4

05:34

9 - Upload Data using AWS Databricks UI.mp4

09:00

10 - Develop spark Application using AWS Databricks Notebook.mp4

16:00

11 - Writing dataframe to DBFS.mp4

06:44

12 - Export and Import AWS Databricks Notebooks.mp4

04:36

2 - AWS Storage Solutions

13 - Getting Started with AWS S3.mp4

05:21

14 - Overview of AWS S3 Glacier.mp4

05:19

15 - Creating S3 Bucket and adding Objects.mp4

05:42

16 - Version Control in AWS S3.mp4

09:18

17 - AWS S3 CrossRegion Replication for Fault Tolerance.mp4

07:40

18 - Setup and configure AWS S3 CLI using IAM user credentials.mp4

06:22

19 - Managing Objects in AWS S3 using CLI.mp4

13:08

3 - AWS S3 and IAM Role

20 - Overview of IAM for Databricks on AWS.mp4

02:51

21 - Creating AWS IAM User.mp4

05:32

22 - Logging into AWS Management Console using IAM User.mp4

02:00

23 - Validate Programmatic Access to AWS IAM User3.mp4

02:57

24 - AWS IAM Identitybased policies.mp4

09:56

25 - AWS IAM User Groups.mp4

02:53

26 - AWS IAM Custom Policies.mp4

09:30

4 - Integration S3 and Glue Catalog

27 - Introduction to Integrating AWS s3 and Glue Catalog with Databricks.mp4

01:38

28 - Create AWS IAM Group for Databricks Developers.mp4

02:08

29 - Creating AWS IAM Users and adding to group.mp4

02:37

30 - Creating AWS s3 Bucket for Databricks Developers.mp4

01:53

31 - Grant Permissions on AWS S3 Bucket to the users in group.mp4

09:09

32 - Attach AWS IAM Policy to grant access to Glue.mp4

04:15

33 - Upload JSON Dataset to s3 to crawl using AWS Glue Crawler.mp4

03:10

34 - Create AWS IAM Custom Service Role for Glue Crawlers.mp4

10:33

35 - Create and Run Glue Crawler to Create Multiple Glue Catalog Tables.mp4

05:01

36 - Overview of Integration of Databricks Clusters and AWS EC2 Instances.mp4

03:49

37 - Create AWS IAM Role or Instance Profile.mp4

03:48

38 - Registering AWS IAM Instance Profile with Databricks Account.mp4

03:04

39 - Attach AWS IAM Instance Profile to Databricks Cluster.mp4

08:14

40 - Grant Permissions on S3 to Databricks Clusters.mp4

06:27

41 - Integrate Databricks Cluster with Glue Catalog via Instance Profile.mp4

11:17

5 - Setup local development environment for databricks

42 - Setup single node databricks cluster.mp4

03:03

43 - Install Databricks Connect.mp4

04:57

44 - Configure Databricks Connect.mp4

05:07

6 - Using Databricks CLI

45 - Install and configure databricks CLI.mp4

06:37

46 - Interacting with File System using Databricks CLI.mp4

11:02

47 - Getting Cluster details using Databricks CLI.mp4

05:55

7 - Spark Jobs Deployment using Notebooks

48 - Modularizing Notebooks.mp4

05:24

49 - Running Job using Notebook.mp4

03:41

50 - Refactor application as Databricks Notebooks.mp4

03:18

51 - Run Notebooks using Development Cluster.mp4

09:51

Description

Build Data Engineering Pipelines on AWS using Databricks core features such as Spark and Delta Lake

What You'll Learn?

Data Engineering leveraging Databricks features
Databricks CLI to manage files, Data Engineering jobs and clusters for Data Engineering Pipelines
Deploying Data Engineering applications developed using PySpark on job clusters.
Deploying Data Engineering applications developed using PySpark using Notebooks on job clusters
Building Data Engineering Pipelines using Spark Structured Streaming on Databricks Clusters
Setting up development environment to develop Data Engineering applications using Databricks.

Who is this for?

Software engineers, aspiring data engineers or data analyst & data scientists

Programmers and Database Administrators with experience in writing SQL queries

BI Analysts looking to enhance their understanding of data engineering, particularly in the context of big data and cloud platforms, can leverage this course to broaden their skill set.

IT professionals who want to transition into roles focused on data engineering and cloud computing

What You Need to Know?

Must have Prior programming experience in SQL and Python.

Experience in Spark Programming is a must.

Prior basic experience or understanding of cloud services like AWS is important.

More details

Description
Get ready to Learn Data Engineering with Databricks on AWS Cloud with this complete course. Gain familiarity with the course details and topics designed to help you succeed.

This comprehensive course is designed to equip you with the skills and knowledge needed to excel in the field of data engineering using two powerful platforms: Databricks and Amazon Web Services (AWS). Data engineering is the backbone of any successful data-driven initiative, and Databricks, a unified analytics platform, has emerged as a leading choice for data engineers and data scientists worldwide. When combined with AWS, a cloud computing powerhouse, you have a robust ecosystem that can handle data at scale, provide advanced analytics capabilities, and support a wide range of data sources and formats.

Learn about Data Engineering with Databricks on AWS with Hands-On Labs
Learn Data Engineering with Databricks on AWS Cloud is a hands-on practice course designed to familiarize you with the core functionality of Databricks by connecting it with AWS to perform Data Engineering. Through hands-on exercises, you'll gain a thorough understanding of Databrick's architecture and how it revolutionizes data engineering in the cloud. You'll explore the seamless integration of Databricks with AWS services, such as Amazon S3 and Glue, unlocking a world of possibilities for managing and analyzing your data.
This course has been meticulously designed to provide you with both a solid theoretical foundation and extensive hands-on practice in the dynamic realms of data engineering, Databricks, and Amazon Web Services (AWS).
The course comprises approximately 50 labs starting from the basics and moving to high levels in terms of complexity.
Who should take this course?
The course "Learn Data Engineering with Databricks on AWS Cloud" is designed for a wide range of individuals who are interested in building expertise in data engineering using Databricks on the AWS Cloud. If you're looking to start a career in data engineering, this course is an excellent choice. It will provide you with the foundational knowledge and practical skills needed to become a successful data engineer. Data scientists and analysts who want to expand their skill set and be able to work with large-scale data processing, data pipelines, and data lakes can greatly benefit from this course. IT professionals who want to transition into roles focused on data engineering and cloud computing can use this course as a stepping stone to acquire the necessary skills and knowledge. Individuals interested in cloud computing, specifically AWS, and its applications in data engineering will gain a deep understanding of cloud-based data engineering solutions.
Requirements
â— Basic knowledge of SQL or writing queries in any language
â— Scripting in Python Willingness to explore, learn, and put in the extra effort to succeed
â— An active AWS Account & know-how of basic cloud fundamentals
â— Programming experience using Python
â— Data Engineering experience using Spark

Who this course is for:
Software engineers, aspiring data engineers or data analyst & data scientists
Programmers and Database Administrators with experience in writing SQL queries
BI Analysts looking to enhance their understanding of data engineering, particularly in the context of big data and cloud platforms, can leverage this course to broaden their skill set.
IT professionals who want to transition into roles focused on data engineering and cloud computing

User Reviews

Rating

average 0

Total votes0

Focused display

SkillCurb is in the business of making an impact through creating easy to understand cutting edge technology courses at affordable prices. We are proud to have our co-instructors from around the world that are certified and have years of practical experience working for the fortune 500 companies build quality courses for our students.We focus on the next generation of technology platforms that will prepare our workforce for what's next to come in the future and secure a job in this competitive market.One of our goals is to set aside part of our revenues to charitable causes that focus on providing education to under-privileged students.Some of the key differentiators of our online courses:1: Our instructors are passionate about technology and have been teaching on average for more than 10 years individually.2: We make sure that our Practical Labs in the courses are the center of focus where students can learn to implement real-life use cases necessary to succeed at work.3: All our courses have Capstone projects in the end where students can apply the skills learnt during the course and create awesome projects which can be showcased on their resumes and interviews.4: All our lectures have Quizzes in the end which forces the student to start applying the key principles learnt.5: We make sure that all complex ideas are broken down and Visually Explained for the students rather than showing them long winded text which at times is hard to conceptualize.6: We make sure that our instructors communicate clearly and are highly proficient in delivering lectures in English.7: We also try to add bonus reference material for the students that they can use for future reference i.e interview questions with answers and additional coding challenges with solutions.

Udemy

View courses Udemy

Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.