Mastering Databricks SQL Warehouse and Spark SQL

Focused View

Durga Viswanatha Raju Gadiraju,Asasri Manthena

7:27:57

179 View

1. Introduction to Mastering Databricks SQL Warehouse and Spark SQL

1. Introduction.html

2. Setup Course Material and Environment for Databricks SQL

1. Install Databricks CLI using Python 3 Pip Command.mp4

02:57

2. Configure Databricks CLI using Token and Profile.mp4

06:46

3. Setup Git Repository for Material and Data Sets related to Databricks SQL Course.mp4

01:54

3. Getting Started with Databricks SQL

1. Overview of Databricks SQL Platform - Introduction.mp4

03:33

2. Run First Query using SQL Editor of Databricks SQL.mp4

03:58

3. Overview of Dashboards using Databricks SQL.mp4

01:46

4. Overview of Databricks SQL Data Explorer to review Metastore Database and Tables.mp4

02:38

5. Use Databricks SQL Editor to develop scripts or queries.mp4

04:38

6. Review Metadata of Tables using Databricks SQL Platform.mp4

04:28

7. Overview of loading data into retail db tables.mp4

01:16

8. Configure Databricks CLI to push data into Databricks Platform.mp4

03:51

9. Copy JSON Data into DBFS using Databricks CLI.mp4

05:48

10. Analyze JSON Data using Spark APIs.mp4

05:57

11. Analyze Delta Table Schemas using Spark APIs.mp4

03:45

12. Load Data from Spark Data Frames into Delta Tables.mp4

02:55

13. Run Adhoc Queries using Databricks SQL Editor to validate data.mp4

03:59

14. Overview of External Tables using Databricks SQL.mp4

03:55

15. Using COPY Command to Copy Data into Delta Tables.mp4

07:19

16. Manage Databricks SQL Endpoints.mp4

04:17

4. Managing Databases using Databricks SQL Warehouse

1. Review Databases using Databricks SQL Data Explorer.mp4

03:08

2. Create Database or Schema using Databricks SQL.mp4

03:53

3. Using IF NOT EXISTS while Creating Databases using Databricks SQL.mp4

02:05

4. Listing or Showing Databases and Getting Metadata of Databases using Databricks.mp4

03:23

5. Understand Default Location of Databricks SQL Database or Schema.mp4

03:20

6. Create Database or Schema using Location in Databricks SQL Warehouse.mp4

03:09

7. Drop Databases in Databricks SQL Warehouse.mp4

04:49

8. Alter Database in Databricks SQL Warehouse.mp4

05:08

9. Comments on Databases in Databricks SQL Warehouse.mp4

05:41

5. Manage Delta Tables in Databricks SQL Warehouse

1. List Databases and Save Databricks SQL Script.mp4

02:10

2. Create Table using Delta Format in Databricks SQL Warehouse.mp4

04:41

3. Understand Location and Using Clause to specify File Format for Databricks.mp4

07:27

4. Create External Table using Delta Format in Databricks SQL Warehouse.mp4

04:26

5. Drop External Table and Delete Folder in Databricks SQL Warehouse.mp4

05:46

6. Overview of DML or CRUD Operations using Databricks SQL.mp4

02:26

7. Insert Records into Databricks SQL Warehouse table.mp4

07:58

8. Insert Multiple Records into Databricks SQL Warehouse table.mp4

04:17

9. Update Existing Records in Databricks SQL Warehouse table.mp4

04:31

10. Update Existing Records in Databricks SQL Warehouse table based on Null Values.mp4

02:56

11. Delete Existing Records in Databricks SQL Warehouse table.mp4

04:09

12. Cleanup Users Tables from Databricks SQL Warehouse Database or Schema.mp4

06:34

6. Setup Data Set for Databricks SQL Views and Copy Commands

1. Getting Started with Databricks fs Commands using Databricks CLI.mp4

04:06

2. Create Folder in DBFS using Databricks CLI Commands.mp4

02:31

3. Copy Files from Local File System into DBFS using Databricks CLI Commands.mp4

05:16

4. Overwrite Files while Copying into DBFS using Databricks CLI Command.mp4

04:33

5. Understand Course Catalog Data in the files uploaded to DBFS.mp4

05:46

6. Options to Analyze Data using Databricks SQL Queries.mp4

01:34

7. Run Select Queries using DBFS Path in From Cluase.mp4

05:00

8. Run Queries using Temporary Views in Databricks SQL.mp4

06:46

9. Run Queries using External Tables in Databricks SQL.mp4

03:39

7. Queries to Process Values in JSON String Columns

1. Queries to Process Values in JSON String Columns.mp4

01:56

2. Get Distinct and Count based on Key using Course Catalog Data.mp4

05:59

3. Filter Data using Basic Databricks SQL Queries using Course Catalog Data.mp4

05:09

4. Exploring Functions using Databricks SQL.mp4

06:22

5. Understand Record Column Values in Course Catalog Table.mp4

02:41

6. Processing JSON String Values using Databricks SQL Queries.mp4

07:37

7. Process Instructors JSON Records using Databricks SQL Queries.mp4

03:34

8. Create View for Instructors using Databricks SQL Queries.mp4

03:58

8. Copy Data into Delta Tables in Databricks SQL Warehouse

1. Create Delta Table for Course Catalog Data Set.mp4

04:46

2. Get File Names along with Data using Databricks SQL Queries.mp4

01:58

3. Overview of Databricks SQL COPY Command.mp4

03:07

4. Copy Data from single file into Delta Tables using Files.mp4

04:19

5. Copy Data from multiple files into Delta Tables using Files.mp4

02:35

6. Copy Data from multiple files into Delta Tables using Pattern.mp4

04:49

7. Create Course Catalog Table in Databricks SQL Warehouse with additional Column.mp4

03:50

8. Copy Data from Files using Queries into Delta Tables.mp4

06:16

9. Validate Course Catalog Table in Bronze Layer.mp4

04:12

9. Insert or Merge Query Results or View into Delta Tables using Databricks SQL

1. Introduction to Insert or Merge Query Results or View into Delta Tables using D.mp4

03:08

2. Create Course Catalog and Instructors Tables using Databricks SQL.mp4

06:35

3. Copy Data into Course Catalog Table from JSON Files using Databricks SQL.mp4

05:30

4. Insert Query Results into Delta Table using Databricks SQL.mp4

06:33

5. Exercise to Create Courses Table and Insert Data.mp4

02:14

6. Copy Instructors Data into Course Catalog Table from new file.mp4

03:23

7. Understand the Concept of Merge or Upsert in DML or CRUD Operations.mp4

02:04

8. Develop Query to Get the latest Instructors Records from Course Catalog Table.mp4

02:43

9. Overview of Merge Statement Syntax using Databricks SQL.mp4

04:40

10. Merge Data into Instructors Table from Course Catalog using Databricks SQL.mp4

09:15

11. Exercise to merge Courses Data from Course Catalog into Courses Table.mp4

03:35

10. Merge Query Results and Data from Delta Table with Delete into Delta Tables u

1. Introduction to Merge Query Results and Data from Delta Table with Delete in.mp4

02:13

2. Overview of Merge with Delete Real Life Scenario.mp4

03:21

3. Create Users Table using Delta format for Merge with Delete.mp4

07:31

4. Merge First Set of Records into Users Table.cmproj.mp4

05:28

5. Perform DML Operations on Users Table in Bronze Layer.mp4

05:26

6. Merge Updated Data into Users Table in Silver Layer.mp4

04:00

7. Scenarios for Merge into Databricks SQL Delta Tables.mp4

04:03

8. Reset Users Table to Merge Data Incrementally into the target table.mp4

04:17

9. Merge Into Databricks Table using Conditions on Updated Timestamp.mp4

07:54

10. Perform Operations to Insert and Update records in Bronze Users Table.mp4

06:29

11. Merge Incremental Data with Delete into Silver Users Table with Condition on.mp4

02:38

12. Exercise to Develop Merge Statement with Condition avoiding Column Names.mp4

02:48

11. Getting Started with Basic SQL Queries using Databricks SQL

1. Introduction to Getting Started with Basic SQL Queries using Databricks SQL.mp4

01:52

2. Exploring Databricks Data Sets.mp4

03:36

3. Create Database and Temporary Table to Preview the Data.mp4

03:16

4. Understand Characteristics of Online Retail Dataset.mp4

03:23

5. Create External Table with Columns and Datatypes.mp4

05:03

6. Get Count of Not Null Values of all Columns using Databricks SQL.mp4

06:32

7. Overview of Stack or Unpivot or Transpose of Data.mp4

02:10

8. Transpose Data using Stack in Databricks SQL.mp4

06:13

9. Transpose Not Null Counts into Multiple Rows using Databricks SQL.mp4

03:44

10. Overview of Unique or Distinct in Databricks SQL.mp4

02:58

11. Overview of Distinct to get unique records using Databricks SQL.mp4

05:00

12. Exercise - Get Distinct Count of Every Column.mp4

03:24

13. Create View using Query for Unique Records.mp4

05:07

14. Structure of SQL Query and Overview of Clauses.mp4

03:14

15. Overview of Comments in Databricks SQL Queries.mp4

06:41

Description

A Comprehensive Course on Databricks SQL Warehouse and Spark SQL for Data Engineers, Data Analysts, BI Developers, etc

What You'll Learn?

Setup Databricks SQL Warehouse Environment using Azure Databricks for hands-on Practice
Getting Started with Databricks SQL for Data Analysis or Data Engineering
Features of Databricks SQL Warehouse - Clusters, Query Editor, Visualizations and Dashboards, etc
Overview of building reports and dashboards using Databricks SQL
Creating Databases and Tables using Databricks SQL or Spark SQL
Writing Basic Queries using Databricks SQL or Spark SQL
DML to load data into Databricks SQL or Spark SQL Tables
Advanced Operations such as Ranking and Aggregations using Databricks SQL or Spark SQL
Processing Semi-Structured Data using Databricks SQL or Spark SQL
In-depth Coverage about Delta Tables including all possible DML Operations such as Insert, Update, Delete, Merge, etc
End to End Life Cycle of Data Analysis of Data in Files using Databricks (Uploading File to Databricks to Reports and Dashboards)

Who is this for?

Data Analysts and BI Developers who want to understand Databricks SQL or Spark SQL Queries to analyze the data

Data Engineers who would like to understand Databricks SQL

QA Analysts or Engineers who would like to understand Databricks SQL for the validation of the data

Business Analysts to analyze the Data in the Data Lake using Databricks SQL or Spark SQL Queries

BI Developers who want to understand how to connect BI Tools to Databricks SQL Endpoint and develop required reports and dashboards

More details

Description
Databricks SQL Warehouse is relatively new technology to build Data Lakehouse or Data Warehouse leveraging powerful Apache Spark Engine where the analytics can be built at scale. As part of this comprehensive course, you will learn all key skills required to master Databricks SQL Warehouse including Spark SQL as the SQLÂ in Databricks SQL Warehouse is based on Spark SQL.
This course also covers most of the curriculum relevant to clear the Databricks Certified Data Analyst Associate Exam offered by Databricks itself.
Here are the high-level details related to this course. This is a beginner level course where you will be able to not only learn syntax and semantics of Databricks SQLÂ or Spark SQL, you will also understand the concepts of the same.
Setup Course Material and Environment for Databricks SQLÂ Warehouse
Managing Databases using Databricks SQL Warehouse
Manage Delta Tables in Databricks SQLÂ Warehouse
Setup Data Set for Databricks SQL Views and Copy Commands
Databricks SQLÂ or Spark SQLÂ Queries to Process Values in JSON String Columns
Copy Data into Delta Tables in Databricks SQL Warehouse
Insert or Merge Spark SQLÂ or Databricks SQLÂ Query Results or View into Delta Tables
Merge Spark SQLÂ or Databricks SQLÂ Query Results and Data from Delta Table with Delete into Delta Tables
Basic SQL Queries using Spark SQLÂ or Databricks SQL
Performing Aggregations using Group By and filtering using Having leveraging Spark SQLÂ or Databricks SQL
Aggregations using Windowing or Analytical Functions including Cumulative Aggregations using Spark SQLÂ or Databricks SQL
Ranking using Windowing or Analytical Functions using Spark SQLÂ or Databricks SQL
Dealing with different file formats such as parquet, json, csv, etc using Spark SQLÂ or Databricks SQL
All Important types of Joins such as Inner, left or right outer, full outer using Spark SQLÂ or Databricks SQL
Visualizations and Dashboards using Databricks SQL Warehouse
We have also provided quite a few exercises along with solutions with explanations through the course.
Key Takeaways of Mastering Databricks SQL and Spark SQLÂ using Databricks SQLÂ Warehouse
Setup Environment to learn Databricks SQL and Spark SQLÂ using Azure
Support via Udemy Q&A backed by our expert team
Data Sets and Material via GitHub Repository along with instructions to practice Databricks SQLÂ or Spark SQL
Life Time Access to High Quality Video Lectures to learn Databricks SQL and Spark SQL
Who this course is for:
Data Analysts and BI Developers who want to understand Databricks SQL or Spark SQL Queries to analyze the data
Data Engineers who would like to understand Databricks SQL
QA Analysts or Engineers who would like to understand Databricks SQL for the validation of the data
Business Analysts to analyze the Data in the Data Lake using Databricks SQL or Spark SQL Queries
BI Developers who want to understand how to connect BI Tools to Databricks SQL Endpoint and develop required reports and dashboards

User Reviews

Rating

average 0

Total votes0

Focused display

20+ years of experience in executing complex projects using a vast array of technologies including Big Data and the Cloud.ITVersity, Inc. - is a US-based organization that provides quality training for IT professionals and we have a track record of training hundreds of thousands of professionals globally.Building an IT career for people with required tools such as high-quality material, labs, live support, etc to upskill and cross-skill is paramount for our organization.At this time our training offerings are focused on the following areas:* Application Development using Python and SQL* Big Data and Business Intelligence* Cloud* Datawarehousing, Databases

Asasri Manthena

Instructor's Courses

3+ years of overall experience and currently working with ITVersity, Inc. An MBA graduate from SDA Bocconi School of Management.ITVersity, Inc. - is a US-based organization that provides quality training for IT professionals and we have a track record of training hundreds of thousands of professionals globally.Building an IT career for people with required tools such as high-quality material, labs, live support, etc to upskill and cross-skill is paramount for our organization.At this time our training offerings are focused on the following areas:* Application Development using Python and SQL* Big Data and Business Intelligence* Cloud* Datawarehousing, Databases

Udemy

View courses Udemy

Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.