Azure Data Engineer Associate (DP-203) Cert Prep: 4 Monitor and Optimize Data Storage and Data Processing

Focused View

Noah Gift

34:25

78 View

01 - Introduction

01 - Course introduction.mp4

02:35

02 - 1. Monitor Data Storage

01 - Implement logging used by Azure Monitor.mp4

01:50

02 - Configure monitoring services.mp4

01:47

03 - Measure performance of data movement.mp4

02:45

04 - Monitor data systempipelinecluster performance.mp4

02:29

05 - Measure query performance.mp4

03:26

06 - Schedule and monitor pipeline tests.mp4

02:45

07 - Interpret a Spark directed acyclic graph (DAG).mp4

02:35

03 - 2. Optimize

01 - Rewrite user-defined functions (UDFs).mp4

01:53

02 - Handle skew in data and data spill.mp4

01:20

03 - Tune shuffle partitionspipelines.mp4

01:32

04 - Optimize resource management.mp4

02:49

05 - Tune queries by using indexers and cache.mp4

01:54

06 - Troubleshoot a failed Spark job and pipeline run.mp4

02:54

04 - Conclusion

01 - Summary and next steps.mp4

01:51

Description

Are you preparing for the Microsoft Azure Data Engineering (DP-203) exam, or seeking a better understanding of how to design and develop data processing? This course, the fourth in a series, can help you. Noah Gift, founder of Pragmatic A.I. Labs and a Python Software Foundation Fellow, covers how to monitor and optimize data storage and data processing. Noah first covers how to monitor data storage and data processing, showing how to: implement logging used by Azure Monitor; configure monitoring services; measure query performance; and more. He then details topics pertaining to optimizing and troubleshooting data storage and processing, including how to: rewrite user-defined functions; tune queries by using indexers and cache; handle skew in data and data spill; troubleshoot a failed Spark job and pipeline run.

More details

User Reviews

Rating

average 0

Total votes0

Focused display

Microsoft Azure

Microsoft Certification

Noah Gift

Instructor's Courses

Author of Practical MLOps, Enterprise MLOps, Developing on AWS with C#, Pragmatic AI, and Python for DevOps. Certified on Multiple MLOps certifications including Google-Professional Machine Learning Engineer and AWS Certified Machine Learning - Specialty. Adjunct Professor at Duke MIDS & Northwestern Graduate Data Science & AI. Held business roles including CTO, general manager, consulting CTO, and cloud architect. Consults with start-ups and other companies on machine learning and cloud architecture. AWS ML Hero, Python Software Foundation Fellow, AWS Subject Matter on Machine Learning, AWS Certified Solutions Architect, AWS Certified Machine Learning Specialist, AWS Certified Big Data Specialist, Google Certified Professional Architect, and AWS Academy Accredited Instructor. Published books, videos, and courses on cloud machine learning, DevOps, Python, Data Science, Big Data and AI. ★ Specialties ★ ° Cloud-native Machine Learning and AI ° Directly teaching cutting edge skills to students that lead to jobs ° Creating world-class content in all forms ° Building Companies ° Shipping new Products ° Leading and growing engineering teams ° Production Machine Learning, Deep Learning, Big Data, and AI ° Serverless Data Engineering ° Advising Early Stage Startups/Consulting CTO services ° Distributed Systems and Scalability

Linkedin Learning

View courses Linkedin Learning

LinkedIn Learning is an American online learning provider. It provides video courses taught by industry experts in software, creative, and business skills. It is a subsidiary of LinkedIn. All the courses on LinkedIn fall into four categories: Business, Creative, Technology and Certifications. It was founded in 1995 by Lynda Weinman as Lynda.com before being acquired by LinkedIn in 2015. Microsoft acquired LinkedIn in December 2016.