Executing Graph Algorithms with GraphFrames on Databricks

Focused View

Janani Ravi

1:34:19

26 View

01. Course Overview

01. Course Overview.mp4

02:03

02. Getting Started with Graph Algorithms in Spark

02. Prerequisites and Course Outline.mp4

01:55

03. Graphs to Model Relationships.mp4

03:24

04. Directed and Undirected Graphs.mp4

02:10

05. Adjacent Nodes Paths Degree of a Node.mp4

03:10

06. GraphFrames on Databricks.mp4

03:16

07. Demo-Installing GraphFrames and NetworkX.mp4

02:30

08. Demo-Creating GraphFrames.mp4

05:33

09. Demo-Indegree and Outdegree.mp4

01:37

10. Demo-Filtering Vertices and Edges.mp4

05:44

03. Stateful Queries and Motifs

11. Motif Finding.mp4

01:49

12. Domain-specific Language for Motif Finding.mp4

02:20

13. Demo-Motif Finding.mp4

03:47

14. Demo-Anonymous Edges Anonymous Vertices and Negated Terms.mp4

03:47

15. Demo-Motif Finding in a Real-world Dataset.mp4

04:44

16. Demo-Motif Finding with Complex Patterns.mp4

03:37

17. Demo-Stateful Queries.mp4

04:51

18. Demo-Sub Graphs.mp4

03:49

04. Implementing Graph Algorithms

19. Breadth-first and Depth-first Traversal.mp4

06:24

20. Demo-Breadth-first Search.mp4

03:49

21. Shortest Path.mp4

04:09

22. Demo-Shortest Path.mp4

02:42

23. Demo-Triangle Count.mp4

03:19

24. Connected Components and Strongly Connected Components.mp4

02:02

25. Demo-Connected Components and Strongly Connected Components.mp4

04:09

26. PageRank.mp4

01:36

27. Demo-PageRank.mp4

04:32

28. Summary and Further Study.mp4

01:31

Description

This course will teach you how to create and represent graph data using GraphFrames in Apache Spark and implement graph algorithms such as Shortest Path and PageRank on Azure Databricks.

What You'll Learn?

The Spark unified analytics engine is one of the most popular frameworks for big data analytics and processing. The GraphFrames package in Apache Spark allows you to represent graphs using a DataFrame-based API. GraphFrames also supports a number of graph algorithms such as Shortest Path, PageRank, Breadth-first search, and connected components.

In this course, Executing Graph Algorithms with GraphFrames on Databricks, you will explore how graphs can be used to model entities and relationships in the real world. First, you will learn about the different kinds of graphs such as directed and undirected graphs, weighted and unweighted graphs. Then, you will discover how graphs can be represented using the GraphFrames API in Apache Spark and how you can compute the properties of a graph such as indegree and outdegree of a vertex and perform filtering operations on vertices and edges.

Next, you will see how you can perform motif searches using GraphFrames in order to detect structural patterns in the graph. After that, you will learn how to use a domain-specific language for motif finding and run stateless and stateful queries on simple as well as complex real-world graphs.

Finally, you will explore the variety of graph algorithms supported by the GraphFrames API including Breadth-first search, Shortest Path, triangle count, connected and strongly connected components, and PageRank.

When you are finished with this course, you will have the skills and knowledge of graph algorithms in Spark needed to implement graph algorithms using the GraphFrames API provided by Spark.

More details

User Reviews

Rating

average 0

Total votes0

Focused display

Microsoft Azure

Janani Ravi

Instructor's Courses

Janani has a Masters degree from Stanford and worked for 7+ years at Google. She was one of the original engineers on Google Docs and holds 4 patents for its real-time collaborative editing framework. After spending years working in tech in the Bay Area, New York, and Singapore at companies such as Microsoft, Google, and Flipkart, Janani finally decided to combine her love for technology with her passion for teaching. She is now the co-founder of Loonycorn, a content studio focused on providing high-quality content for technical skill development. Loonycorn is working on developing an engine (patent filed) to automate animations for presentations and educational content.

Pluralsight

View courses Pluralsight

Pluralsight, LLC is an American privately held online education company that offers a variety of video training courses for software developers, IT administrators, and creative professionals through its website. Founded in 2004 by Aaron Skonnard, Keith Brown, Fritz Onion, and Bill Williams, the company has its headquarters in Farmington, Utah. As of July 2018, it uses more than 1,400 subject-matter experts as authors, and offers more than 7,000 courses in its catalog. Since first moving its courses online in 2007, the company has expanded, developing a full enterprise platform, and adding skills assessment modules.