Spark Programming in Python for Beginners with Apache Spark 3

Focused View

Scholar Nest

6:35:30

173 View

01.01-big_data_history_and_primer.mp4

05:52

01.02-understanding_the_data_lake_landscape.mp4

06:44

01.03-what_is_apache_spark-an_introduction_and_overview.mp4

08:50

02.01-spark_development_environments.mp4

02:52

02.02-mac_users-apache_spark_in_local_mode_command_line_repl.mp4

12:05

02.03-windows_users-apache_spark_in_local_mode_command_line_repl.mp4

05:49

02.04-mac_users-apache_spark_in_the_ide-pycharm.mp4

07:55

02.05-windows_users-apache_spark_in_the_ide-pycharm.mp4

08:36

02.06-apache_spark_in_cloud-databricks_community_and_notebooks.mp4

04:34

02.07-apache_spark_in_anaconda-jupyter_notebook.mp4

04:33

03.01-execution_methods-how_to_run_spark_programs.mp4

05:01

03.02-spark_distributed_processing_model-how_your_program_runs.mp4

03:11

03.03-spark_execution_modes_and_cluster_managers.mp4

04:54

03.04-summarizing_spark_execution_models-when_to_use_what.mp4

02:24

03.05-working_with_pyspark_shell-demo.mp4

04:31

03.06-installing_multi-node_spark_cluster-demo.mp4

05:37

03.07-working_with_notebooks_in_cluster-demo.mp4

06:59

03.08-working_with_spark_submit-demo.mp4

02:55

03.09-section_summary.mp4

01:42

04.01-creating_spark_project_build_configuration.mp4

06:10

04.02-configuring_spark_project_application_logs.mp4

10:51

04.03-creating_spark_session.mp4

08:26

04.04-configuring_spark_session.mp4

09:12

04.05-data_frame_introduction.mp4

07:44

04.06-data_frame_partitions_and_executors.mp4

05:25

04.07-spark_transformations_and_actions.mp4

11:02

04.08-spark_jobs_stages_and_task.mp4

08:34

04.09-understanding_your_execution_plan.mp4

09:34

04.10-unit_testing_spark_application.mp4

05:02

04.11-rounding_off_summary.mp4

05:27

05.01-introduction_to_spark_apis.mp4

05:12

05.02-introduction_to_spark_rdd_api.mp4

13:14

05.03-working_with_spark_sql.mp4

02:37

05.04-spark_sql_engine_and_catalyst_optimizer.mp4

02:54

05.05-section_summary.mp4

01:17

06.01-spark_data_sources_and_sinks.mp4

06:44

06.02-spark_dataframereader_api.mp4

05:01

06.03-reading_csv_json_and_parquet_files.mp4

07:59

06.04-creating_spark_dataframe_schema.mp4

06:07

06.05-spark_dataframewriter_api.mp4

06:10

06.06-writing_your_data_and_managing_layout.mp4

12:51

06.07-spark_databases_and_tables.mp4

05:34

06.08-working_with_spark_sql_tables.mp4

08:43

07.01-introduction_to_data_transformation.mp4

02:44

07.02-working_with_dataframe_rows.mp4

05:02

07.03-dataframe_rows_and_unit_testing.mp4

04:03

07.04-dataframe_rows_and_unstructured_data.mp4

06:08

07.05-working_with_dataframe_columns.mp4

10:33

07.06-creating_and_using_udf.mp4

10:02

07.07-misc_transformations.mp4

15:34

08.01-aggregating_dataframes.mp4

08:58

08.02-grouping_aggregations.mp4

04:25

08.03-windowing_aggregations.mp4

05:28

09.01-dataframe_joins_and_column_name_ambiguity.mp4

07:40

09.02-outer_joins_in_dataframe.mp4

07:25

09.03-internals_of_spark_join_and_shuffle.mp4

08:46

09.04-optimizing_your_joins.mp4

12:17

09.05-implementing_bucket_joins.mp4

08:57

10.01-final_word.mp4

00:34

9781803246161_Code.zip

Description

If you are looking to expand your knowledge in data engineering or want to level up your portfolio by adding Spark programming to your skillset, then you are in the right place. This course will help you understand Spark programming and apply that knowledge to build data engineering solutions. This course is example-driven and follows a working session-like approach. We will be taking a live coding approach and explaining all the concepts needed along the way.

In this course, we will start with a quick introduction to Apache Spark, then set up our environment by installing and using Apache Spark. Next, we will learn about Spark execution model and architecture, and about Spark programming model and developer experience. Next, we will cover Spark structured API foundation and then move towards Spark data sources and sinks.

Then we will cover Spark Dataframe and dataset transformations. We will also cover aggregations in Apache Spark and finally, we will cover Spark Dataframe joins.

By the end of this course, you will be able to build data engineering solutions using Spark structured API in Python.

All the resources for the course are available at https://github.com/PacktPublishing/Spark-Programming-in-Python-for-Beginners-with-Apache-Spark-3

More details

User Reviews

Rating

average 0

Total votes0

Focused display

Python

Apache Spark

Scholar Nest

Instructor's Courses

ScholarNest is a small team of people passionate about helping others learn and grow in their careers by bridging the gap between their existing and required skills. Together, they have over 40+ years of experience in IT as a developer, architect, consultant, trainer, and mentor. They have worked with international software services organizations on various data-centric and Big Data projects. It is a team of firm believers in lifelong continuous learning and skill development. To popularize the importance of continuous learning, they started publishing free training videos on their YouTube channel. They conceptualized the notion of continuous learning, creating a journal of our learning under the Learning Journal banner.

PacktPub

View courses PacktPub

Packt is a publishing company founded in 2003 headquartered in Birmingham, UK, with offices in Mumbai, India. Packt primarily publishes print and electronic books and videos relating to information technology, including programming, web design, data analysis and hardware.