Data Analysis with Python and PySpark

Author

Think big about your data! PySpark brings the powerful Spark big data processing engine to the Python ecosystem, letting you seamlessly scale up your data tasks and create lightning-fast pipelines.

In Data Analysis with Python and PySpark you will learn how to:

    Manage your data as it scales across multiple machines
    Scale up your data programs with full confidence
    Read and write data to and from a variety of sources and formats
    Deal with messy data with PySparks data manipulation functionality
    Discover new data sets and perform exploratory data analysis
    Build automated data pipelines that transform, summarize, and get insights from data
    Troubleshoot common PySpark errors
    Creating reliable long-running jobs

Data Analysis with Python and PySpark is your guide to delivering successful Python-driven data projects. Packed with relevant examples and essential techniques, this practical book teaches you to build pipelines for reporting, machine learning, and other data-centric tasks. Quick exercises in every chapter help you practice what youve learned, and rapidly start implementing PySpark into your data systems. No previous knowledge of Spark is required.

Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

About the technology
The Spark data processing engine is an amazing analytics factory: raw data comes in, insight comes out. PySpark wraps Sparks core engine with a Python-based API. It helps simplify Sparks steep learning curve and makes this powerful tool available to anyone working in the Python data ecosystem.

About the book
Data Analysis with Python and PySpark helps you solve the daily challenges of data science with PySpark. Youll learn how to scale your processing capabilities across multiple machines while ingesting data from any sourcewhether thats Hadoop clusters, cloud data storage, or local data files. Once youve covered the fundamentals, youll explore the full versatility of PySpark by building machine learning pipelines, and blending Python, pandas, and PySpark code.

What\'s inside

    Organizing your PySpark code
    Managing your data, no matter the size
    Scale up your data programs with full confidence
    Troubleshooting common data pipeline problems
    Creating reliable long-running jobs

About the reader
Written for data scientists and data engineers comfortable with Python.

About the author
As a ML director for a data-driven software company, Jonathan Rioux uses PySpark daily. He teaches the software to data scientists, engineers, and data-savvy business analysts.

Table of Contents

1 Introduction
PART 1 GET ACQUAINTED: FIRST STEPS IN PYSPARK
2 Your first data program in PySpark
3 Submitting and scaling your first PySpark program
4 Analyzing tabular data with pyspark.sql
5 Data frame gymnastics: Joining and grouping
PART 2 GET PROFICIENT: TRANSLATE YOUR IDEAS INTO CODE
6 Multidimensional data frames: Using PySpark with JSON data
7 Bilingual PySpark: Blending Python and SQL code
8 Extending PySpark with Python: RDD and UDFs
9 Big data is just a lot of small data: Using pandas UDFs
10 Your data under a different lens: Window functions
11 Faster PySpark: Understanding Sparks query planning
PART 3 GET CONFIDENT: USING MACHINE LEARNING WITH PYSPARK
12 Setting the stage: Preparing features for machine learning
13 Robust machine learning with ML Pipelines
14 Building custom ML transformers and estimators

Download pdf

ISBN-10

1617297208

ISBN-13

978-1617297205

Publisher

Manning

Price

48.49

File Type

PDF

Page No.

456

Jonathan Rioux

Discover more of the author’s books, see similar authors, read author blogs and more ...

About the Author

As a data scientist for an engineering consultancy Jonathan Rioux uses PySpark daily. He teaches the software to data scientists, engineers, and data-savvy business analysts.

Data Analysis with Python and PySpark

Python

Data Analysis

Data Mining

PySpark

Jonathan Rioux

Manning

Jonathan Rioux

Publishers opinion

Publishers opinion

About the Author

Similar Books

Other Authors' Books

Other Publishing Books

User Reviews

Rating