Companies Home Search Profile

Building Automated Data Extraction Pipelines with Python

Focused View

Tech Academy

3:25:07

182 View
  • 1. Introduction.mp4
    00:20
  • 2. Understanding the importance of automated data extraction.html
  • 3. Identifying use cases for automated data extraction.html
  • 4. Web Scraping Overview.html
  • 5. Introduction to Python libraries for data extraction.html
  • 1. Installing Python on Windows.mp4
    03:38
  • 2. Installing Python on Mac OS.mp4
    05:28
  • 3. Updating Pip.mp4
    01:43
  • 4. Create and activate a virtual environment.mp4
    04:19
  • 5. Install Scrapy.mp4
    06:11
  • 6. Install Beautiful Soup.mp4
    05:49
  • 7. Note on Text Editors.html
  • 8. Installing Visual Studio Code Text Editor.mp4
    06:00
  • 9. Best practices for data extraction pipelines.html
  • 1. What we will extract.mp4
    04:04
  • 2. Writing Python script for basic data extraction - Part 1.mp4
    06:35
  • 3. Writing Python script for basic data extraction -Part 2.mp4
    06:35
  • 4. Prototyping the script - Part 1.mp4
    06:23
  • 5. Prototyping the script - Part 2.mp4
    04:06
  • 6. Prototyping the script - Part 3.mp4
    07:02
  • 7. Prototyping the script - Part 4.mp4
    06:36
  • 8. Prototyping the script - Part 5.mp4
    11:37
  • 9. Extracting data with the script.mp4
    06:26
  • 1. Creating a Scrapy project.mp4
    03:44
  • 2. Components of a scrapy project.mp4
    08:27
  • 3. Scrapy architecture.mp4
    06:13
  • 4. Creating a spider Part 1.mp4
    05:43
  • 5. Creating a spider Part 2.mp4
    10:07
  • 6. Extracting data with scrapy shell Part 1.mp4
    05:07
  • 7. Extracting data with scrapy shell Part 2.mp4
    11:59
  • 8. Running the spider to extract data.mp4
    06:47
  • 1. Create and activate a virtual environment.html
  • 2. Install Python Packages.mp4
    01:54
  • 3. Creating a Python file.mp4
    03:30
  • 4. Creating Variables.mp4
    07:09
  • 5. Enabling Gmail Security.mp4
    01:34
  • 6. Creating Functions Part 1.mp4
    11:20
  • 7. Creating Functions Part 2.mp4
    13:52
  • 8. Creating Functions Part 3.mp4
    10:27
  • 9. Extracting data with the Python Script.mp4
    04:22
  • Description


    Data Extraction and Scraping Techniques Using Python

    What You'll Learn?


    • How to automate data extraction pipelines using Python
    • How to scrape data from e-commerce websites using Python
    • How to use Scrapy to build scalable and efficient web scrapers
    • How to use Requests to make HTTP requests to web servers
    • Scrape data with BeautifuSoup
    • Scrape data with Scrapy
    • Scrape e-commerce Data with Python
    • How to use Beautiful Soup to parse HTML
    • How to install and set up Python libraries for data extraction
    • How to use Python libraries for data extraction
    • Common use cases for automated data extraction
    • The importance of automated data extraction
    • Python 3.x installed on the computer

    Who is this for?


  • Data analysts and data scientists who want to expand their skills and automate the data collection process.
  • Business analysts who need to extract data from websites to inform business decisions.
  • Researchers who need to extract data from a variety of sources for their research projects.
  • Web developers who want to build web scrapers for their projects.
  • Digital marketers who want to extract data from social media platforms and other online sources.
  • Students who want to learn practical skills in data extraction and scraping.
  • Professionals who want to switch careers to a data-related field.
  • Anyone who wants to learn how to automate the process of collecting data from the web.
  • What You Need to Know?


  • A computer with internet access and the ability to run Python
  • Basic knowledge of Python programming language
  • Basic knowledge of HTML, CSS, and JavaScript
  • Text editor or integrated development environment (IDE) for Python coding
  • Comfortable using the command-line interface (CLI)
  • More details


    Description

    In the age of Big Data, the ability to effectively extract, process, and analyze data from various sources has become increasingly important. This  course will guide you through the process of building automated data extraction pipelines using Python, a powerful and versatile programming language. You will learn how to harness Python's vast ecosystem of libraries and tools to efficiently extract valuable information from websites, APIs, and other data sources, transforming raw data into actionable insights.


    This  course is designed for data enthusiasts, analysts, engineers, and anyone interested in learning how to build data extraction pipelines using Python. By the end of this course, you will have developed a solid understanding of the fundamental concepts, tools, and best practices involved in building automated data extraction pipelines. You will also gain hands-on experience by working on a real-world project, applying the skills and knowledge acquired throughout the course. We will be using two popular Python Libraries called BeautifulSoup and Scrapy  f to build our  data pipelines.


    Beautiful Soup is a popular Python library for web scraping that helps extract data from HTML and XML documents. It creates parse trees from the page source, allowing you to navigate and search the document's structure easily.

    Beautiful Soup plays a crucial role in data extraction by simplifying the process of web scraping, offering robust parsing and efficient navigation capabilities, and providing compatibility with other popular Python libraries. Its ease of use, adaptability, and active community make it an indispensable tool for extracting valuable data from websites.


    Scrapy is an open-source web crawling framework for Python, specifically designed for data extraction from websites. It provides a powerful, flexible, and high-performance solution to create and manage web spiders (also known as crawlers or bots) for various data extraction tasks.

    Scrapy plays an essential role in data extraction by offering a comprehensive, high-performance, and flexible web scraping framework. Its robust crawling capabilities, built-in data extraction tools, customizability, and extensibility make it a powerful choice for data extraction tasks ranging from simple one-time extractions to complex, large-scale web scraping projects. Scrapy's active community and extensive documentation further contribute to its importance in the field of data extraction.


    Who this course is for:

    • Data analysts and data scientists who want to expand their skills and automate the data collection process.
    • Business analysts who need to extract data from websites to inform business decisions.
    • Researchers who need to extract data from a variety of sources for their research projects.
    • Web developers who want to build web scrapers for their projects.
    • Digital marketers who want to extract data from social media platforms and other online sources.
    • Students who want to learn practical skills in data extraction and scraping.
    • Professionals who want to switch careers to a data-related field.
    • Anyone who wants to learn how to automate the process of collecting data from the web.

    User Reviews
    Rating
    0
    0
    0
    0
    0
    average 0
    Total votes0
    Focused display
    Tech Academy
    Tech Academy
    Instructor's Courses
    Tech Academy is a UK based e-learning provider that offers a range of high quality elearning solutions that teach real life  technical skills that are essential and relevant in today's commercial environment. Our instructors have a wealth of experience in their respective courses and currently provide consultancy services to fortune 100 companies. Our courses are presented in HD and are clear and concise to enable retention and provide intuitive learning experience for absolute beginners.
    Students take courses primarily to improve job-related skills.Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company.
    • language english
    • Training sessions 33
    • duration 3:25:07
    • Release Date 2023/06/11