Skip to main content

Course

Streamlined Data Ingestion with pandas

IntermediateSkill Level

4.8+

Updated 11/2024

Learn to acquire data from common file formats and systems such as CSV files, spreadsheets, JSON, SQL databases, and APIs.

Start Course for Free

PythonData Preparation

4 hr

16 videos

53 Exercises

4,500 XP

62,262

Statement of Accomplishment

Create Your Free Account

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Loved by learners at thousands of companies

Training a Team?

Try for Business

Course Description

Before you can analyze data, you first have to acquire it. This course teaches you how to build pipelines to import data kept in common storage formats. You’ll use pandas, a major Python library for analytics, to get data from a variety of sources, from spreadsheets of survey responses, to a database of public service requests, to an API for a popular review site. Along the way, you’ll learn how to fine-tune imports to get only what you need and to address issues like incorrect data types. Finally, you’ll assemble a custom dataset from a mix of sources.

Prerequisites

Intermediate Python Intermediate SQL

1

Importing Data from Flat Files

Practice using pandas to get just the data you want from flat files, learn how to wrangle data types and handle errors, and look into some U.S. tax data along the way.

Introduction to flat files

Get data from CSVs

Get data from other flat files

Modifying flat file imports

Import a subset of columns

Import a file in chunks

Handling errors and missing data

Specify data types

Set custom NA values

Skip bad data

2

Importing Data From Excel Files

Automate data imports from that staple of office life, Excel files. Import part or all of a workbook and ensure boolean and datetime data are properly loaded, all while learning about how other people are learning to code.

Introduction to spreadsheets

Get data from a spreadsheet

Load a portion of a spreadsheet

Getting data from multiple worksheets

Select a single sheet

Select multiple sheets

Work with multiple spreadsheets

Modifying imports: true/false data

Set Boolean columns

Set custom true/false values

Modifying imports: parsing dates

Parse simple dates

Get datetimes from multiple columns

Parse non-standard date formats

3

Importing Data from Databases

Combine pandas with the powers of SQL to find out just how many problems New Yorkers have with their housing. This chapter features introductory SQL topics like WHERE clauses, aggregate functions, and basic joins.

Introduction to databases

Connect to a database

Load entire tables

Refining imports with SQL queries

Selecting columns with SQL

Selecting rows

Filtering on multiple conditions

More complex SQL queries

Getting distinct values

Counting in groups

Working with aggregate functions

Loading multiple tables with joins

Joining tables

Joining and filtering

Joining, filtering, and aggregating

4

Importing JSON Data and Working with APIs

Learn how to work with JSON data and web APIs by exploring a public dataset and getting cafe recommendations from Yelp. End by learning some techniques to combine datasets once they have been loaded into data frames.

Introduction to JSON

Load JSON data

Work with JSON orientations

Introduction to APIs

Get data from an API

Set API parameters

Set request headers

Working with nested JSONs

Flatten nested JSONs

Handle deeply nested data

Combining multiple datasets

Concatenate dataframes

Merge dataframes

Streamlined Data Ingestion with pandas

Course
Complete

Earn Statement of Accomplishment

Add this credential to your LinkedIn profile, resume, or CV
Share it on social media and in your performance reviewEnroll Now

Don’t just take our word for it

*4.8

from 1,669 reviews

83%

15%

1%

0%

0%

Sort by

Adama

8 hours ago

Moch Riyadi

yesterday

Mustafa

yesterday

Victorien

2 days ago

Chirangeeve

3 days ago

Po-Tsung

3 days ago

Adama

Moch Riyadi

Mustafa

FAQs

What data sources does this course teach me to import from?

You will import data from flat files, Excel workbooks, databases using SQL, and APIs, all using pandas in Python.

What prior knowledge do I need?

You should know intermediate Python and basic SQL. Experience with pandas is helpful but not strictly required, as you learn pandas import methods throughout.

What real-world data is used in the exercises?

You work with U.S. tax data from flat files, coding survey data from Excel files, public service request data from databases, and review site data from APIs.

Will I learn to handle data type issues during import?

Yes. The course teaches you to fine-tune imports by addressing incorrect data types, handling errors, and loading only the data you need.

How is the course organized?

It has 4 chapters covering flat files, Excel files, databases, and APIs, with 53 exercises total. Most learners finish in about 3.5 hours.

Join over 19 million learners and start Streamlined Data Ingestion with pandas today!

Create Your Free Account

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Grow your data skills with DataCamp for Mobile

Make progress on the go with our mobile courses and daily 5-minute coding challenges.