Welcome to Module 2 of our Pandas tutorial series! In this module, we’ll delve into basic data manipulation techniques with Pandas. You’ll learn how to load data from various sources, explore and inspect datasets, and perform indexing and selection operations to extract specific data of interest.

1. Loading Data

Reading Data from Different Sources

Pandas provides versatile tools for reading data from various sources, including CSV, Excel, SQL databases, and more. You can use functions like pd.read_csv(), pd.read_excel(), and pd.read_sql() to load data into Pandas DataFrames.

import pandas as pd

# Reading data from a CSV file
df = pd.read_csv('data.csv')

# Reading data from an Excel file
df = pd.read_excel('data.xlsx')

# Reading data from a SQL database
import sqlite3
conn = sqlite3.connect('mydatabase.db')
df = pd.read_sql('SELECT * FROM mytable', conn)


import pandas as pd

df = pd.read_csv('https://raw.githubusercontent.com/pandas-dev/pandas/main/doc/data/titanic.csv')


Setting the Index

The index is essential for data retrieval and manipulation. You can set a specific column as the index when reading data or using the .set_index() method on an existing DataFrame.

# Setting the 'ID' column as the index
df = df.set_index('ID')

# Alternatively, set the index during data reading
df = pd.read_csv('data.csv', index_col='ID')

2. Data Exploration and Inspection

Head, Tail, and Sample Data

To quickly inspect your data, use methods like .head(), .tail(), and .sample(). These methods display the first few, last few, or random rows of your DataFrame.

# Display the first 5 rows

# Display the last 5 rows

# Display a random sample of rows

Info and Describe Methods

The .info() method provides essential information about your DataFrame, such as the data types and the presence of missing values. The .describe() method calculates summary statistics for numeric columns.

# Display DataFrame info

# Calculate summary statistics

Shape and Dimensions

You can determine the shape (number of rows and columns) of your DataFrame using the .shape attribute and the number of rows or columns using len().

# Get the shape of the DataFrame

# Get the number of rows

# Get the number of columns

3. Indexing and Selection

Selecting Columns and Rows

Pandas provides multiple ways to select specific columns and rows. You can use square brackets [], .loc[], and .iloc[] for different selection purposes.

# Select a single column by name
column = df['Column_Name']

# Select multiple columns by names
columns = df[['Column1', 'Column2']]

# Select rows by label using .loc[]
selected_rows = df.loc['Label']

# Select rows by position using .iloc[]
selected_rows = df.iloc[0]

Boolean Indexing

You can filter rows based on conditions using Boolean indexing. This allows you to extract rows that meet specific criteria.

# Boolean indexing to filter rows
filtered_data = df[df['Column'] > 50]

# Combine multiple conditions using & (and) or | (or)
filtered_data = df[(df['Column1'] > 50) & (df['Column2'] < 30)]

Setting Values

You can set values for specific cells, rows, or columns using assignment.

# Set a value for a specific cell
df.at['Label', 'Column'] = new_value

# Set values for an entire column
df['Column'] = new_values

# Set values for multiple columns
df[['Column1', 'Column2']] = new_values

Congratulations! You’ve completed Module 2 of our Pandas tutorial. In this module, you’ve learned essential techniques for basic data manipulation with Pandas, including loading data from various sources, exploring and inspecting datasets, and performing indexing and selection operations.

In the next module, we’ll dive deeper into data cleaning and preparation with Pandas, including handling missing data and transforming data to make it suitable for analysis.

Leave a Reply

Your email address will not be published. Required fields are marked *