Pandas - lesson 4 | HackIndia Skills

Pandas - lesson 4 | HackIndia Skills | HackIndia

Example

Inspection script

Put this in a notebook or a small Python file after loading your raw data. It works well for a college placement dataset, UPI transactions, product orders, or any tabular CSV.

import pandas as pd

students = pd.read_csv("data/raw/students.csv")

print("Shape:", students.shape)
print("Columns:", students.columns.tolist())

print("\nFirst rows")
print(students.head())

print("\nLast rows")
print(students.tail())

print("\nInfo")
students.info()

print("\nNumeric summary")
print(students.describe())

print("\nMissing values")
print(students.isna().sum().sort_values(ascending=False))

print("\nDuplicate rows:", students.duplicated().sum())

Example

Quality report

Create a small report from each raw file. Commit it in your repo as reports/quality_notes.txt or print it from scripts/inspect.py.

This report is not a fancy dashboard. It is proof that you inspected the data before changing it.

# reports/quality_notes.txt
File: data/raw/students.csv
Shape: 500 rows, 8 columns
Possible key column: student_id
Missing values: email has 23, graduation_year has 4
Duplicate rows: 2
Type issues: fees is str, expected numeric rupees
Range issues: attendance_percent has values above 100
Notes: clean column names before joining with placements.csv

Inspect Shape Types and Quality

First inspect, then clean

What to inspect

Basic checks

Inspection script

Use isna

Trusting dtypes too much

Quality report

Do this now

Quick check