Pandas - lesson 3 | HackIndia Skills

Pandas - lesson 3 | HackIndia Skills | HackIndia

Example

Load a CSV

read_csv() reads a CSV path and returns a DataFrame. pandas detects common missing-value markers by default, so blanks often become missing values automatically.

Do not clean inside the raw file. Load first, then clean in code in later lessons.

import pandas as pd

orders = pd.read_csv("data/raw/orders.csv")

print(orders.head())
print(orders.shape)
print(orders.columns)

Example

Load normal JSON

JSON can be shaped in different ways. A common beginner-friendly shape is a list of records, where each record becomes one row.

If AI generates sample JSON for you, check that it is valid JSON, uses double quotes, and has the same keys in most records.

[
  {"customer_id": 1, "name": "Asha", "city": "Pune"},
  {"customer_id": 2, "name": "Rahul", "city": "Jaipur"}
]

import pandas as pd

customers = pd.read_json("data/raw/customers.json")

print(customers.head())
print(customers.shape)
print(customers.columns)

Example

Load JSON lines

Some APIs and data exports store one JSON object per line. This is called line-delimited JSON or JSON Lines.

For this format, pass lines=True. If you also use chunksize, pandas returns an iterator instead of loading the whole file at once.

{"order_id": 101, "amount": 249, "status": "paid"}
{"order_id": 102, "amount": 99, "status": "failed"}

import pandas as pd

orders_log = pd.read_json("data/raw/orders_log.jsonl", lines=True)

for chunk in pd.read_json("data/raw/orders_log.jsonl", lines=True, chunksize=1000):
    print(chunk.shape)

Tip

Keep loading boring

Create one small loading script before you start cleaning. It should load each file and print head, shape, and columns.

This gives reviewers confidence that your project runs. It also helps you catch wrong paths before you waste time debugging cleaning code.

import pandas as pd

orders = pd.read_csv("data/raw/orders.csv")
customers = pd.read_json("data/raw/customers.json")

print("orders", orders.shape, list(orders.columns))
print(orders.head())

print("customers", customers.shape, list(customers.columns))
print(customers.head())

Load CSV and JSON Files

One loading idea

Pick raw files

Load a CSV

Load normal JSON

Load JSON lines

Common loading mistakes

Keep loading boring

Do this now

Quick check