Pandas CSV Errors and Fixes: A Practical Troubleshooting Guide

When pd.read_csv() crashes, the error message can look intimidating. But CSV is not a formal binary format — it is plain text governed by conventions. Every error message is simply Pandas telling you that the structure it discovered on disk violates one of its parsing assumptions.

The fastest way to resolve CSV errors is not to guess parameters randomly. It is to inspect the raw file lines, diagnose the exact mismatch, and apply the surgical argument that solves it. This guide covers the most common Pandas CSV errors, reproducible examples, and their correct fixes.

ParserError: Error tokenizing data

The classic error traceback looks like this:

ParserError: Error tokenizing data. C error: Expected 3 fields in line 4, saw 4

This happens when a data row contains more delimiters than the header row defined. Consider this sample file, customers.csv:

id,name,city
1,Alice,London
2,Acme, Inc.,New York
3,Carol,Paris

Calling pd.read_csv("customers.csv") immediately raises ParserErrorbecause the unquoted comma inside Acme, Inc. splits line 4 into four fields:2, Acme, Inc., and New York.

How to diagnose the exact offending line

Rather than guessing what broke, inspect the raw line directly with standard Python before touching pandas arguments:

# Inspect line 1 (header) and line 4 (the reported error line)
with open("customers.csv", "r", encoding="utf-8") as f:
    for line_num, line in enumerate(f, start=1):
        if line_num in (1, 4):
            print(f"Line {line_num}: {repr(line.strip())}")

# Line 1: 'id,name,city'
# Line 4: '2,Acme, Inc.,New York'

If the CSV producer is within your control, fix the generation script so that text fields with delimiters are properly wrapped in quotes: 2,"Acme, Inc.",New York. You can also paste or drop your file into ourfree online CSV validatorto instantly detect unclosed quotes, stray commas, and mismatched column counts directly in your browser.

Handling malformed rows with on_bad_lines

When you cannot regenerate a third-party CSV and must ingest whatever data is valid, useon_bad_lines. In pandas 1.3.0 and newer, on_bad_lines replaced the old deprecated error_bad_lines=False and warn_bad_lines=True parameters.

import pandas as pd

# Skip bad rows and print a warning for each one
df = pd.read_csv("customers.csv", on_bad_lines="warn")

# Skip silently without warnings
df_clean = pd.read_csv("customers.csv", on_bad_lines="skip")

You can also pass a custom callback function to log or quarantine offending rows, provided you use engine="python":

bad_rows = []

def record_bad_line(line: list[str]) -> None:
    bad_rows.append(line)
    return None  # Returning None drops the line

df = pd.read_csv(
    "customers.csv",
    engine="python",
    on_bad_lines=record_bad_line,
)

print(f"Loaded {len(df)} rows; quarantined {len(bad_rows)} invalid rows.")

UnicodeDecodeError and character encoding

When a file contains characters outside standard ASCII, pandas tries to decode them with its default encoding, utf-8. If the file was saved in another encoding, you will see:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 14: invalid continuation byte

To diagnose the encoding, inspect the first few raw bytes in binary mode:

with open("customers.csv", "rb") as f:
    print(f.read(32))
# b'\xef\xbb\xbfid,name,city\r\n1,Ren\xe9,Paris'

Common encodings and their fixes

• Byte Order Mark (BOM): If the raw bytes start with b'\xef\xbb\xbf', the file was created by Microsoft Excel as "UTF-8 CSV". Use encoding="utf-8-sig" to strip the mark cleanly so your first column is named id instead of \ufeffid.
• Windows ANSI / CP1252: Common in exports from legacy Windows applications. Use encoding="cp1252".
• Western European (Latin-1): Use encoding="latin-1" or encoding="iso-8859-1".

# Read UTF-8 files exported from Excel that carry a BOM
df = pd.read_csv("customers.csv", encoding="utf-8-sig")

# Read Windows-1252 exports
df_win = pd.read_csv("windows_export.csv", encoding="cp1252")

Avoid blindly using errors="ignore": Setting encoding_errors="ignore"simply drops unparseable bytes. It removes accents from names (René becomes Ren), corrupts currency symbols, and turns text into mangled gibberish ("mojibake") without raising an alarm.

Delimiter issues (data in one column)

A CSV file is not always comma-delimited. If a file uses semicolons, tabs (TSV), or pipes, calling pd.read_csv("file.csv") will not throw an exception — instead, it will swallow every line into a single column.

Check df.shape and df.columns immediately after loading:

df = pd.read_csv("europe.csv")
print(df.shape)    # (100, 1) — expected (100, 4)!
print(df.columns)  # Index(['name;age;city;country'], dtype='object')

Fix it by passing the explicit separator:

# Semicolon delimited (standard in European Excel exports)
df = pd.read_csv("europe.csv", sep=";")

# Tab delimited (TSV)
df_tab = pd.read_csv("data.tsv", sep="\t")

# Pipe delimited
df_pipe = pd.read_csv("data.txt", sep="|")

If you do not know the delimiter in advance, you can ask pandas to sniff it usingsep=None with the Python engine:

# Automatically detect delimiter using Python's csv.Sniffer
df = pd.read_csv("unknown.csv", sep=None, engine="python")

EmptyDataError: No columns to parse

When a file has 0 bytes or contains only whitespace/comments, pandas raises:

pandas.errors.EmptyDataError: No columns to parse from file

This happens often in batch data pipelines where an upstream process creates empty placeholder files before queries complete. To protect your pipeline, check file size defensively:

import os
from pathlib import Path
import pandas as pd

def safe_read_csv(file_path: str | Path, **kwargs) -> pd.DataFrame:
    path = Path(file_path)
    if not path.exists():
        raise FileNotFoundError(f"File does not exist: {path}")
    if path.stat().st_size == 0:
        print(f"Warning: {path} is 0 bytes. Returning empty DataFrame.")
        return pd.DataFrame()
    return pd.read_csv(path, **kwargs)

df = safe_read_csv("daily_batch.csv")

DtypeWarning: Mixed types in columns

On large files, you might see this warning during import:

DtypeWarning: Columns (4) have mixed types. Specify dtype option on import or set low_memory=False.

The default C parser reads files in memory chunks to infer column types. If column 4 contains integers in the first 200,000 rows and strings like "N/A" or "Unknown" later on, pandas cannot assign a uniform numeric type without converting the column to slow Python objects.

# Fix 1: Explicitly specify dtypes for ambiguous columns (recommended)
df = pd.read_csv("records.csv", dtype={"zip_code": str, "account_id": str})

# Fix 2: Read entire file into memory before type inference (high memory usage)
df = pd.read_csv("records.csv", low_memory=False)

Setting low_memory=False silences the warning by loading the entire file into memory before making type decisions, but does not fix schema issues. Specifying dtype=...explicitly is always the safer, faster practice.

If your dataset is too large to load in memory at once, use chunksize with an explicit dtype dictionary as detailed in our guide on reading large CSV files with pandas chunksize.

FileNotFoundError and save errors

Errors also happen when saving DataFrames back to disk with df.to_csv():

• FileNotFoundError: [Errno 2] No such file or directory: pandas will create the CSV file, but it will never create missing folders. Use pathlib.Path("folder/file.csv").parent.mkdir(parents=True, exist_ok=True)before saving. See the complete save DataFrame to CSV guide.
• PermissionError: [Errno 13] Permission denied: The file is currently open in Microsoft Excel or another process that holds an exclusive write lock. Close the file in Excel and retry.
• Unwanted Unnamed: 0 columns: When you save with default settings, pandas writes the row index to the file. On read, that index becomes an extra column. Always save with index=Falseunless you specifically need the row numbers. See our guide on pandas to CSV without index.
• Repeated headers when appending: If you append rows with mode="a", remember to passheader=not path.exists() so the column titles are not duplicated. See theappending to CSV guide.

For interactive formatting tests and instant preview generation, test your snippets in theonline Pandas to CSV converter.