Build a reproducible Python data analysis

8 GB RAM for modest CSV files; 16 GB or more for multi-gigabyte inputs. No GPU required.

Component	Role	Price	Link
Polars 1.43.2	typed lazy data cleaning and transformation	Free / 免费MIT open-source Python packagechecked 2026-08-02	https://github.com/pola-rs/polars/releases/tag/py-1.43.2
DuckDB 1.5.5	SQL validation, joins and portable Parquet output	Free / 免费MIT open-source databasechecked 2026-08-02	https://github.com/duckdb/duckdb/releases/tag/v1.5.5
uv 0.12.1	locked Python environment and command execution	Free / 免费open-source package and project managerchecked 2026-08-02	https://github.com/astral-sh/uv/releases/tag/0.12.1

Create a locked analysis project

uv init analysis
cd analysis
uv add 'polars==1.43.2' 'duckdb==1.5.5' pyarrow jupyterlab
mkdir -p data/raw data/derived results
sha256sum data/raw/* > data/raw/SHA256SUMS
uv lock
uv run python src/prepare.py
uv run python src/analyze.py

Analysis invariants

raw_data_immutable: true
primary_key: replace_with_column
required_columns: []
allowed_missing_fraction: 0.0
expected_row_count:
  min: 1
  max: null
unit_columns: {}
date_timezone: UTC
outputs:
  - data/derived/analysis.parquet
  - results/model-summary.json
  - results/figures/
rule: A failed invariant stops the pipeline before modeling.

Known pitfalls

• Never edit files under data/raw in place.
• Polars expression semantics can change across major versions; keep uv.lock.
• CSV type inference is not a data contract; provide schemas for important columns.
• Record missing-data and exclusion decisions in code, not only notebooks.

Browser only. Choose a runtime sized for the data; do not upload restricted data without approval.

Component	Role	Price	Link
Google Colab Pro hosted notebook service current 2026-08-02	managed notebook compute and optional accelerators	Subscription (rate not published) / 订阅制（费率未公开）unverified regional subscription and compute-unit price; confirm in the official purchase flowchecked 2026-08-02	https://colab.research.google.com/signup
DuckDB 1.5.5	portable local SQL layer inside the notebook	Free / 免费software is free; notebook compute is separatechecked 2026-08-02	https://duckdb.org/

Notebook bootstrap cell

%pip install -q 'duckdb==1.5.5' 'polars==1.43.2' pyarrow
import hashlib, pathlib, platform
import duckdb, polars as pl
print({"python": platform.python_version(), "duckdb": duckdb.__version__, "polars": pl.__version__})
for path in sorted(pathlib.Path('/content/data/raw').glob('*')):
    if path.is_file():
        print(path.name, hashlib.sha256(path.read_bytes()).hexdigest())

Known pitfalls

• Hosted runtimes are ephemeral; export code, lock information and results before disconnect.
• The live price varies by region and resource use.
• Notebook execution order can hide stale state; restart and run all before release.
• Mounting personal cloud drives can broaden data access.