Data & Business Intelligencemaster · ~240 min

Past the Row Limit — a Local Columnar Warehouse for the Questions Your Spreadsheet Can't Answer

Why can you open January's sales export in seconds but not answer a question that spans the last two years?

Because a spreadsheet is a thing you open, and every export you have ever received is a separate thing you would have to open. Each file is small enough to be harmless on its own, which is exactly why nobody ever fixes the problem — and so the two-year question, the one that would actually change a decision, gets answered by pulling one quarter, eyeballing a trend, and calling it a read on the business. This capstone builds the missing layer between a spreadsheet that has run out of rows and a cloud warehouse you do not need: an immutable landing zone that keeps every export forever, a partitioned columnar warehouse compiled from it that you can destroy and rebuild in under a minute, a serving layer of named business definitions, and the operational discipline — reconciliation, idempotent loads, integrity sweeps, and a written handoff — that makes the number on the screen something you are willing to defend in a meeting.

🤝 Want this built for you? Start a request — our AI scopes it, estimates a price range, and matches you with a vetted AI Advisor. →

Two ways to open it

Just this build

$10 one-time

Unlocks this full walkthrough on your account — no subscription, and no account needed until you check out.

Best value Over 160 projects + the full course

$99 / year

Over 160 project walkthroughs, the 6-week course, and your certificate.

🔒 Secure checkout by Stripe · Already a member? Sign in

What you'll do

A local analytical warehouse you own end to end, with no cloud account and no monthly bill: an append-only landing zone whose original export files are never edited, fingerprinted with SHA-256 so a re-run can refuse work it has already done; a two-phase typed ingest that pins an explicit schema instead of letting DuckDB infer one, quarantines unparseable rows into an inspectable rejects table, and adds the not-null assertions that the rejects table provably does not give you; money stored as DECIMAL rather than DOUBLE, with the float failure demonstrated at your own data's scale rather than asserted; a partitioned Parquet warehouse compiled from the landing zone that turned 371 MB of CSV into 29 MB on disk and answers a month-scoped aggregate by opening 1 file out of 30 instead of all 30, proven with EXPLAIN ANALYZE rather than claimed; a serving layer of versioned views that preserves that pruning; a reconciliation step that ties the warehouse back to the raw exports row-for-row and cent-for-cent before anyone is allowed to publish a figure; a safe partition-replace routine built because the obvious options are traps — APPEND silently duplicates every row on a re-run and OVERWRITE deletes the entire warehouse rather than the partition you named, both verified; a memory and threads configuration with the exact out-of-memory error and its fix; an integrity sweep that names the specific corrupt file; a rebuild-from-originals drill you have actually run; a read-only serving path; and a written runbook naming who runs this, on what cadence, and what to do when a month arrives malformed.