Bad Data Will Kill More AI Projects Than Bad Models

Personal content — not employer-related. This article is published in a personal capacity and shares thoughts and experiences based on public industry patterns — not a description of employer or client systems, and not professional instructions. It does not reflect the views, systems, projects, or policies of any current or past employer, client, or financial institution. See the full site disclaimer.

By mid-2024 almost every team had a GenAI demo. The ones that stalled when they touched real data did not fail on prompt engineering. They failed on stale tables, fuzzy definitions, and pipelines that had been broken for weeks without anyone noticing.

Context: After a year of ChatGPT-era experiments, many enterprises hit the same wall. Prototypes used curated samples. Production used the warehouse. The gap was almost always data quality and operational truth - not which model you picked.

This is my read on why the model gets blamed, and what had to exist before GenAI or classical ML work survived contact with real data.

Why the model gets blamed

Models compress patterns in the inputs. They do not know that customer_status changed meaning in March, that yesterday’s pipeline failed, that marketing and finance use different revenue definitions, or that 3% of rows are duplicates from a replay bug.

A bad model is consistently wrong. Bad data is confidently wrong. That is worse for trust.

Failures I kept seeing

Data quality path before AI consumes data

Freshness lies. Dashboard says updated hourly. Metadata says six hours ago. Retrieval or scoring treats it as current.

What helped: freshness targets per dataset, monitored separately from “job succeeded”. Put an as_of timestamp into whatever context the model or copilot sees.

-- example freshness SLI view (lab / notes style)
CREATE OR REPLACE VIEW ops.dataset_freshness AS
SELECT
  'curated.orders' AS dataset,
  TIMESTAMP_DIFF(CURRENT_TIMESTAMP(), MAX(_loaded_at), MINUTE) AS lag_minutes
FROM curated.orders;

Reconciliation gaps. Training looks fine. Serving does not. Or product analytics disagree with finance.

What helped: automated checks against an authoritative source. Block downstream AI jobs when variance crosses a threshold.

Schema drift. New enum, nullable column, JSON shape change - feature logic breaks quietly.

What helped: contract tests at ingest, quarantine path, alert owners before consumers break.

Ownership vacuum. Pipeline green, data wrong. Nobody owns the definition.

What helped: named business and technical owners on datasets that feed AI or critical reporting.

Scope and consent. Training or RAG pulling data that should not have been in scope.

What helped: access tags and filters in the infrastructure - not only “please be careful” in the prompt. By late 2024 RAG was already common in enterprise pilots; this mattered.

A minimum bar that actually helped

Control	Example
Freshness	`orders_curated` lag ≤ 15 min p95
Completeness	< 0.1% null on business keys
Uniqueness	`order_id` unique per partition
Referential integrity	every `product_id` in catalog
Reconciliation	daily revenue ±0.5% vs finance
Lineage	source -> transform -> feature or index

Keep evaluation sets versioned separately from production feeds so you notice drift before users do.

Contracts beat tribal knowledge

A short published agreement - schema, freshness, quality rules, owners, change policy - stopped AI and BI teams from rebuilding trust from scratch every quarter.

Prompt text like “only use recent data” does nothing if the platform cannot define recent or verified.

Closing

In 2024 the edge was not who had a model. It was who could trust inputs at scale. Freshness, definitions, reconciliation, and ownership had to move before prompt tuning or model swaps.