# Enterprise RAG Hygiene & Verification

Category: Retrieval & Pipelines
Stack: Mosaic AI Vector Search, LlamaParse, Pydantic

## Overview
Eliminating tabular data hallucinations in financial and regulatory documents through Markdown ingestion and deterministic regex assertions.

## The Problem
Flattening tabular data into raw text strings causes language models to misassociate headers and hallucinate metrics when retrieving complex financial or regulatory documents.

## Production Rule
Extract tables directly as Markdown tables or structured JSON. Enforce an automated regex-based verification step between context retrieval and final generation to flag unsupported numbers.

## Code / Specification
File: `verify_faithfulness.py`

```python 3.11
import re
from typing import Set

def extract_numbers(text: str) -> Set[str]:
    cleaned = re.sub(r"[\$,]", "", text)
    return set(re.findall(r"\b\d+(?:\.\d+)?\b", cleaned))

def verify_response_faithfulness(generated_answer: str, context_chunks: list[str]) -> bool:
    source_text = " ".join(context_chunks)
    source_numbers = extract_numbers(source_text)
    answer_numbers = extract_numbers(generated_answer)
    unsupported = answer_numbers - source_numbers
    return len(unsupported) == 0
```