Chunking

Operator uses chunking_strategy=by_title so chunks respect section structure, but the source was partitioned with fast and has few real Title elements.

by_title starts new chunks at Title/section boundaries — it depends on accurate Title detection, which comes from good partitioning (often hi_res). Verify Titles exist before relying on by_title; otherwise chunks degrade toward size-only splits.

RAG recall is poor at chunk boundaries; the agent sets overlap to a very large value to compensate, ballooning the index.

Set overlap to a modest fraction of max_characters to preserve boundary context without exploding index size or duplicating content across many chunks. Tune against retrieval metrics rather than maximizing overlap blindly.

Agent sets max_characters=512 expecting it as a soft target, then is surprised when a large Table element is split mid-table.

max_characters is a hard cap on chunk size; new_after_n_chars is the soft target. Set max_characters with table-aware expectations (large tables may be split or isolated). Tune new_after_n_chars below max_characters to avoid frequent hard splits.