Chunking Strategies

We cannot send a 200-page PDF to the retriever as one piece. We cut it into chunks. Cut too small and the meaning breaks; cut too big and the search becomes vague. Let us compare the main strategies on a real FD policy and see their actual output.

01Why chunk size matters

02Fixed-length vs recursive chunking

Fixed-length cuts every N characters, even in the middle of a word. Recursive chunking tries to cut at natural boundaries first (blank lines, then lines, then spaces), and only goes smaller if needed. Here is the real output on our FD policy:

from langchain_text_splitters import RecursiveCharacterTextSplitter, MarkdownHeaderTextSplitter, CharacterTextSplitter

policy = """# Fixed Deposit Policy

## 1. Eligibility
Any resident individual above 18 years can open an FD. Minors can open an FD through a guardian.

## 2. Interest rates
Rates depend on tenure. Senior citizens get an extra 0.50% on all tenures.

## 3. Premature withdrawal
Breaking an FD early reduces the interest rate by 1%. Tax-saver FDs cannot be withdrawn before 5 years."""

fixed = CharacterTextSplitter(separator="", chunk_size=120, chunk_overlap=0)
print("FIXED:", len(fixed.split_text(policy)))
for c in fixed.split_text(policy)[:2]: print(repr(c))

rec = RecursiveCharacterTextSplitter(chunk_size=200, chunk_overlap=20)
chunks = rec.split_text(policy)
print("RECURSIVE:", len(chunks))
for c in chunks: print(repr(c))

md = MarkdownHeaderTextSplitter(headers_to_split_on=[("#", "doc"), ("##", "section")])
for d in md.split_text(policy): print(d.metadata, "|", d.page_content[:60])

See the difference: fixed-length broke "FD t" | "hrough" in the middle of a word, while recursive gave exactly one clean section per chunk. The Markdown splitter (last part) also attached the section name as metadata.

03Semantic chunking

Semantic chunking embeds each sentence and starts a new chunk where the meaning changes, that is, where the similarity between neighbouring sentences drops sharply.

04Structure-aware and parent–child chunking

• Structure-aware: split on the document’s own structure: headings, sections, list items, table boundaries. Works very well for policies, manuals and Markdown/HTML (use the elements from Unstructured, Session 32).
• Parent–child: index small child chunks for precise search, but send their larger parent section to the LLM for full context.

05Metadata design for filtering

Every chunk should carry metadata. It enables filters (“only FD policies from 2026”), citations (“p.4”) and access control (“HR documents only for HR”).

{
  "text": "Breaking an FD early reduces the interest rate by 1%.",
  "source": "FD_Policy_2026.pdf",
  "page": 4,
  "section": "3. Premature withdrawal",
  "doc_type": "policy",
  "department": "retail_banking",
  "effective_date": "2026-04-01",
  "access_level": "all_employees",
  "chunk_id": "fd2026-p4-c2"
}

06Summary

• Good chunks hold one complete idea; start at 300–800 tokens with overlap.
• Recursive chunking is the safe default; fixed-length can break words.
• Semantic chunking cuts where meaning changes.
• Structure-aware and parent–child chunking give precise search with full context.
• Rich metadata enables filters, citations and access control.

07Quick quiz

08References

• LangChain text splitters
• Pinecone: Chunking strategies