LLM-Powered Data Extraction: Ditching Regex for Structured Prompts

Regex has a well-earned reputation: powerful for simple patterns, nightmarish for anything complex. You write ten lines of

^(?:[a-z0-9!#$%&'*+/=?^_

Large language models offer a different path. Instead of teaching machines a pattern language, you describe what you want in plain English. The model learns context, handles variations, and adapts without regex recompilation. At LavaPi, we've seen teams cut data extraction code by 60–70% while improving accuracy on edge cases.

Here's why this matters and how to implement it.

The Problem with Regex at Scale

Regex excels at fixed, narrow problems: validating email addresses, parsing timestamps, extracting phone numbers. But the moment your data becomes semistructured—invoices, contracts, support tickets, API responses—regex becomes a liability.

Consider extracting customer details from an unstructured email:

textCopy
Hi, I'm John Smith, reached at john.smith@company.com or 555-0123.
My account number is ACC-2024-001A.

You'd need separate patterns for:

• Names (handling prefixes, suffixes, hyphenation)
• Email addresses (RFC 5322 is 6,000+ words)
• Phone numbers (parentheses, dashes, dots, international formats)
• Account IDs (variable length, alphanumeric, sometimes with dashes)

Each pattern is fragile. One typo breaks it. New formats require new patterns. Testing becomes tedious.

Structured Prompts: The Cleaner Alternative

LLMs work better with description than pattern matching. You define the output structure, describe the extraction task, and let the model handle variations:

pythonCopy
import json
from anthropic import Anthropic

client = Anthropic()

def extract_customer_data(text: str) -> dict:
    prompt = f"""Extract customer information from the text below.
    
Return a JSON object with these fields (omit if not found):
- name: Full customer name
- email: Email address
- phone: Phone number (normalized format)
- account_id: Account or reference number

Text:
{text}

Respond with valid JSON only, no markdown."""
    
    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=500,
        messages=[
            {"role": "user", "content": prompt}
        ]
    )
    
    return json.loads(response.content[0].text)

# Usage
text = "Hi, I'm John Smith, reached at john.smith@company.com or 555-0123. My account number is ACC-2024-001A."
result = extract_customer_data(text)
print(result)

Output:

jsonCopy
{
  "name": "John Smith",
  "email": "john.smith@company.com",
  "phone": "+1-555-0123",
  "account_id": "ACC-2024-001A"
}

Improving Reliability with Schema Validation

For production systems, enforce output structure using JSON Schema:

typescriptCopy
import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();

const extractionSchema = {
  type: 'object',
  properties: {
    name: { type: 'string' },
    email: { type: 'string', format: 'email' },
    phone: { type: 'string' },
    account_id: { type: 'string' }
  },
  required: ['name']
};

async function extractWithValidation(text: string) {
  const response = await client.messages.create({
    model: 'claude-3-5-sonnet-20241022',
    max_tokens: 500,
    messages: [
      {
        role: 'user',
        content: `Extract data matching this schema: ${JSON.stringify(extractionSchema)}\n\nText: ${text}`
      }
    ]
  });

  return JSON.parse(response.content[0].type === 'text' ? response.content[0].text : '');
}

When to Use This Approach

LLM extraction works best for:

• Semistructured data: PDFs, emails, support tickets
• Variable formats: Different templates, layouts, language styles
• Complex context: Requires understanding domain knowledge or relationships
• Edge cases: Handles typos, abbreviations, and unusual formatting

Stick with regex for:

• Simple, fixed patterns (dates in YYYY-MM-DD format)
• High-throughput systems where latency matters
• Fully structured data (already validated JSON, CSV)

The Bottom Line

Regex isn't going away, but for data extraction at scale, LLMs are more maintainable and robust. You get cleaner code, better error handling, and the flexibility to change requirements without rewriting patterns. Teams building extraction pipelines—whether for document processing, API integration, or log analysis—should seriously consider this shift.

Start with a small proof-of-concept. Extract one data type, measure accuracy, and compare it to your current approach. You'll likely find the cognitive overhead drops significantly.