AI Drawing Explorer

The Problem

The client is a major aircraft interiors manufacturer, a Tier 1 supplier to Boeing and Airbus. They specialize in premium seating, galleys, lavatories, and fleet refurbishment. Their engineers had a painful workflow problem.

When responding to RFPs, engineers needed to answer questions like:

• "Have we built something like this before?"
• "What was the bill of materials for this specific component?"
• "Show me similar drawings to this reference image"

The answer was buried somewhere in tens of thousands of architectural diagrams scattered across SharePoint. Engineers spent hours, sometimes days, manually searching through folders, opening PDFs, and cross-referencing drawing numbers. Tedious, error-prone, and eating into time that should have been spent on actual engineering work.

The Solution

I built Drawing Explorer, a multi-modal search platform that lets engineers find exactly what they need in seconds.

The Ingestion Pipeline

The real challenge was making 15,000+ pages of complex engineering PDFs actually searchable. These weren't simple text documents. They were architectural drawings with specific notation, drawing numbers, reference codes, and visual components that mattered as much as any text.

I engineered an ingestion pipeline that:

• Processed every PDF page as both text and image

• Used multimodal LLMs to extract structured metadata: drawing numbers, drawing lists, reference codes, part specifications

• Generated dual embeddings: text embeddings for semantic search AND image embeddings for visual similarity

• Automated drawing classification: categorizing drawings by type and component

This approach was critical. Sometimes engineers know what they're looking for in words. Sometimes they have a reference image and want to find similar drawings. The system handles both.

The Search Experience

Drawing Explorer provides three ways to find what you need:

Image Search

Upload a reference image, a photo, a screenshot, a scan from another document, and the system returns visually similar drawings from the archive. This uses vector search across the image embeddings.

Semantic Search

Ask for what you want in plain English: "lavatory door hinge assemblies" or "galley electrical schematics for 787." The AI understands intent and returns relevant results, not just keyword matches.

Traditional Search & Filtering

Sometimes you know the exact drawing number or need to filter by project, date, or component type. Full text search and faceted filtering handle the precise lookup cases.

Bill of Materials Workflow

Finding drawings is only half the job. Engineers needed to compile their findings into a bill of materials for RFP responses. I built a save-to-BOM feature directly into the search interface. Engineers can bookmark relevant drawings as they search, then export their curated list.

Technical Architecture

Stack:

• React frontend with table-based search interface
• Flask backend handling search orchestration and ingestion
• Azure AI Search for hybrid search (vector + keyword)
• Azure Blob Storage for document and image storage
• Azure OpenAI for embeddings and metadata extraction

Key Design Decisions:

• Dual embedding strategy: Text and image embeddings indexed separately, enabling true multi-modal search
• Hybrid search: Combined vector similarity with traditional keyword search for best-of-both-worlds retrieval
• Structured metadata extraction: LLM-powered extraction of drawing-specific metadata (numbers, references, classifications) for precise filtering
• Progressive loading: Large result sets paginated efficiently for fast initial response

Results

Drawing Explorer transformed how engineers prepare RFP responses. What used to take hours of manual searching now takes seconds. Engineers can find that specific diagram they vaguely remember, discover similar past work they didn't know existed, and compile their findings without leaving the tool.

The real value is in the hours given back to actual engineering work, and the confidence that comes from knowing you're not missing relevant prior art buried somewhere in the archive.