This search infrastructure is designed for AI agent use only. It provides backend search capabilities across multiple search engines to help agents locate and retrieve information from the 30,000+ documents and 20,000+ images.
The search bots operate independently in the background, accessible only to other AI agents for information retrieval, fact-checking, and cross-referencing tasks. Users do not have direct access to search functionality; instead, they browse the AI-curated codex.
Purpose: Coordinates searches across multiple search engines simultaneously
Capabilities:
Search Engines Integrated:
Configuration:
coordinator:
max_concurrent_searches: 10
timeout_seconds: 30
cache_ttl: 3600
deduplication: true
relevance_threshold: 0.7
Purpose: Fast full-text search across indexed documents
Technology:
Features:
Index Structure:
{
"document_id": "unique-id",
"title": "document title",
"content": "full text content",
"summary": "AI-generated summary",
"date": "2024-01-01",
"category": "legal",
"source": "SDNY",
"verification_level": 1,
"entities": ["person1", "location1"],
"tags": ["keyword1", "keyword2"]
}
Purpose: Search and retrieve images from the 20,000+ image database
Capabilities:
Technology:
Search Methods:
Purpose: Natural language understanding and semantic search
Technology:
Capabilities:
Example Queries:
Purpose: Search by people, places, organizations
Capabilities:
Entity Types:
Purpose: Find relationships and connections between documents
Capabilities:
Algorithms:
Purpose: Verify claims and find supporting evidence
Capabilities:
Process:
Purpose: Validate document sources and authenticity
Capabilities:
Verification Levels:
Base URL: http://internal-api.epstein-codex.local/api/v1/
Authentication: Internal service token (not exposed to public)
POST /search/documents
{
"query": "search terms",
"filters": {
"category": ["legal", "financial"],
"date_range": {"start": "2005-01-01", "end": "2008-12-31"},
"source": ["SDNY"],
"verification_level": [1, 2]
},
"limit": 100,
"offset": 0
}
POST /search/images
{
"query": "search terms",
"search_type": "text|visual|metadata",
"filters": {
"category": ["evidence", "location"],
"date_range": {},
"location": "Little St. James"
}
}
POST /search/semantic
{
"query": "natural language query",
"context": "additional context",
"max_results": 50
}
POST /search/entities
{
"entity_type": "person|location|organization",
"entity_name": "name",
"relationship_type": "connected_to|mentioned_with",
"depth": 2
}
POST /search/cross-reference
{
"document_id": "doc-123",
"relationship_types": ["cites", "cited_by", "related", "similar"],
"max_depth": 3
}
POST /search/fact-check
{
"claim": "statement to verify",
"context": "additional context",
"source_types": ["court", "government", "media"]
}
google_search:
api_key: ${GOOGLE_SEARCH_API_KEY}
cx: ${GOOGLE_CUSTOM_SEARCH_CX}
safe_search: off
num_results: 10
rate_limit: 100/day
bing_search:
api_key: ${BING_SEARCH_API_KEY}
endpoint: https://api.bing.microsoft.com/v7.0/search
num_results: 10
rate_limit: 1000/month
duckduckgo:
no_api_key_required: true
rate_limit: respectful
safe_search: off
azure_search:
api_key: ${AZURE_SEARCH_KEY}
endpoint: ${AZURE_SEARCH_ENDPOINT}
index_name: epstein-documents
scoring_profile: relevance-boost
elasticsearch:
hosts: [internal-es-cluster:9200]
index: epstein-codex
shards: 5
replicas: 2
max_results: 10000
Agents communicate via internal message queue (RabbitMQ):
{
"request_id": "uuid",
"requesting_agent": "document-analysis-agent",
"search_type": "semantic",
"query": {
"text": "search query",
"filters": {},
"options": {}
},
"priority": "normal|high|urgent",
"timeout": 30
}
{
"request_id": "uuid",
"status": "success|partial|failed",
"results": [
{
"document_id": "doc-123",
"title": "Document Title",
"relevance_score": 0.95,
"summary": "...",
"url": "internal://...",
"metadata": {}
}
],
"total_found": 150,
"search_time_ms": 245,
"sources_searched": ["elasticsearch", "azure", "google"]
}
from search_api import SearchClient
search = SearchClient(service_token=SERVICE_TOKEN)
# Search for related documents
results = search.documents(
query="Jeffrey Epstein Palm Beach",
filters={
"date_range": {"start": "2005-01-01", "end": "2008-12-31"},
"category": ["legal", "investigation"]
},
limit=50
)
for doc in results:
# Process each document
analyze_document(doc)
# Find all documents mentioning an entity
entity_docs = search.entities(
entity_name="Little St. James",
entity_type="location",
include_related=True
)
# Build entity profile
profile = build_entity_profile(entity_docs)
# Find all documents connected to a source document
related = search.cross_reference(
document_id="doc-123",
relationship_types=["cites", "cited_by", "mentions"],
max_depth=2
)
# Build citation network
network = build_citation_network(related)
This search infrastructure operates entirely in the background, accessible only to AI agents for maintaining and organizing the Epstein Files Codex.
Last Updated: December 2024