Executive summary: A six‑month benchmark compares FAISS 1.9, Chroma 0.6, and Pinecone 1.6 on 100 million 768‑dimensional embeddings, finding FAISS achieves the lowest p99 latency and cost, Pinecone is the most expensive but offers managed SaaS benefits, and Chroma provides native metadata filtering.

Key results: FAISS 1.9 hits 2.1ms p99 latency for 100M vectors using IVF1024 and PQ48 indexing on 16 vCPU/64GB RAM; Chroma 0.6 offers native DuckDB‑backed metadata filtering with 100M-scale support but lower recall at 100k QPS; Pinecone 1.6 costs about $10,920/month for 100M vectors at 50k QPS, 11x FAISS EC2 cost.

Pinecone guidance: Start with a minimum of 3 s1.xlarge pods for 100M vectors and scale only if thresholds are met; autoscaling beta is available for burst workloads.

Developer tips: Tune FAISS nprobe for balance between recall and latency; leverage Chroma’s native filtering; choose Pinecone pod size to balance cost and performance.

Chroma recommendations: Native filtering helps with datasets under 150M vectors and under ~100 attributes; note parquet read latency and higher memory for larger attribute sets require manual indexing for high‑cardinality attributes.

Benchmark setup: Tests run on self‑hosted AWS instances (c6i.4xlarge and c6i.8xlarge) with 100M embeddings from all-MiniLM-L6-v2, measuring p50/p95/p99 latency, recall@10, max throughput, and monthly cost, including sustained 100k QPS for 30 minutes.

Metadata filtering comparison: FAISS with external stores (e.g., Redis) adds latency; Chroma’s native filtering adds about 1.2ms for 10 attributes, versus FAISS+Redis about 4.7ms, with performance dropping as attributes exceed ~100.

Cost comparison: Pinecone’s 3 s1.xlarge pods run about $10,920/month; FAISS self‑hosted is around $972/month, with costs scaling alongside Pinecone pod size and other configurations.

Case study: An e‑commerce migration showed moving from Chroma 0.5 to FAISS 1.9 with Redis reduced p99 latency from 2.4s to 2.1ms, improved recall from 89% to 98.7%, boosted throughput to 142k QPS, and cut monthly costs from around $4k to $2.9k.

Index tuning: For FAISS, set nprobe to 32 to achieve about 98.7% recall at 2.1ms p99, and avoid setting nprobe equal to the number of clusters (1024) to prevent latency spikes.

Usage scenarios: FAISS is best for self‑hosted, high throughput, and cost sensitivity; Chroma suits embedded workloads needing native filtering for moderate datasets; Pinecone fits zero‑ops, multi‑region, and very large datasets.