Milvus

𝗖𝗮𝗻 𝗝𝗲𝘃 𝗿𝗲𝗽𝗹𝗮𝗰𝗲 𝗮 𝗿𝗲𝗿𝗮𝗻𝗸𝗲𝗿 𝗶𝗻 𝗥𝗔𝗚? We tested three setups on the same Milvus shortlist: 𝗻𝗼 𝗿𝗲𝗿𝗮𝗻𝗸𝗶𝗻𝗴, 𝗾𝘄𝗲𝗻𝟯.𝟳-𝘁𝗲𝘅𝘁-𝗿𝗲𝗿𝗮𝗻𝗸, 𝗮𝗻𝗱 𝗝𝗲𝘃. On 80 SciFact queries, qwen3.7-text-rerank improved nDCG@10 by 𝟬.𝟬𝟰𝟰𝟲 over no reranking, while Jev improved it by 𝟬.𝟬𝟳𝟳𝟴. Jev ranked best of the three, but its P50 reranking latency was 𝟭𝟬.𝟮× 𝗵𝗶𝗴𝗵𝗲𝗿 𝘁𝗵𝗮𝗻 𝗾𝘄𝗲𝗻, with an estimated cost per run of 𝟲.𝟳×. Why does Jev behave differently? Its interface is built around 𝘀𝘁𝗮𝘁𝗲 + 𝗾𝘂𝗲𝘀𝘁𝗶𝗼𝗻𝘀. For reranking, we keep the relevance question fixed and change the state for each query-candidate pair: “𝗜𝘀 𝘁𝗵𝗶𝘀 𝗰𝗮𝗻𝗱𝗶𝗱𝗮𝘁𝗲 𝗿𝗲𝗹𝗲𝘃𝗮𝗻𝘁 𝘁𝗼 𝘁𝗵𝗲 𝗾𝘂𝗲𝗿𝘆?” Jev returns a structured Yes/No judgment with probabilities, and we use 𝗣(𝗬𝗲𝘀) as the relevance score for sorting. In this experiment, 30 candidates meant 30 independent Jev requests running concurrently, so the latency includes that API overhead. We also tested using probability directly for filtering. At a 𝟬.𝟱 𝘁𝗵𝗿𝗲𝘀𝗵𝗼𝗹𝗱, top-5 precision reached 𝟰𝟵.𝟰%, but 𝟭𝟴.𝟴% 𝗼𝗳 𝗴𝗼𝗹𝗱-𝗿𝗲𝗹𝗲𝘃𝗮𝗻𝘁 𝗱𝗼𝗰𝘂𝗺𝗲𝗻𝘁𝘀 𝘄𝗲𝗿𝗲 𝗳𝗶𝗹𝘁𝗲𝗿𝗲𝗱 𝗼𝘂𝘁. That makes Jev attractive for offline retrieval, data cleaning, and evaluation, but much harder to use as-is for latency-sensitive online RAG. Next, we’re testing whether batching multiple candidate questions into one Jev request can bring that latency down.

𝗪𝗲 𝗶𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗲𝗱 𝗝𝗲𝘃 𝗶𝗻𝘁𝗼 𝘁𝗵𝗿𝗲𝗲 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗲 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀 𝘁𝗼 𝘁𝗲𝘀𝘁 𝘁𝗵𝗮𝘁 𝗶𝗱𝗲𝗮. First, in 𝗗𝗲𝗲𝗽𝗦𝗲𝗮𝗿𝗰𝗵𝗲𝗿, we used Jev to decide whether an Agentic Search workflow had gathered enough evidence to stop. Across 100 multi-hop questions, Jev matched DeepSeek V4 Flash at 93.25% Recall@5. Yet median decision latency fell from 2.23s to 0.55s, with estimated API cost at roughly one-seventh. Second, in 𝗠𝗲𝗺𝗦𝗲𝗮𝗿𝗰𝗵, we tested coding-agent memory reranking on 2,172 private Chinese and English queries. Jev improved Recall@5 from 74.71% to 79.41%, but remained behind the specialist Voyage rerank-3 at 81.87%. Third, in 𝗩𝗲𝗰𝘁𝗼𝗿 𝗚𝗿𝗮𝗽𝗵 𝗥𝗔𝗚, Jev reranked and filtered candidate relationships for multi-hop retrieval. It outperformed GPT-4o-mini, but trailed GPT-5-mini—by just 1 point on HotpotQA and 4.13 points on the harder MuSiQue benchmark. Jev is compelling for frequent, bounded judgments such as search stopping, filtering, routing, and simpler reranking. But as tasks become more domain-specific or reasoning-heavy, stronger generative models and specialist rerankers still earn their place. Want to dig into the experiments? • DeepSearcher — search-stopping experiment, code, and evaluation: https://github.com/zilliztech/deep-searcher/blob/master/evaluation/jev_stopping/README.md • MemSearch — Jev reranking integration and evaluation: https://github.com/zilliztech/memsearch/blob/main/evaluation/reranking-evaluation.md • Vector Graph RAG — Jev relationship filtering, evaluation, and cost analysis: https://github.com/zilliztech/vector-graph-rag/blob/main/evaluation/jev/README.md

46K GitHub stars for Milvus! ⭐ Huge thanks to everyone who has contributed code, shared feedback, reported issues, built with Milvus, or helped grow the community. This milestone wouldn’t exist without you. And while we’re celebrating, here are a few things we’ve been working on lately: → 𝗪𝗲 𝗹𝗮𝘂𝗻𝗰𝗵𝗲𝗱 𝗠𝗶𝗹𝘃𝘂𝘀 𝟯.𝟬 (https://x.com/zilliz_universe/status/2100250608926183893?s=20), bringing lake-native vector search and a much more powerful retrieval engine. → 𝗪𝗲'𝗿𝗲 𝗵𝗼𝘀𝘁𝗶𝗻𝗴 𝗮 𝘄𝗲𝗯𝗶𝗻𝗮𝗿 (https://zilliz.com/event/upgrade-to-milvus-30) on September 23 about how to upgrade to Milvus 3.0, as well as the availability of Milvus cool features on Zilliz Cloud. → 𝗔𝗻𝗱 𝘄𝗲’𝗿𝗲 𝗵𝗲𝗮𝗱𝗶𝗻𝗴 𝘁𝗼 𝗣𝗮𝗿𝗶𝘀 🇫🇷 (https://luma.com/bfo1andh) for our next Milvus Meetup at Criteo on October 7. We’d love to meet more of you there and talk vector search, retrieval, and what it takes to run AI systems at scale. 46K stars is a great reason to celebrate, but we’re even more excited about what we get to build next — together with all of you. Thank you for being part of Milvus. 💙 #Milvus #OpenSource #VectorDatabase #AIInfrastructure

Real-world data is rarely flat. A driving video, for example, has an overall scene—an intersection, a highway ramp, a residential street—but it also contains dozens of individual observations: vehicles, colors, positions, movements, and more. Finding the right video often means searching both levels at once. Our new StructArray Hybrid Search demo shows how Milvus 3.0 makes that possible. With a query like “a white truck waiting at an intersection,” Milvus can: • understand the intersection at the video level • identify the white truck within nested observations • combine both signals into one ranked result StructArray keeps parent records and their child elements together, while still allowing vectors inside those nested elements to be searched directly. That means less data modeling complexity—and more context preserved during retrieval. If you’re building search for multimodal datasets, documents with sections, products with variants, or any other parent-child data, this pattern is worth exploring. Try it now: https://demos.milvus.io/structarray-search/ GitHub Code: https://github.com/zc277584121/milvus3-demos/tree/main/demos/structarray-search

A shopper searching for “a compact dark wood desk for a small home office” cares about semantic relevance. A commerce ranking may also need to consider rating, popularity, price, and freshness. 𝗧𝗵𝗲 𝗠𝗶𝗹𝘃𝘂𝘀 𝟯.𝟬 𝗙𝘂𝗻𝗰𝘁𝗶𝗼𝗻 𝗖𝗵𝗮𝗶𝗻 𝗿𝗲𝗿𝗮𝗻𝗸 𝗱𝗲𝗺𝗼 𝘀𝗵𝗼𝘄𝘀 𝗵𝗼𝘄 𝘁𝗼 𝗰𝗼𝗺𝗯𝗶𝗻𝗲 𝘁𝗵𝗲𝗺 𝗶𝗻 𝗼𝗻𝗲 𝗿𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗽𝗮𝘁𝗵: • BGE-M3 converts the query into an embedding. • Milvus runs COSINE vector search and recalls the top 20 products. • A server-side Function Chain calculates popularity, price affinity, freshness, and a normalized semantic score. • An XGBoost model combines the features into a new business score. • Milvus returns the reranked results. One thing worth noting: the final ranking happens inside Milvus, not in FastAPI or the frontend. This pattern is useful whenever semantic similarity is only the starting point. Product search, for example, may also rank results by rating, popularity, price, and freshness. Instead of retrieving candidates and rebuilding the ranking pipeline in another service, the retrieval layer can apply those signals and return the final ranked result set. Try it: https://demos.milvus.io/function-chain-rerank/ Code: https://github.com/zc277584121/milvus3-demos/tree/main/demos/function-chain-rerank

Jev is built to decide, not to chat. That makes it a surprisingly good fit for search. Milvus first retrieves candidate passages with vector search and metadata filters. Jev then looks at those passages and judges whether they actually help with the task. In practice, the flow looks like this: Query -> Milvus retrieves -> Jev judges -> your application acts For the common reranking case, JevRerankFunction takes a query, candidate documents, and top_k, then returns them scored and reordered. We also put together nine small tutorials to show what else this pattern can do: route a query, check whether a cached answer can be reused, screen documents before indexing, or decide whether an agent has enough evidence to stop searching. Each example uses a small dataset, so it's easy to follow what Milvus retrieved, what Jev decided, and what happened next. Pick the example that feels closest to what you're building and give it a try. Try it now: https://milvus.io/docs/search_with_jev.md

Millions of questions. How does Milvus help find the right one? Traditional full-text retrieval finds questions through the words they contain. In a growing question bank, students and teachers also need questions that test similar concepts with different wording and fit the intended curriculum and difficulty level. To support these needs, this Milvus-based architecture combines semantic retrieval, BM25 keyword search, and scalar filtering in a five-step pipeline: 1. Data collection and cleaning. Collect question text, explanations, and metadata; standardize fields such as course, education level, concepts, and difficulty. 2. Vectorization. Use education-tuned embeddings for semantic matching and Milvus’s built-in BM25 function for keyword retrieval. 3. Hybrid index construction. Create HNSW/COSINE indexes for dense vectors, sparse indexes for BM25, and scalar indexes for metadata filtering. 4. Hybrid retrieval. Combine semantic and keyword search under metadata constraints, then merge results with WeightedRanker or RRFRanker. 5. Reranking and output. Use a Cross-Encoder to rerank candidates for test assembly, mistake-based review recommendations, and search by question. The design pairs semantic matching with lexical relevance and explicit teaching constraints, helping students and teachers find questions suited to their learning tasks. #Milvus #EdTech

SQ, PQ, or PRQ: how do they compress vectors in Milvus HNSW? HNSW can deliver high recall, but its full precision vectors and layered graph consume substantial memory. Milvus offers HNSW_SQ, HNSW_PQ, and HNSW_PRQ to compress the vectors used in graph construction and distance calculations. Each method encodes vectors differently: • SQ quantizes each dimension. SQ8 and SQ6 use 8 and 6 bits per dimension, with independently estimated ranges. SQ4U uses 4-bit unsigned values and one shared uniform range. • PQ splits the vector into subvectors. Each subvector is represented by a codebook index. • PRQ starts with PQ, then encodes residual error in stages. It retains additional information beyond the initial approximation, with additional training, build, and distance calculation costs. All three quantized index variants use the same HNSW framework, but quantization can change which vectors are connected during construction and how candidates are ranked during retrieval. They can also use higher precision refinement to rerank candidates and potentially improve recall, while adding compute, storage, and memory costs. To compare SQ, PQ, and PRQ, set a recall target first, then measure memory after loading, throughput, and build time—including the cost of any refinement needed to reach that target.

🎤 Our Solutions Architect, Yulan Yan, will speak at Tokyo AI’s 𝗥𝗲𝗹𝗶𝗮𝗯𝗹𝗲 𝗮𝗴𝗲𝗻𝘁𝘀: 𝗺𝗲𝗺𝗼𝗿𝘆, 𝗲𝘅𝗲𝗰𝘂𝘁𝗶𝗼𝗻 𝘀𝘁𝗮𝘁𝗲 𝗮𝗻𝗱 𝗿𝗲𝗰𝗼𝘃𝗲𝗿𝘆 in Tokyo on 14 October! Her talk, “Agent Memory in Production: Patterns from Real-World AI Applications,” will explore four recurring memory patterns across AI companions, enterprise agents, robotics, and personal AI devices. Drawing on systems using Milvus and Zilliz Cloud, she’ll examine what to remember, when to consolidate interactions, and how to prioritise recent and older memories as applications evolve. 📣 𝗕𝗿𝗶𝗻𝗴 𝘆𝗼𝘂𝗿 𝗮𝗴𝗲𝗻𝘁 𝗺𝗲𝗺𝗼𝗿𝘆 𝗾𝘂𝗲𝘀𝘁𝗶𝗼𝗻𝘀! 𝗥𝗲𝗴𝗶𝘀𝘁𝗲𝗿 𝘁𝗼𝗱𝗮𝘆: https://luma.com/9sta84i8

𝗞𝗹𝗶𝗻𝗴 𝗔𝗜 now serves 100M+ users across 224 countries and regions, with Milvus powering the shared vector search platform behind Kling AI and various products across its parent company Kuaishou. Previously, their Elasticsearch setup tied storage to compute, coupled reads with writes, and capped vector dimensions at 4,096—making it hard to support fast-growing content while running heavy live queries. By adopting Milvus, the database team separated read and write resources and scaled storage independently from compute. Today, this architecture supports production at massive scale: 🔹 𝟭𝟬𝗕+ 𝘃𝗲𝗰𝘁𝗼𝗿𝘀 in a single largest production cluster 🔹 𝟭𝟬,𝟬𝟬𝟬+ 𝗽𝗲𝗮𝗸 𝗤𝗣𝗦 while concurrently handling live writes 🔹 𝗠𝘂𝗹𝘁𝗶-𝗱𝗮𝘁𝗮𝗰𝗲𝗻𝘁𝗲𝗿 𝗱𝗶𝘀𝗮𝘀𝘁𝗲𝗿 𝗿𝗲𝗰𝗼𝘃𝗲𝗿𝘆 with full searchable replicas, ensuring zero downtime for searches and writes even during full site outages Read the full story to see how Kling AI scales vector search: 👉 https://zilliz.com/customers/kling-ai

𝗞𝗹𝗶𝗻𝗴 𝗔𝗜 now serves 100M+ users across 224 countries and regions, with Milvus powering the shared vector search platform behind Kling AI and various products across its parent company Kuaishou. Previously, their Elasticsearch setup tied storage to compute, coupled reads with writes, and capped vector dimensions at 4,096—making it hard to support fast-growing content while running heavy live queries. By adopting Milvus, the database team separated read and write resources and scaled storage independently from compute. Today, this architecture supports production at massive scale: 🔹 𝟭𝟬𝗕+ 𝘃𝗲𝗰𝘁𝗼𝗿𝘀 in a single largest production cluster 🔹 𝟭𝟬,𝟬𝟬𝟬+ 𝗽𝗲𝗮𝗸 𝗤𝗣𝗦 while concurrently handling live writes 🔹 𝗠𝘂𝗹𝘁𝗶-𝗱𝗮𝘁𝗮𝗰𝗲𝗻𝘁𝗲𝗿 𝗱𝗶𝘀𝗮𝘀𝘁𝗲𝗿 𝗿𝗲𝗰𝗼𝘃𝗲𝗿𝘆 with full searchable replicas, ensuring zero downtime for searches and writes even during full site outages Read the full story to see how Kling AI scales vector search: 👉 https://zilliz.com/customers/kling-ai

"Code only" vs. "Code with explanation"—similar questions don't always mean identical requirements. That's why traditional semantic caching often breaks. To solve this, Zilliz CTO James Luan tested 𝗝𝗲𝘃, a budget-friendly model that acts as a gateway guard before serving cached outputs. Jev evaluates 5 critical dimensions in real time: • 𝗧𝗮𝘀𝗸 𝗙𝗶𝘁: Does the answer directly solve the current query? • 𝗖𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝘁 𝗠𝗮𝘁𝗰𝗵: Does it meet specific parameters (e.g., region, version, limits)? • 𝗙𝗼𝗿𝗺𝗮𝘁 & 𝗖𝗼𝗺𝗽𝗹𝗲𝘁𝗲𝗻𝗲𝘀𝘀: Is it fully complete in the requested style and language? • 𝗧𝗶𝗺𝗲 𝗦𝗲𝗻𝘀𝗶𝘁𝗶𝘃𝗶𝘁𝘆: Does it rely on outdated or changing data? • 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗔𝘃𝗮𝗶𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆: Is all required context present to support the response? The test results on 1,152 real-world examples: • 𝗭𝗲𝗿𝗼 𝗶𝗻𝗰𝗼𝗿𝗿𝗲𝗰𝘁 𝗿𝗲𝘂𝘀𝗲𝘀 recorded. • 𝟮𝟮𝟳 𝘃𝗮𝗹𝗶𝗱 𝗿𝗲𝘀𝗽𝗼𝗻𝘀𝗲𝘀 𝘀𝗮𝘃𝗲𝗱, avoiding costly re-generations. • 𝗠𝗲𝗱𝗶𝗮𝗻 𝗹𝗮𝘁𝗲𝗻𝗰𝘆 𝗽𝗲𝗻𝗮𝗹𝘁𝘆: Only 0.79s. Semantic caching is a tradeoff between LLM costs and response accuracy. Jev shows that adding a cheap verifier makes that tradeoff significantly safer. Check out the complete benchmark breakdown (feat. Laya and GPTCache reranker): https://milvus.io/blog/jevs-cheap-decisions-make-semantic-caching-worth-another-try.md