🗽 LanceDB Featured in Times Square

Thank you to Microsoft for Startups for featuring LanceDB as part of its invite-only Pegasus Program! We joined Pegasus earlier this year as we continue building the new foundation for AI data.

🤖 How To Train Your Robot: The LanceDB Edition

LeRobot now reads Lance datasets natively, so training pulls frames straight from object storage with a true global shuffle and no download. On all of DROID (27.6M frames, 369 GB), 10k SmolVLA steps on 8 H100s took 1h27m vs 2h00m for the standard reader on a local NVMe copy, with identical loss. GPUs waited on data 1.7% of the time vs 37.4%.

Read more →

🎥 Train on the Right Data: Mining a Robot Fleet with LanceDB

VIDEO

Finding fleet moments that resemble a robot's edge case is hard when data is organized by episode instead of time step. Follow the full loop on DROID: store each of 32,212 time steps as a row with pixels, proprioception, physics, and instruction, add DINOv2 embeddings at ingest, run one hybrid vector and full-text scan instead of a post-filter, and save the result as a versioned training slice.

Watch the demo →

🔍 From 10 Million to 10 Billion: Vector Search Built to Grow

Five-bit RaBitQ hit 96.2% recall at 1,614 QPS on 10M 768d vectors, vs 91.5% for PQ384 at 1,047 QPS. At 10B vectors, distributed search held an 18.05 ms p50 and reached 1,066 QPS at 256 concurrent requests. A one-bit distance bound prunes over 99% of candidates before full scoring, and switching from fast to normal at query time lifts recall from 75.0% to 93.1% with no index rebuild.

Read more →

📚 Also Published

📺 Feature of the Week

We launched a new series! Feature of the Week will spotlight what our engineers and open source contributors are building and the work behind it!

You can find all the videos in the Feature of the Week playlist. Check out the latest episodes:

🎤 Talks & Recordings

How Exa Indexes the Entire Web: Ray Data and Lance at Billion-Document Scale

Lei Xu (CTO @ LanceDB) · Hubert Yuan (Software Engineer @ Exa)

At Ray Summit 2026, Lei Xu (LanceDB) and Hubert Yuan (Exa) walk through exa-d, Exa's framework on Lance and Ray Data for reprocessing embeddings, extracted text, and search signals across hundreds of billions of web pages, where dependency graphs drive execution, fragment-level patching avoids write amplification, and Ray Actors enable stateful model reuse.

Watch the recording →

📅 Upcoming Events

Open Intelligence Summit — October 12-13, 2026 · San Francisco, CA

LanceDB is sponsoring the closing reception at Open Intelligence Summit, hosted by DigitalOcean and Nvidia. Check out LanceDB CTO Lei Xu's session: Build an Open Multimodal Data Stack for Search, Curation, and Training.

Register →

Long Horizon — October 20-21, 2026 · San Francisco, CA

LanceDB is sponsoring Long Horizon by Encord! We help frontier labs, physical AI teams, and enterprises train on, search, and curate their data at massive scale, accelerating the data flywheel so teams can build better models faster. If that sounds like you, come find us on the Expo floor!

Register →

Reverie — November 5, 2026 · San Francisco, CA

Reverie is a one-day technical summit for researchers and engineers building the data systems behind world models, generative video, physical AI, multimodal search, and agentic research.

Apply to Attend →

🌟 Open Source Releases

Project Description
Lance v12.0.0
Release notes		 • Breaking: namespace merge_insert takes a list of key columns for composite-key upserts, a Rust API change (#8915); json_extract predicates no longer route to JSON indices and fall back to a full scan (#9101)
• IVF index improvements: Shared partition scans across batch vector queries (#7640), proportional hierarchical k-means for training (#9050), automatic partition splitting/joining in single optimize pass (#9051), and metric-aware initial probe budgets (#9195)
• Full-text search performance: Deferred phrase confirmation with score bounds (#8749), bulk intersection for wide AND queries (#9030), and bounded classic conjunction scoring (#9033)
• Row ID performance: Fragment reuse indexes open 10–2200× faster (#8887); cached row ID index with ranked bitmap lookups (#8930)
• Python API additions: LanceDataset.slice() method (#8059), Bitmap binding for RoaringBitmap (#7837), custom ObjectStoreProvider registration (#8522), and dataset deep clone (#9181)
LanceDB v0.39.0
Release notes		 • Breaking: materialized views can carry function-bound computed columns (#4119), and refreshing a computed column now recomputes rows whose inputs or definition changed instead of only filling nulls (#4161)
• Remote connections run SQL synchronously or in the background with execute_query/execute_query_async (#4070)
• Breaking: get_job/job_history are replaced by open_job, describe_job, and query_job_events, which expose per-fragment progress and failure info (#4130)
• Function management APIs expanded: drop_function, list_functions, support for nested blob signatures and nullable named outputs (#4097, #4108, #4109, #4123)
lance-namespace v0.12.0 – v0.13.0
Release notes		 • Merge insert operations now support multiple columns as the merge key (#363) — breaking change to the spec
• Added adaptive IVF probe query fields for dynamic tuning of vector index searches (#367)
lance-context v0.6.6 – v0.6.7
Release notes		 • Bounded memory for WAL-backed rollout pagination (#230), large-blob compaction (#229), MemWAL merges via ROLLOUT_MERGE_MAX_BYTES (#242), and Context point-reads with deferred payload loading (#243)
• New blob-size histogram metric and warning logs for oversized blobs to help diagnose storage patterns (#240)
• Fixed scheduler starvation where MergeWal could block Compact operations (#239)

🫶 Community Contributions

Thank you to contributors from Bytedance, Baidu, Adobe, Huawei, Tencent, DeepL, LumaLabs for improvements across storage, indexing, query execution, distributed processing, and ecosystem integrations in LanceDB, Lance, and the broader ecosystem.

Notable contributions this month:

  • @summaryzb — Java scan APIs can now target specific vector index segments, so distributed vector search can split a query across workers by segment
  • @sezruby — Batched vector queries now read each probed IVF partition once and score every query against it, and added a Java batch vector search binding
  • @jiaoew1991 — Sped up row-ID and scan paths: streaming dataset version runs with a cursor, faster dense bitmap decoding, less system-column pipeline overhead, and single-producer Python reader batches
  • @dshepelev15 — Kept the row-ID index cached with ranked bitmap lookups (get 422 to 126 ns on a 17.4B-row table), and cut object-store requests by batching page reads and reading large metadata in concurrent chunks
  • @zhangyue19921010 — Added an experimental MinHash LSH scalar index for Jaccard text similarity across Rust, Python, and Java, plus matched-row offsets from update_columns and removal of an O(C²) commit validation step
  • @jonasdedden — Fixed Python API annotations (CompactionOptions, create_index_uncommitted, a misspelled option) and decoded deprecated substrait timestamp literals as microseconds
  • @wombatu-kun — Widened the RaBitQ FastScan accumulator so distances above 1024 rotated dims no longer overflow, and kept commits working when an existing index can't be opened
  • @LuciferYang — About 30 hardening fixes across indexing and query paths, including rejecting malformed IVF/PQ/RaBitQ metadata instead of panicking, -0.0/0.0 float filter equality, Float16 literal coercion, and merge_insert routing fixes
  • @ddupg — Restored Python index retraining, aligned Java inverted index options with Rust, and pruned scalar index segments outside the queried fragments

A heartfelt thank you to our community contributors of Lance and LanceDB this past month:

@1fanwang • @a-erofeev • @abhisheklearn12 • @adityabil • @adityavaid • @ali2arslan • @amunra • @bilalatique • @chakshu-dhannawat • @chensammi • @chenyu-x • @crossthebridgetpa • @cyb3rb1ade • @dajiaohuang • @dcfocus • @ddupg • @dentiny • @desty • @dshepelev15 • @easyrev • @emecii • @everysympathy • @fanng1 • @farazshaikh • @fede-kamel • @fightboxing • @foobar • @gabriel39 • @gallardot • @hfutatzhanghb • @hiltonhe • @importcpp • @ivscheianu • @jackylee-ch • @james-rms • @jan-exa • @jayzhou2309 • @jennifermell • @jiaoew1991 • @jiaqizho • @jimmy-xie-fleet • @joaquinhuigomez • @jonasdedden • @jtbandes • @kaiqijinwow • @kamronis • @keunhong • @leoreeyang • @lichuang • @lindseyz1205 • @luciferyang • @majin1102 • @manucorporat • @maswin • @maxfreedompollard • @mikewhb • @mohammadhijjawi97 • @ningsh7 • @nitish-1303 • @notaarushi019 • @nyl3532016 • @pengw0048 • @prrao87 • @qiuyuhang • @risto0211 • @rotty3000 • @rudra-g-23 • @rupertmaiti2005 • @ruslan-shaydullin • @sapnilb15 • @sbrunk • @sezruby • @shoemoney • @siddarthareddy8 • @simpleqt • @slachiewicz • @smarra • @sravan1011 • @summaryzb • @sunyuechi • @szetohoyan • @taozhiyi13 • @terapyon • @ther1sing3un • @timsaucer • @tyagiquamar • @u70b3 • @venkata91 • @vgrigoriu • @winklemad • @wombatu-kun • @xiaguanglei • @xloya • @xtangxtang • @xuqianjin-stars • @yanghua • @yangjunz • @yhz5613813 • @youssef-tharwat • @zhangstar333 • @zhangyue19921010 • @ziwenzhang

🤝 Lance Community Sync Recap

The Lance 12.0.0 SDK ships with the Lance 2.2 file format as the default, and this month's syncs covered the 11.0.0 and 12.0.0 releases alongside a governance change to PR-based voting. Design discussion centered on a production-ready redesign of stable row IDs, Transactions V2, a proposal to narrow Lance's core scope, and a separate proposal for a tiered manifest structure. The community also worked through the format proposal backlog and better ways for teams building on Lance to upstream their changes.

The next Lance Community Sync will take place on Thursday, October 9 @ 9am PT.