AI/ML News & Innovations Hub

Why Image Generation Needs More Than Bigger Models with Fatih Porikli - #773

Text-to-image models have become remarkably good at producing realistic images. But realism isn’t the same as correctness. Ask for several distinct people, a specific composition, or a high-resolution image generated locally, and today’s models still struggle in surprising ways. In this episode, Fatih Porikli, Vice President of Technology at Qualcomm, joins me to discuss what remains unsolved in image generation and several approaches his team presented at CVPR to address those challenges. We explore why better training objectives can improve controllability, how separating scene planning from rendering may lead to more reliable image generation, techniques for generating 16-megapixel images efficiently on edge devices, and new methods for eliminating the visible artifacts that often appear in AI-powered image editing. Along the way, we discuss reinforcement learning for image generation, agentic image generation pipelines, on-device AI, and what the next phase of progress in generative vision systems is likely to look like. 🗒️ Full show notes: https://twimlai.com/go/773

Comment on NVIDIA’s Global Context ViT Achieves SOTA Performance on CV Tasks Without Expensive Computation by VoiceAILabs

I liked how GC ViT pairs global self-attention with token generation to avoid the usual quadratic blow-up while still modeling long-range context — that seems really practical for high-res image tasks. I've noticed similar gains when shaving attention overhead for on-device models at VoiceAILabs VoiceAILabs , where small architecture changes can make deployment much more realistic.

Comment on Megvii UPerNet Performs Multi-Level Visual Scene Interpretation at a Glance by John Mick

Combining heterogeneous datasets into Broden+ seems just as important as the network design itself. The multi-task approach is especially interesting because scene, object, part, material, and texture labels exist at different levels of granularity. I wonder how UPerNet handles conflicting or overlapping annotations when the same visual region appears across datasets.

Building Multimodal Workflows with a Local LLM

Image inputs and structured outputs with Gemma 4 and Ollama The post Building Multimodal Workflows with a Local LLM appeared first on Towards Data Science .

Comment on AI Offers Video Game Design Possibilities Far Beyond Virtual Reality by luma dream machin

The discussion about ai offers video game design possibilities far beyond virtual reality raises some really valid points. This perspective is refreshing. luma dream machin

MS, 'MAI-이미지-2.6' 공개...아레나 '텍스트-이미지' 분야 2위 등극

마이크로소프트(MS)가 자체 개발한 이미지 생성 모델의 성능을 크게 끌어올리며 오픈AI와 구글, 메타, xAI가 경쟁하는 생성 AI 시장에서 존재감을 강화하고 있다. 최신 모델이 이미지 생성 모델 평가에서 2위에 오르면서 MS가 자체 AI 모델 개발에 투자해 온 성과가 가시화되고 있다는 평가다.MS가 10일(현지시간) 텍스트-이미지 생성 모델 ‘MAI-이미지-2.6(MAI-Image-2.6)’을 공개했다.MAI-이미지-2.6은 아레나의 이미지 생성 모델 사용자 선호도 평가(Text-to-Image Arena)에서 1336점으로 전체

Uber expands Uber Bike to 100 more Indian cities

Uber has expanded its Uber Bike service to 100 additional cities in India, taking its presence to over 220 cities across the country. The expansion will cover 18 states and one Union Territory. The new cities include Jammu, Tirunelveli, Silchar and Jamnagar, among others. Uber Bike will offer two-wheeler rides for short-distance travel as well as first and last-mile connectivity. The expansion is part of Uber’s strategy to strengthen its presence beyond major metros and build a multimodal mobility platform in India. The company currently offers bikes, autos, cars, airport rides and intercity services through its platform. Uber said the expansion will create flexible earning opportunities for tens of thousands of local drivers. According to the company, more than 2 million Indians earn through its platform. As part of its Independence Day campaign, Uber is also offering first-time riders a free first ride for a limited period. Uber Bike includes safety features such as real-time driver ID checks, an in-app SOS button, trip sharing and a 24x7 safety line.

Accuracy of GPT-4 Vision to extract exact numbers from graphs

This is a case where in-context training has previously been shown to help on a vision task. Contemporaneous with this old forum topic is a paper showing that examples of “how to turn vision into readings” can improve the actual readings provided: arXiv.org The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision) Large multimodal models (LMMs) extend large language models (LLMs) with multi-sensory skills, such as visual understanding, to achieve stronger generic intelligence. In this paper, we analyze the latest model, GPT-4V(ision), to deepen the... Newer models are able to use larger imagery input, but do not upsize images themselves (something you can do, at expense). There is a transition from tiles to patches, at least as a billing method, in new models, giving linear relation between area and input tokens billed. Then, with gpt-5.6 (on the API, where developers know what is being done to images), the default image downsize cap is “original”—where an image such as 3600×2400 can be sent without downsize, providing more information in the large context attention sequence rather than in the individual semantic embedding that covers a large area with a small input. That, along with further post-training, should imply higher-quality positional answering in graphs with new models and big images. With reasoning.effort other than “none” on OpenAI gpt-5.2+ models, you do not have control over sampling constraints; thus, it is expected that each answer would differ. You can…

Comment on DeepMind’s Zipper: Fusing Unimodal Generative Models into Multimodal Powerhouses by Seating Chart Generator

https://seatingchartgenerator.app can suggest a new approach to planning the seating arrangement, which will make the process much more efficient.

Microsoft's MAI-Image-2.6 Jumps to #2, Beating GPT Image 2 in 3D

Microsoft's MAI-Image-2.6 debuts at #2 on Arena's text-to-image leaderboard, closing within 45 points of GPT Image 2 across every major image category.

Silent Telepathy/V2K run using OpenAI

Need help locating these within this platform. They’ve been removed before but there needs to be a mechanism to prevent them from being rebuilt.

Meta is back with Muse Glimmer: local, agentic, multimodal, and open source

Comment on The Future of Vision AI: How Apple’s AIMV2 Leverages Images and Text to Lead the Pack by Wilfried Glynn

GIF-focused users can browse [a filterable list of GIF-tagged NSFW subreddits]. redd18.com

Feature Engineering for Multimodal Data: From Laptop to Cluster with LanceDB

Run a multimodal feature-engineering pipeline in ten minutes with geneva-examples, then rebuild its CLIP embedding step with LanceDB's feature engineering API, the same calls you'd use to enrich your own tables on LanceDB Enterprise.

Comment on NVIDIA’s OMCAT: A Breakthrough in Cross-Modal Temporal Understanding for Multimodal AI by Mark

Awesome work, NVIDIA team! OMCAT and OCTAV are a huge step forward for multimodal AI—finally tackling the tricky challenge of cross-modal temporal alignment with a clever blend of RoTE and a purpose-built dataset. Can't wait to see how this pushes AVQA and temporal reasoning forward. Congrats on the release!

Comment on CMU & Inspired Cognition’s DocPrompting Improves Code Generation by Retrieving Relevant Documentation by Rafael Costa

Great insight on how documentation retrieval bridges the gap for unseen libraries—reminds me of how designers also need contextual, personalized references when exploring symbolic elements like birth flowers for tattoos. Just as DocPrompting uses real docs to ground code generation, NanoPic’s birth flower tattoo generator helps users visualize meaningful, month-specific botanical designs directly on their skin—selecting their birth month, uploading a photo, and adding placement notes for a realistic fine-line preview. It’s a thoughtful, AI-assisted workflow for ideation grounded in personal symbolism: NanoPic’s birth flower tattoo generator

Comment on DeepMind’s JetFormer: Unified Multimodal Models Without Modelling Constraints by genesiscard com

This was a helpful and informative read. Everything was explained clearly.

Introducing Shieldstral.

Shieldstral introduces a 3B open-weights multimodal safety classifier that outperforms models up to 7x its size.

Exclusive: Gen AI startup Simplismart set to raise $9 Mn in Series B led by Dallas Venture Capital

Generative artificial intelligence startup Simplismart is set to raise nearly Rs 97 crore (around $9 million) in a Series B funding round led by Dallas Venture Capital, with participation from existing investors Accel India and Shastra VC, as well as new investor Micromax Informatics. The board of Simplismart has approved a resolution to issue 12,100 CCPS at an issue price of Rs 37 lakh each to raise the capital, according to its regulatory filings reviewed by Entrackr. Dallas Venture Capital will invest Rs 44.84 crore, followed by Accel India with Rs 24.09 crore. Shastra VC will infuse Rs 22.24 crore, while Micromax Informatics will invest Rs 5.19 crore. Tarusa Capital and Simraan Teckchandani will also participate in the round with investments of Rs 37 lakh each. According to Entrackr's estimates, the Bengaluru-based company will be valued at around Rs 826 crore post allotment. The fresh capital will be used to support the company's long term growth plans, including business expansion, working capital requirements, and other general corporate purposes. Founded in 2022 by former Oracle and Google engineers Amritanshu Jain and Devansh Ghatak, Simplismart develops AI infrastructure software that enables enterprises to deploy, manage, and optimize production grade AI models without writing code. Its inference-first platform improves GPU utilization and reduces inference costs across workloads such as large language models (LLMs), vision language models, speech recognition, and…

New Realtime models on the API: gpt-realtime-2.1 and gpt-realtime-2.1-mini

Realtime 2.1 is significantly harder to use for structured voice agents than realtime-mini We’ve spent the last couple of weeks migrating a production outbound voice agent from realtime-mini to gpt-realtime-2.1-mini and, despite following the new prompting guide closely, we’ve hit a number of behavioural regressions. Our use case is a highly structured telephony workflow (cold calling), where the agent follows explicit conversation steps, conditional branches and tool calls. The main issues we’ve observed are: Instruction leakage. The model frequently narrates its own behaviour: “Let me continue with the call…” “I’ll guide you through the next step…” “Let’s move on to the next question…” “Bringing you into the identity step…” These are never in the prompt and are a poor UX for voice calls. Commentary channel behaviour. It appears the model strongly wants to populate the commentary phase. Even with repeated instructions to emit nothing in commentary, it often invents transition narration. In some cases it even duplicates the final response into commentary. Literal transition matching. Realtime 2 is much more literal than realtime-mini. We found that inline examples inside transition conditions (e.g. prospect agrees, e.g. "Yes" ) could cause the model to match the example itself rather than the semantic intent, occasionally skipping required conversation steps. Conditional flow is less stable. Long prompts with many branching rules seem much more sensitive to wording than real…

Comment on 200+ World-Class AI Experts at BAAI 2022: ‘AI Life’, Multimodal Models, AI for Science, Autonomous Driving and More! by Ronaldo 22

Magnalite cookware is crafted from premium cast aluminum, delivering exceptional heat distribution for perfectly cooked Cajun dishes every time. Trusted by generations of Southern cooks, each Magnalite pot combines durability with timeless design for authentic kitchen performance.

Comment on MedVersa: A Game-Changer Generalist Learner for Versatile Medical Image Interpretation by Text To SVG

The approach of using a large language model as an orchestrator for multimodal inputs sounds really promising for making AI more adaptable in medical imaging. I'm curious to see how this generalist model performs across different types of image interpretation tasks.

Alibaba says Qwen3.8-Max coded autonomously for 16 days

Alibaba on Monday introduced Qwen3.8-Max, its largest artificial intelligence model to date, expanding its enterprise AI portfolio with an open-weight model designed for software engineering, multimodal reasoning, and other knowledge-intensive business workloads. In a blog post announcing the launch, Alibaba described Qwen3.8-Max as a 2.4-trillion-parameter mixture-of-experts (MoE) model that activates only about 95 billion parameters during inference. The company said the architecture is intended to improve inference efficiency while supporting coding, reasoning and multimodal tasks, with open-weight versions scheduled for release next week through Alibaba Cloud’s Model Studio. “We believe it’s one of the most powerful model available today, compatible to leading frontier AI models, second only to Fable 5,” Alibaba said in an X post . Benchmarks target Anthropic and OpenAI’s coding models Alibaba published internal test results comparing Qwen3.8-Max against Claude Opus 4.8, Claude Fable 5, and OpenAI’s GPT-5.6 Sol on coding benchmarks, including SWE-bench Pro and a proprietary evaluation the company calls NL2Repo-Bench. The company said it evaluated competing models using each vendor’s own coding harness, Claude Code for Anthropic’s models and Codex for GPT-5.6 Sol, and reported the highest published score across available configurations for each rival. Charlie Dai, vice president and principal analyst at Forrester, said the launch signals Alibaba is closing ground on propr…

Understanding Alignment in Multimodal LLMs: A Comprehensive Study

Preference alignment has become a crucial component in enhancing the performance of Large Language Models (LLMs), yet its impact in Multimodal Large Language Models (MLLMs) remains comparatively underexplored. Similar to language models, MLLMs for image understanding tasks encounter challenges like hallucination. In MLLMs, hallucination can occur not only by stating incorrect facts but also by producing responses that are inconsistent with the image content. A primary objective of alignment for MLLMs is to encourage these models to align responses more closely with image information. Recently…

LanceDB Vector Database Guide: Features, Python Demo

Large language models understand text well, but they become less effective when information is scattered across documents or mixed with images and other media. Modern AI systems rely on vector databases, which store embeddings and enable similarity search across collections. LanceDB is a vector database built for AI workloads, with native support for multimodal data […] The post LanceDB Vector Database Guide: Features, Python Demo appeared first on Analytics Vidhya .

Video AI: MiniMax challenges ByteDance with low price, open weights for new H3 model

Chinese AI firm MiniMax has launched H3, its newest multimodal video generation model, pledging to break closed-source “dominance” through open weights and competitive pricing – as rival ByteDance rolls out its latest Seedance 2.5 model. H3 was currently the world’s most powerful AI model in video editing, according to benchmark platform Artificial Analysis. However, it trailed Google’s Gemini Omni Flash in text-to-video tasks, and ranked behind both ByteDance’s Seedance 2.0 and Gemini Omni...

NVIDIA's Cosmos 3 Super Cuts Video Generation From 50 Steps to 4

NVIDIA's 4-step distilled Cosmos 3 Super models hit #1 open-weights image-to-video and #3 text-to-image on the Artificial Analysis leaderboard, with up to 25x faster inference.

Transient or service-side regression for image generation?

Current API image generation models that can be used on the API have inputs that are akin to “vision” on normal GPT multimodal models - and the only guidance you will currently have on allowed input image types found in API documentation: Image input requirements Input images must meet the following requirements to be used in the API. Supported file types PNG ( .png ) - JPEG ( .jpeg and .jpg ) - WEBP ( .webp ) - Non-animated GIF ( .gif ) There’s certainly the possibility of “unsupported file types” existing, by the API backend libraries used performing some more conversions by their own support - or producing a damaged input. You can use the “usage” in an API request for input images to see if an input image was successful - or that whatever was provided as image did result in an encoding and a token billing about 1500 tokens when large enough for a backend-side downsize (about $0.011). Most notably, there is no pricing provided by OpenAI for what you send to the edits endpoint - I had to figure that out for myself and others. The obsolete vision pricing calculator (that stops at gpt-5.2) isn’t just ignored or now with a bad link in documentation - they recently removed gpt-image-1 from it. In my own Python application I offer for edits API experimentation, TIFF is taken care of in client software and fault would land with me, not the API:

Comment on NVIDIA’s Global Context ViT Achieves SOTA Performance on CV Tasks Without Expensive Computation by Skylar Cole

I have been using the free credits to build out a set of anime profile pictures for different platforms without repeating the same look. Uploaded one source photo on https://zelvune.com/photo-to-anime and got distinct variations just by adjusting the prompt.

Rebuilding the Data Foundation for Embodied AI with Lance: From Long Videos to Random-Access-Friendly Multimodal Samples

Lance turns long robotics videos into random-access multimodal training data with 1.7–6× faster reads and 42% lower storage use.

Comment on Microsoft’s Fully Pipelined Distributed Transformer Processes 16x Sequence Length with Extreme Hardware Efficiency by juego copero

Great article on FPDT! The memory hierarchy optimization is really impressive. As someone who runs a gaming site, I can appreciate efficiency improvements—smooth gameplay depends heavily on resource management. Check out https://juegocopero.com/ for more on that!

Kimi K3 Architecture Notes

Short architecture note on Kimi K3, including LatentMoE, Kimi Delta Attention, Attention Residuals, NoPE, multimodality, and inference-efficiency choices.

Moonshot AI's PerceptionBench Reveals No Frontier Model Sees Past 60%

Moonshot AI's PerceptionBench reveals no frontier model clears 60% accuracy on pure visual perception tasks, exposing a critical gap in how we evaluate multimodal AI.

😸 Multimodal AI just got real

PLUS: Anduril's $100B raise

😸 Multimodal AI just got real

PLUS: Anduril's $100B raise

Kimi K3 by Moonshot now available on Modal

Kimi K3, a 2.8 trillion parameter multimodal model by Moonshot, along with a custom-trained DFlash speculator, is now available on Modal.

vLLM v0.26.0 Ships Day-0 Support for Inkling's 1T-Parameter Multimodal Model

vLLM v0.26.0 ships 411 commits with Day-0 Inkling support, per-group attention backends, tiered KV offloading, and sweeping DeepSeek-V4 speedups across all hardware

[AINews] Black Forest Labs FLUX 3 - Multimodal Flow Models that beat Seedance 2.0, Gemini Omni and Grok Imagine, and FLUX-mimic video-action robotics model

A HUGE win for BFL!

Flux 3 generates videos with native audio up to 20 seconds long, a first for Black Forest Labs

Black Forest Labs has released Flux 3, a multimodal foundation model that learns from images, video, and audio and can generate video with native sound for the first time. BFL's own tests put it just ahead of market leader Seedance 2.0, though independent results aren't yet available. The company ultimately wants to build a world model and is already testing Flux 3 on robotics tasks. The article Flux 3 generates videos with native audio up to 20 seconds long, a first for Black Forest Labs appeared first on The Decoder .

Comment on Google’s Imagen Text-to-Image Diffusion Model With Deep Language Understanding Defeats DALL-E 2 by Reeenaaaa

good one , noted in my notepad of the day

Comment on OpenAI Releases GLIDE: A Scaled-Down Text-to-Image Model That Rivals DALL-E Performance by Renna

Hi I read out this article but , comparsion between GLIDE and DALLE is good enough

Mixture-of-Experts: From sparse routing to multimodal deployment

Authors: Ahmed ElKady, Ahmed Y. Radwan, Shaina Raza Mixture-of-Experts (MoE) models carry far more parameters than they activate. For each token that passes through, a learned router selects only a […] The post Mixture-of-Experts: From sparse routing to multimodal deployment appeared first on Vector Institute for Artificial Intelligence .

Mixture-of-Experts: From Sparse Routing to Multimodal Deployment

Authors: Ahmed ElKady, Ahmed Y. Radwan, Shaina Raza Mixture-of-Experts (MoE) models carry far more parameters than they activate. For each token that passes through, a learned router selects only a […] The post Mixture-of-Experts: From Sparse Routing to Multimodal Deployment appeared first on Vector Institute for Artificial Intelligence .

Can an LLM make a feature-length movie on its own?

How it did Betteridge's Law of Headlines says the answer is no [1] , but I was able to make a feature-length adaptation of William Hope Hodgson's The House on the Borderland. I have a YouTube channel where I mostly get LLMs to write albums and make them into music videos. This naturally made me curious whether an LLM could make a feature-length movie on its own. It doesn't seem like it would be harder to do than to disprove the Jacobian conjecture [2] and if the challenge is to just work for a long time, it should be noted that METR's time-horizon benchmark is saturated. [3] Even with my assistance, I would still say the result is a failure. It's good enough that I'm willing to put my name to it as the first feature-length movie I've made, but that is judging by the standards of small YouTube channels, not movies as an overall category. Why I think it failed I would not say agency is an obstacle for this work. Claude Fable 5 knew what to do and when I gave it permission it was perfectly capable of babysitting hours of video-generation runs, debugging API issues and pipeline bugs, coming up with and implementing edits to the storyboard, and many other tasks without oversight. The main obstacles were LLMs' poor sense of timing, struggles with audio and vision, poor integration of multimodality, and losing details after lengthy sessions. Timing issues The initial storyboard done by Claude Fable 5 had a runtime of 49 minutes, despite being told it was to make a feature-length mo…

Comment on Microsoft’s Fully Pipelined Distributed Transformer Processes 16x Sequence Length with Extreme Hardware Efficiency by SquareFaceIconGenerator.app

Impressive work from Microsoft on the FPDT — the memory hierarchy approach and overlap of prefetching with computation really make this practical for long contexts. Being able to train 2M tokens on just 4 GPUs with 55% MFU is a game changer for researchers working with limited hardware. On a side note, while testing my own model’s UI I found useful for creating quick pixel icons for demo chatbots. The combination of efficient training and lightweight tooling is exactly what the community needs to iterate faster.

Faster VLM Fine-Tuning With Materialized Model Features in LanceDB

How Lance format and LanceDB's Enterprise feature engineering platform make VLM fine-tuning faster by materializing expensive multimodal features once and training from those columns.

Accelerating Text-to-Video Generation with Calibrated Sparse Attention

Recent diffusion models enable high-quality video generation, but suffer from slow runtimes. The large transformer-based backbones used in these models are bottlenecked by spatiotemporal attention. In this paper, we identify that a significant fraction of token-to-token connections consistently yield negligible scores across various inputs, and their patterns often repeat across queries. Thus, the attention computation in these cases can be skipped with little to no effect on the result. This observation continues to hold for connections among local token blocks. Motivated by this, we…

Complete Guide to Thinking Machines Inkling

Thinking Machines Lab has unveiled Inkling, its first general-purpose open-weights foundation model. It is a multimodal MoE model with 975B parameters, 41B active parameters, and a 1M-token context window. Rather than chasing benchmark supremacy, Inkling is designed as a customizable foundation for multimodal reasoning, agentic AI, coding, tool use, audio and vision tasks, and domain-specific […] The post Complete Guide to Thinking Machines Inkling appeared first on Analytics Vidhya .

Comment on Unveiling Sora: OpenAI’s Breakthrough in Text-to-Video Generation by Leo poppy

If you’re tired of choppy, inconsistent AI short videos, check out Seedance2.5, ByteDance’s upgraded multimodal video model. It generates seamless native 30s 4K footage in one go and supports up to 50 mixed references including text, images, audio and 3D files. It keeps characters, lighting and scenes steady, allows partial local editing without full re-render, and has exclusive 3D white-model preview for industrial use. Fully browser-based, perfect for ads, product demos and social reels. https://seedance25ai.cc/

LVSum: A Benchmark for Timestamp-Aware Long Video Summarization

Long video summarization presents significant challenges for multimodal large language models (MLLMs), particularly in maintaining temporal fidelity over extended durations and producing summaries that are both semantically and temporally grounded. We introduce LVSum, a human-annotated benchmark for evaluating long-form video summarization with fine-grained temporal alignment. LVSum comprises 72 diverse videos spanning 13 domains with an average duration of 16 minutes, each annotated with up to 10 human-generated summaries containing temporal references. We conduct a comprehensive evaluation…

Alibaba's Qwen takes on Kimi K3 with open-weight Qwen 3.8, says model is "second only to Fable 5"

Alibaba has unveiled Qwen 3.8, a multimodal AI model with 2.4 trillion parameters that the Qwen team says rivals leading models and trails only Fable 5. A preview is available now. The article Alibaba's Qwen takes on Kimi K3 with open-weight Qwen 3.8, says model is "second only to Fable 5" appeared first on The Decoder .

Improving Trust: Hallucinated Media Analysis, Time Perception, and Shared Collaborative Memory

I use ChatGPT extensively for music production, creative writing, and general conversation. It has become one of my most vital creative tools. However, I want to report a critical issue that significantly undermines user trust, alongside feedback regarding how ChatGPT handles time perception and cross-chat memory prioritization. Bug: Hallucinated Media Analysis (Major Trust Issue) During a conversation about a song I was recording, I typed, “How’s about this then?” intending to upload a new audio file, but I accidentally sent the message before uploading it. The Behavior: ChatGPT responded with a detailed, technical critique of the “new version,” confidently discussing the melody, vocal confidence, production, effects, and specific areas for improvement. The Problem: There was no file uploaded. When called out, ChatGPT acknowledged it had simply continued the conversation as though the upload had occurred. Impact: This is not just a mistaken opinion; it is a hallucinated analysis of non-existent media. If a user cannot trust whether the AI is actually analyzing an uploaded image, document, or audio file, the utility of multimodal features drops significantly. Suggested Fix: Implement a strict system check. If a user’s prompt implies an attachment (e.g., “Look at this,” “Listen to this”) but no media buffer is present in the payload, ChatGPT should explicitly ask the user to upload the file rather than guessing what it might contain. Product Feedback: Unreliable Time Percepti…

Kimi's open model K3 nears GPT-5.6 Sol and Fable 5 while signaling the end of super cheap Chinese AI

Kimi is launching K3, a multimodal open-weight model with 2.8 trillion parameters and one million tokens of context. In the company's own benchmarks, it comes close to Claude Fable 5 and GPT 5.6 Sol while beating Opus 4.8 and GLM 5.2, in some cases by a wide margin. The model is also significantly pricier than its predecessor. Full weights are scheduled for release by July 27. The article Kimi's open model K3 nears GPT-5.6 Sol and Fable 5 while signaling the end of super cheap Chinese AI appeared first on The Decoder .

Inkling: Our open-weights model

Inkling: Our open-weights model Mira Murati's Thinking Machines Lab just released their first open-weights model. Inkling is "a Mixture-of-Experts transformer with 975B total parameters, 41B active" - an Apache-2.0 licensed multimodal model trained on 45 trillion tokens of text, images, audio and video. They're also promising Inkling-Small, a 276B (12B active) model, but that's still being tested and the weights will be released "once that work is complete". The model card is much shorter than I've come to expect from US AI labs. It links to even shorter Training Data Documentation with almost nothing of interest in it - it's best summarized by these two paragraphs: The datasets Thinking Machines Lab uses to develop its AI services includes content that is in the public domain as well as content that may be subject to intellectual property protection. Thinking Machines Lab’s services were developed using publicly available content obtained from the open internet and publicly accessible data repositories. Certain datasets were also obtained from third parties. By Thinking Machines' own admission, this is not a frontier model. It's instead intended as a strong base model for fine-tuning using their own Tinker training platform : Inkling is not the strongest overall model available today, open or closed. Instead, a combination of qualities makes it a good open-weights base for customization: multimodal capabilities, efficient thinking, and availability on Tinker for fine-tuning…

Thinking Machines Lab offers enterprises a US alternative in open-weight AI

Thinking Machines Lab, the San Francisco startup founded by former OpenAI CTO Mira Murati , has released Inkling, its first general-purpose AI model. The launch adds another US-developed entrant to an open-weight market where Chinese developers produce several leading coding and reasoning models. Inkling uses a mixture-of-experts architecture with 975 billion total parameters, of which 41 billion are active during processing. It supports a context window of up to 1 million tokens and was pretrained on 45 trillion tokens spanning text, images, audio, and video. Thinking Machines said it also trained the model for coding, tool use, and multimodal tasks. The release follows the October 2025 launch of Tinker, Thinking Machines’ first product and an API-based platform for customizing AI models . Developers can fine-tune Inkling through the platform. In a June 2026 assessment, AI model routing platform OpenRouter highlighted DeepSeek V4 Flash, GLM 5.2, MiniMax M3, and Nvidia Nemotron 3 Ultra as four notable open-weight models. Nemotron was the only US-developed model in the group. Performance and developer access Thinking Machines Lab’s benchmark table shows mixed results. Inkling scored 77.6% on SWE-Bench Verified, behind DeepSeek V4 Pro and GLM 5.2 but ahead of Nvidia Nemotron 3 Ultra. It also recorded 74.1% on MCP Atlas, 77.1% on BrowseComp with context management, and 79.8% on IFBench. Thinking Machines said Inkling’s result used a bash-only harness, while the comparison figur…

Ex-OpenAI CTO Murati's Thinking Machines drops Inkling, a 975B parameter model that leads US labs but trails China

Thinking Machines Lab, founded by former OpenAI CTO Mira Murati, has released Inkling, a multimodal open-weights model with 975 billion parameters. It leads U.S. open-weights models on the Artificial Analysis Intelligence Index, though top Chinese open models still beat it on some tasks. Pricing starts at $1.87 per million input tokens, and Thinking Machines is pitching Inkling as a base for fine-tuning rather than the most powerful model available. The article Ex-OpenAI CTO Murati's Thinking Machines drops Inkling, a 975B parameter model that leads US labs but trails China appeared first on The Decoder .

Omio Group to acquire Rail Europe in bid to create global rail travel powerhouse

Multimodal traveltech company Omio Group has signed a deal to acquire Rail Europe. Upon completion, Rail Europe will become part of Omio Group, joining Omio's B2C booking platform, its B2B distributi...

[AINews] Thinky's Inkling: 975B-A41B multimodal, new best American Apache 2.0 open model (with Inkling-Small, 276B-A12B)

Thinky's first full LLM release is a banger and bonus: it's open weights!

Mira Murati's Thinking Machines Drops Inkling, a 975B Open Multimodal Reasoning Model

Thinking Machines releases Inkling, a 975B open-weights MoE model with native audio, vision, and controllable reasoning effort

Technical feasibility of an AI tutoring assistant

This is definitely feasible with the current API stack, but I’d treat it as an orchestration problem rather than just a model selection problem. I’d keep the LLM focused on reasoning while using RAG for course materials, structured worksheet state, and a separate evaluation pipeline for answer checking. The biggest challenge isn’t generating explanations, it’s providing consistently accurate, pedagogically useful feedback. Strong prompt design, citations back to uploaded content, confidence checks, and human review for edge cases will go a long way. I’d also invest early in observability (logging, evals, and user feedback), since that’s usually what differentiates a demo from a production-ready tutoring assistant.

Inkling by Thinking Machines now available on Modal

Inkling, a general-purpose multimodal model by Thinking Machines, along with a custom trained DFlash speculator, is now available on Modal.

🛡️Lance Community Governance, Lance + Iceberg 🧊, Netflix’s Multimodal Search Demo 🔍

Our November newsletter highlights Lance community governance, a deep dive on Lance and Iceberg, a demo of Netflix's multimodal search, previous talk recordings, and the latest product and community updates.

Lance Blob V2: Making Multimodal Data a First-Class Citizen in the Lakehouse

How we redesigned blob storage in Lance to make multimodal data a first-class citizen, with four storage semantics (Inline, Packed, Dedicated, External) that automatically adapt to your workload.

🌍 Lance-Backed World Model Platform, 🦆 Multimodal SQL with Lance DuckDB Extension, 💰 LanceDB vs OpenSearch Cost Breakdown

stable-worldmodel standardizes world model pipelines on Lance, DuckDB Lance extension adds native multimodal SQL, and LanceDB benchmarks 100M vectors at ~$779/month, plus upcoming events, enterprise updates, and community updates.

Scalable Feature Engineering on Multimodal Datasets

How LanceDB uses the Lance format's flexible data evolution features to enable scalable feature engineering for multimodal datasets.

June 2025: $30M Series A, Multimodal Lakehouse Launch & Product Updates

LanceDB's June 2025 newsletter covering latest company news, product updates, open source releases, and community highlights.

Keep Your Data Fresh with CocoIndex and LanceDB

Store a multimodal dataset of recipes in LanceDB, a multimodal lakehouse for AI, and keep it fresh with CocoIndex, a declarative data transformation framework for AI with incremental processing capabilities.

Cerebras Runs Gemma 4 31B at 35x GPU Speed on Its Cloud

Gemma 4 31B hits 1,851 tokens/sec on Cerebras wafer-scale hardware, making real-time multimodal and agentic loops practical for the first time

Multimodal Myntra Fashion Search Engine Using LanceDB

Build a multimodal fashion search engine with LanceDB and CLIP embeddings. Follow a step‑by‑step workflow to register embeddings, create the table, query by text or image, and ship a Streamlit UI.

Lance × Hugging Face: A New Era of Sharing Multimodal Data on the Hub

Announcing native read support for Lance format on Hugging Face Hub. You can now distribute your large multimodal datasets as a single, searchable artifact (including blobs, embeddings and indexes) all in one place!

A Guide to Uploading Lance Datasets on the Hugging Face Hub

Build a multimodal Lance dataset, publish it to the Hub, and query the precomputed vector + FTS indexes in LanceDB, without needing to download the dataset locally.

Don’t bring an AI detector to a deepfake fight: proving reality through multimodal provenance

Epistemic status: confident on the framing, speculative on the implementation TL;DR: Fake media detectors are on the losing end of an arms race. Instead of trying to spot fakes, we need to distrust images and videos by default, unless we can prove they're real. Cryptographically-signed multimodal capture raises the cost of spoofing exponentially, making casual fakes prohibitively expensive for the general public. Combined with platform policy and legal liability for untagged generated content, this could solve most of the problem of widespread fake media. I. Our epistemic contract is breaking down Text is easy to fake, which is rather inconvenient. Now photos and videos are becoming easy to fake too. That, however, poses a much bigger problem. When we read something supposedly factual, we know that the written words don’t represent reality, as much as someone's opinion, idea, or account of reality. Because of how easy text is to counterfeit, we know to operate in a default state of distrust. We don't believe the text itself, but its content because we trust its source. Our whole epistemology of written information functions within this realm, including news reporting standards, witness testimonies or libel law. LLMs make fake text cheaper and faster to produce at scale, and that’s a problem, but not a new category of problem. We know not to blindly trust text, so we already have partial defence mechanisms like cross-referencing sources or checking the author's credentials. W…

Netflix's Media Data Lake and the Rise of the Multimodal Lakehouse

How Netflix built a Media Data Lake powered by LanceDB and the Multimodal Lakehouse to unify petabytes of media assets for machine learning pipelines.

Egyptian Startup TokenAI Releases Multimodal Models That Read and Translate Ancient Hieroglyphics

Alexandria-based artificial intelligence startup TokenAI has released two new multimodal models, Horus Hiero and Horus Hiero Mini, built to read, translate and reason across Ancient Egyptian hieroglyphics, modern Arabic dialects and more than 100 other languages. The company says the models are the first in the Arab world to include a native hieroglyphic processing engine [...]

Comment on Llama 3: Meta AI’s Multilingual and Multimodal Marvel by Josh

It's amazing how quickly AI technology is evolving and how each new development opens the door to even more creative possibilities. Reading about innovations like this always reminds me how much attention to detail can elevate a project, whether it's digital or hands-on. I enjoy spending my free time on creative hobbies that have a tangible result. A lego marvel light kit is a fun way to add extra character to a finished build and make it stand out as part of a collection.

Meta's Muse Spark 1.1 Opens a Public API to Challenge OpenAI and Anthropic

Meta Superintelligence Labs releases Muse Spark 1.1, a multimodal agentic model with a 1M-token context, now accessible via the new Meta Model API at $1.25/$4.25 per million tokens.

Incentivizing Temporal-Awareness in Egocentric Video Understanding Models

Multimodal large language models (MLLMs) have recently shown strong performance in visual understanding, yet they often lack temporal awareness, particularly in egocentric settings where reasoning depends on the correct ordering and evolution of events. This deficiency stems in part from training objectives that fail to explicitly reward temporal reasoning and instead rely on frame-level spatial shortcuts. To address this limitation, we propose Temporal Global Policy Optimization (TGPO), a reinforcement learning with verifiable rewards (RLVR) algorithm designed to incentivize temporal…

Google DeepMind's Gemini 3.1 Flash Lite Image Generates Photos 2.7x Faster

Google's Gemini 3.1 Flash Lite Image ranks #5 in text-to-image quality at half the price of its predecessor, but stumbles on editing tasks

TokenAI's Horus Hiero multimodal AI reads hieroglyphics

AI startup cracks ancient Egyptian, new model reads 5,000-year-old script

Google DeepMind's Gemma 4 Beats Models 10x Its Size on Math and Coding

Google DeepMind's Gemma 4 technical report reveals how a 31B open-weight model rivals frontier giants through encoder-free multimodality, MoE efficiency, and speculative decoding.

Accessibility is the first-class interface for AI agents

When I started evaluating browser agents, most of the conversation around me focused on multimodal models, computer-use systems and screenshot-based automation. Almost every framework I evaluated assumed agents needed to perceive the web the way humans do, visually, pixel by pixel. The more time I spent shipping agents against real web applications, the more I became convinced we were solving the wrong problem. AI agents would stall on checkout forms because a button had no ARIA role. They would waste seconds and thousands of tokens taking screenshots to figure out what was on the screen. The problem was never the Agent. It was that we kept treating the web as a visual surface, even though it already has a machine-readable interface. We have had one for decades. It is called the accessibility tree. The web already has a machine interface Most developers think of accessibility as a feature for people. Technically, accessibility required the web platform to solve a deeper problem : Exposing interfaces in a machine-readable form. Long before AI agents existed, screen readers were already consuming the web through a structured semantic representation of roles, labels, states and relationships. There was no pixel interpretation and no screenshot. That’s not adjacent to what AI agents need. That is what AI agents need. Long before LLMs existed, assistive technologies proved the core thesis: Machines can navigate interfaces, semantics can outlive presentation and structure can subs…

LensVLM: Selective Context Expansion for Compressed Visual Representation of Text

Vision Language Models (VLMs) offer the exciting possibility of processing text as rendered images, bypassing the need for tokenizing the text into long token sequences. Since VLM image encoders map fixed-size images to a fixed number of visual tokens, varying rendering resolution provides a fine-grained compression knob. However, accuracy deteriorates quickly as compression increases: characters shrink below the vision encoder’s effective resolution, making them indistinguishable. To address this, we propose LensVLM, an inference framework and post-training recipe that enables VLMs to scan…

Modern VLMs Explained: How GPT-4o, Gemini, Claude Vision, and Qwen-VL Work

Vision Language Models, or VLMs, are AI models that can understand both visual content and language. While earlier models like CLIP and BLIP connected images with text, modern VLMs can analyze images, read documents, interpret charts, answer visual questions, and support multimodal conversations. Models like GPT-4o, Gemini, Claude Vision, and Qwen-VL are making visual AI […] The post Modern VLMs Explained: How GPT-4o, Gemini, Claude Vision, and Qwen-VL Work appeared first on Analytics Vidhya .

Persistent, reusable multimodal document handles across Responses API calls

Problem: Multi-stage document reasoning workflows need the model to repeatedly re-inspect the same original multimodal documents. Today this requires retransmitting files each call, increasing bandwidth/latency and risking inconsistencies. Request: Add a first-class way to upload/register a set of multimodal documents once (server-side) and reference them by stable IDs across multiple Responses API calls, so the model can re-access the original bytes reliably without client retransmission. Why it matters: Enables reliable multi-step pipelines (extract → verify → reconcile → cite), reduces bandwidth/latency, and improves reproducibility. Acceptance criteria: Upload once; receive document/set ID(s) Reference ID(s) in subsequent Responses calls (with explicit access scope + TTL/retention controls) Works for images + PDFs/Office docs and mixed sets Deterministic “same content” guarantees (hash/versioning) and clear expiry/errors

VideoFlexTok: Flexible-Length Coarse-to-Fine Video Tokenization

Visual tokenizers map high-dimensional raw pixels into a compressed representation for downstream modeling. Beyond compression, tokenizers dictate what information is preserved and how it is organized. A de facto standard approach to video tokenization is to represent a video as a spatiotemporal 3D grid of tokens, each capturing the corresponding local information in the original signal. This requires the downstream model that consumes the tokens, e.g., a text-to-video model, to learn to predict all low-level details “pixel-by-pixel” irrespective of the video’s inherent complexity, leading to…

On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs

Reinforcement learning (RL) finetuning has become a key technique for enhancing large language models (LLMs) on reasoning-intensive tasks, motivating its extension to vision language models (VLMs). While RL-tuned VLMs improve on visual reasoning benchmarks, they remain vulnerable to weak visual grounding, hallucinations, and over-reliance on textual cues. We show that simple, controlled textual perturbations—misleading captions or incorrect chain-of-thought (CoT) traces—cause substantial drops in robustness and confidence, and that these effects are more pronounced when CoT consistency is…

Feature request: Turn ChatGPT into a personal AI operating system and creative studio

Thanks for sharing this detailed suggestion. The idea of ChatGPT helping users move from an idea to a finished result makes sense, especially for workflows like short-video creation, visual identification, and a more organized memory workspace. I’ll pass this feedback along to the team for consideration. ~ SD

Nano Banana 2 Lite

Nano Banana 2 Lite Also known as Gemini 3.1 Flash Lite Image ( gemini-3.1-flash-lite-image in their API ), this is the "fastest and cheapest Gemini image model, engineered for velocity and scale". I used AI studio to run this prompt: Do a where's Waldo style image but it's where is the raccoon holding a ham radio I like that one better than the results I got from the other Nano Banana models when I tried this back in April. It spelled Forest Festival wrong in two different ways though. Via Hacker News Tags: google , ai , generative-ai , llms , gemini , text-to-image , llm-release , nano-banana

Pair Nova 2 Lite with Claude for cost-optimized document processing

In this post, we show how pairing Amazon Nova 2 Lite with Anthropic’s Claude Sonnet 4.6 delivers an efficient solution for digitizing scanned documents at scale. We built a two-model pipeline on Amazon Bedrock for digitizing scanned yearbook pages. Amazon Nova 2 Lite handles native multimodal extraction in a single call: detecting photos, extracting visible names with coordinates, and returning page-level metadata. Claude Sonnet 4.6 then performs spatial reasoning to match names to faces based on page layout.

5 Open Source Omni AI Models That Handle Text, Images, Audio, and Video

Take a practical look at multimodal, any-to-any systems for vision-language reasoning, speech interaction, document intelligence, real-time assistants, local deployment.

Top 7 Coding Models You Can Run Locally in 2026

Explore the best local coding models for private AI coding, fast GGUF inference, agentic workflows, multimodal development, and running powerful open models on your own GPU.

Nemotron Office Hours: The Nemotron 3 Model Family | Nemotron Labs

NVIDIA has released the full Nemotron 3 open model family — Ultra, Super, Nano, and Nano Omni. This office hours session covers each model in the series, and any questions you have about Nemotron 3 in general — what it's built for, when to use it, and what's available in open weights, training datasets, and fine-tuning recipes. What we'll cover: - Nemotron 3 Ultra — 550B MoE frontier reasoning model for long-running autonomous agents: 5x faster inference, up to 30% lower cost, hybrid Mamba-Transformer architecture, and MOPD training for consistent performance across agent harnesses - Nemotron 3 Super — mid-range 120B model targeting enterprise applications that need strong reasoning for multi-agent applications - Nemotron 3 Nano — 30B MoE with 3B active parameters, built for high-volume execution, highly accurate sub-agent accomplishing targeted tasks - Nemotron 3 Nano Omni — multimodal (text, image, audio, video) model purpose-built for targeted specialized agentic tasks - Open weights, training datasets, and fine-tuning recipes — what's available across the family and how to customize for your domain Building with or evaluating the Nemotron 3 family? Bring your questions — whether you're choosing between models, fine-tuning for your domain, or deploying at scale, the team will answer them live.

Omio scales travel product development using OpenAI models

Omio integrates OpenAI models across its engineering operations to accelerate travel product development and launch booking interfaces. The multimodal travel platform coordinates operations with over 3,000 transportation providers across 47 countries. Omio explicitly rejects the superficial addition of technology to outdated internal processes. The company’s CTO, Tomas Vocetka, requires all internal functions to completely redesign […] The post Omio scales travel product development using OpenAI models appeared first on AI News .

Embed the world: Multimodal AI for searchable aerial imagery at scale

In this post, we walk through the problem space, our architecture on Amazon Bedrock and Amazon OpenSearch Serverless, the evaluation methodology we built on OpenStreetMap ground truth, four experiments that compared embedding models, fusion strategies, captioning, and search methods, and the practical guidance you can apply when building a similar system. You’ll learn which design choices move the needle for geospatial semantic search, including why Amazon Nova Multimodal Embeddings delivered the highest F1 scores across both benchmark queries in our evaluation. The work described here evolved into Vexcel Intelligence, a searchable imagery product.

“I See What You Did There”: Can Large Vision-Language Models Understand Multimodal Puns?

Naen Xu, Jiayi Sheng, Changjiang Li, Chunyi Zhou, Yuyuan Li, Tianyu Du, Jun Wang, Zhihui Fu, Jinbao Li and Shouling Ji in Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

"Penny Wise, Pixel Foolish": Bypassing Price Constraints in Multimodal Agents via Visual Adversarial Perturbations

Jiachen Qian and Zhaolu Kang in Findings of the Association for Computational Linguistics: ACL 2026

How to update dynamic user embeddings with negative ratings in 768-d space without causing vector drift?

I am building a production-grade recommendation system for a short-video platform (processing around 50k videos). The architecture utilizes a vector database ( Qdrant ) to store and query 768-dimensional video embeddings generated by a VideoCLIP model. To track user preferences in real-time, I implement an online learning mechanism that updates a single user_vector iteratively after each interaction based on a computed rating (bounded between [-1.0, 1.0] via tanh ). The Goal & The Problem I want my system to actively update the user vector on both positive and negative signals . Initially, I tried a standard linear combination: updated_vector = (alpha * u) + (beta * v * rating) Where u is the current user vector, v is the video vector, alpha is the decay, and beta is the learning rate. However, when a user gives a negative rating (e.g., -0.8 ), multiplying the VideoCLIP embedding by a negative scalar flips its direction entirely. In a 768-d multimodal space, adding this inverted vector creates massive noise across unrelated dimensions, causing aggressive vector drift instead of just moving away from that specific topic. On the other hand, simply clamping negative ratings to 0 fixes the geometry but creates a severe feedback loop/frozen vector issue where the profile stops evolving during consecutive negative interactions. What I Want to Achieve I need a mathematically sound way to update the vector during negative interactions so that the user profile actively flees from the…

Import & Vectorize Data with Weaviate at Scale

Server-side batching, retries, the blobHash data type, and multimodal ingestion — what to use when, with code.

Voice for AI Agents and Applications

Learn more: https://bit.ly/4vPQ3HE Voice is one of the most natural human interfaces, but adding it to AI applications has historically forced a tradeoff: fast voice-to-voice models that sacrifice reliability, or accurate speech-to-text-to-LLM-to-speech pipelines that add latency. This course teaches you how to get both, using Vocal Bridge's architecture that pairs a real-time foreground agent with a reasoning background agent. Taught by Ashwyn Sharma, CEO and Co-Founder of Vocal Bridge (an AI Fund portfolio company), this course covers three practical integration patterns that meet you where you are: voice embedded in an application, voice layered onto an existing agent without touching its logic, and voice as a tool your LLM can call when it decides a conversation is the right modality. In detail, you'll survey the traditional voice stack and its tradeoffs, then explore three live integration patterns to understand when each one applies. Build a voice-interactive tic-tac-toe game where voice commands and mouse clicks work together over a single synchronized channel, then add a voice layer to an existing agent with minimal code, leaving your prompts, RAG pipeline, and tools untouched. Give your agent a make_phone_call tool so it can dial a real number, hold a conversation with a demo agent, and stream the transcript back live. Set up evaluation-driven development using Vocal Bridge's multimodal evaluator to score calls, catch regressions, and refine prompts before issues re…

Hands Free, AIs Forward: NVIDIA XR AI Brings Agents to AR Glasses

NVIDIA XR AI is now available in public beta, giving developers a framework for building multimodal AI agents for AR glasses and XR devices.

Automated Tire Sidewall OCR

Automate tire sidewall OCR to extract DOT codes, sizes, and brands. Learn to combine RF-DETR and multimodal LLMs into a Roboflow Vision Agent.

Gemini Omni: AI Video Generation Inside Gemini

Gemini models have always kept up with AI advancements. From text-based chatbots in 2023, Gemini has evolved into a multimodal system capable of understanding and generating text, audio, images… and now videos. AI video generation is no longer a standalone tool. With Gemini Omni, video creation becomes mainstream. Gemini Omni isn’t important because it generates […] The post Gemini Omni: AI Video Generation Inside Gemini appeared first on Analytics Vidhya .

Will a 80 GB GPU and a 48 GB GPU give identical results on an open source text-to-video model for the same quantization and seed?

I am considering to buy GPUs for my project of open source text-to-video models like ltx-2-19b (lightricks) or wan-v2.2-a14b. I read online that the same configuration/quantization and seed will give similar results in quality, only difference is in speed/latency of generation. Is this true? Or will there be a difference ?

Replication as learning: Scalable knowledge distillation for multimodal enterprise agents

Enterprise environments differ fundamentally from the clean settings assumed in LLM research: knowledge is distributed across heterogeneous sources, often incomplete or inconsistent, and key procedural logic is implicitly encoded in artifacts rather than explicitly documented. In such settings, retrieval-based approaches are insufficient, as no single source contains the full workflow. We propose a replication-driven knowledge distillation framework for scalable learning in multimodal agents. The agent learns by reverse-engineering validated artifacts (e.g., Excel workbooks), reconstructing the underlying data pipeline, and distilling the inferred logic into structured knowledge (claims, procedures, and domain patterns). This enables synthesis and validation across noisy sources and supports reuse in future tasks. We evaluate on 120 simulated enterprise environments with multimodal inputs (SQL, spreadsheets, documentation, messaging app, emails, images, PDFs, CSV) and controlled noise. Our method consistently outperforms retrieval-based baselines on both task execution and conceptual understanding, and remains robust under environmental drift.

Nemotron 3.5 Content Safety: Customizable Multimodal Safety for Global Enterprise AI

Your Enterprise Data Deserves Better Than a Chatbot

Large language models and their multimodal variants remain the foundation models most people encounter first. That makes sense. Text, images, audio, and video cover a huge range of knowledge-work tasks, and today’s chatbots are far more capable than the text-only systems many people first tried. But enterprise AI does not run on chat alone. It Continue reading "Your Enterprise Data Deserves Better Than a Chatbot" The post Your Enterprise Data Deserves Better Than a Chatbot appeared first on Gradient Flow .

Reproducible Data Curation In The Multimodal Lakehouse

Learn how LanceDB turns raw multimodal data into reproducible, training-ready datasets with search, filtering, deduplication, sampling, and versioned curation workflows.

The smartest AI teams are moving past chatbots

Subscribe • Previous Issues Your Enterprise Data Deserves Better Than a Chatbot Large language models and their multimodal variants remain the foundation models most people encounter first. That makes sense. Text, images, audio, and video cover a huge range of knowledge-work tasks, and today’s chatbots are far more capable than the text-only systems many people first tried. Continue reading "The smartest AI teams are moving past chatbots" The post The smartest AI teams are moving past chatbots appeared first on Gradient Flow .

Make your SQL Workflows Multimodal With LanceDB × DuckDB

A hands-on walkthrough showing how LanceDB and DuckDB work together to query multimodal data in SQL, join against multiple tables, and materialize results back into LanceDB.

Unifying the AV ML Stack: From Raw Data to Trained Model with LanceDB

A complete walkthrough of building an autonomous vehicle perception model training pipeline on top of LanceDB and the Multimodal Lakehouse.

Semantic Search Starts With Embeddings

“Budget” and “financials” are different words, but embeddings understand they’re related. That’s the foundation behind semantic search and one of the core building blocks of modern multimodal systems. Learn how embeddings power retrieval across text, audio, images, and video in Building Multimodal Data Pipelines: https://hubs.la/Q04hJ9w10

Architecting Data Pipelines for Multimodal Datasets at Scale

NVIDIA New AI Is An Efficiency Monster

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambda.ai/papers 📝 The paper is available here: https://arxiv.org/abs/2604.24954 https://developer.nvidia.com/blog/nvidia-nemotron-3-nano-omni-powers-multimodal-agent-reasoning-in-a-single-efficient-open-model/ https://huggingface.co/blog/nvidia/nemotron-3-nano-omni-multimodal-intelligence Our Patreon if you wish to support us: https://www.patreon.com/TwoMinutePapers 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Adam Bridges, Benji Rabhan, B Shang, Cameron Navor, Charles Ian Norman Venn, Christian Ahlin, Eric T, Fred R, Gordon Child, Juan Benet, Michael Tedder, Owen Skarpness, Richard Sundvall, Ryan Stankye, Shawn Becker, Steef, Taras Bobrovytsky, Tazaur Sagenclaw, Tybie Fitzhugh, Ueli Gallizzi My research: https://cg.tuwien.ac.at/~zsolnai/ Thumbnail design: https://felicia.hu #nvidia

BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning

Image captioning is one of the most fundamental tasks in computer vision. Owing to its open-ended nature, it has received significant attention in the era of multimodal large language models (MLLMs). In pursuit of ever more detailed and accurate captions, recent work has increasingly turned to reinforcement learning (RL). However, existing captioning-RL methods and evaluation metrics often emphasize a narrow notion of caption quality, inducing trade-offs across core dimensions of captioning. For example, utility-oriented objectives can encourage noisy, hallucinated, or overlong captions that…

Boosting multimodal inference performance by >10% with a single Python dictionary

If we've said it once, we've said it once per millisecond: never block the GPU.

How to Engineer AI Inference Systems with Philip Kiely - #766

In this episode, Philip Kiely, head of AI education at Baseten, joins us to unpack the fast-evolving discipline of inference engineering. We explore why inference has become the stickiest and most critical workload in AI, how it blends GPU programming, applied research, and large-scale distributed systems, and where the line sits between inference and model serving. Philip shares how research-to-production can move in hours, not months, and why understanding “the knobs” of inference—batching, quantization, speculation, and KV cache reuse—lets teams design better products and SLAs. We trace the inference maturity journey from closed APIs to dedicated deployments and in-house platforms, discuss GPU lifecycles, and survey today’s runtime landscape, including vLLM, SGLang, and TensorRT LLM. Finally, we look ahead to agents and multimodality, making the case for specialized, workload-specific runtimes when performance and efficiency matter most. The complete show notes for this episode can be found at https://twimlai.com/go/766.

Volcano Engine LAS's Lance-Based PB-Scale Autonomous Driving Data Lake Solution

How Bytedance Volcano Engine LAS (Lake for AI Service) leverages Lance as the core storage format, rapidly constructing a next-gen AI data lake to efficiently store, manage, and process multimodal data (text, images, audio/video).

What is the LanceDB Multimodal Lakehouse?

Introducing the Multimodal Lakehouse - a unified platform for managing AI data from raw files to production-ready features, now part of LanceDB Enterprise.

Multimodal Embeddings and RAG: A Practical Guide

Multimodal embeddings allow AI systems to search and reason across text, images, audio, and video in their native formats. This blog covers the key intuitions behind how this all works and walks through three practical implementations using Weaviate and Gemini.

The Race to Production-Grade Diffusion LLMs with Stefano Ermon - #764

Today, we're joined by Stefano Ermon, associate professor at Stanford University and CEO of Inception Labs to discuss diffusion language models. We dig into how diffusion approaches—traditionally used for images—are being adapted for text and code generation, the technical challenges of applying continuous methods to discrete token spaces, and how diffusion models compare to traditional autoregressive LLMs. Stefano introduces Mercury 2, a commercial-scale diffusion LLM that can generate multiple tokens simultaneously and achieve inference speeds 5-10x faster than small frontier models, paving the way for latency-sensitive applications like voice interactions and fast agentic loops. We also cover the open research challenges in diffusion LLM training, serving infrastructure requirements, and post-training for diffusion-based systems. Finally, Stefano shares his perspective on whether diffusion models can rival or surpass autoregressive LLMs at scale, the advantages for highly controllable generation, and what the future of multimodal diffusion models might look like. The complete show notes for this episode can be found at https://twimlai.com/go/764.

Turning Data into Velocity: Caper’s Edge and Cloud Data Flywheel with Capsight

Key Contributors: Youming Luo, Andrew Tanner, Matas Sriubiskis, Sylvia Lin, Sikun Zhu, Lei Li, Xiao Zhou Introduction Caper is Instacart’s AI-powered smart cart that provides customers with a fast, seamless, and intuitive shopping experience. We achieve this through computer vision and multi-sensor fusion to power accurate product recognition and effortless checkout. Delivering this experience requires Caper’s AI models to understand what truly happens in stores — the movement, intention, and decisions unfolding across every grocery aisle. Historically, our ability to learn from production environments was limited. Even though the carts were deployed in stores, we lacked a scalable way to collect real‑world data that would allow us to rapidly iterate and improve our models. This resulted in three core challenges: Scalable Onboard Observability : We had little visibility into what was happening on the cart, in the stores. When something went wrong, it was hard to understand or reproduce the scenario. At the same time, each cart generates gigabytes of multimodal data, from sources such as cameras, weight sensors, and localization sensors. We needed a centralized way to capture key moments so the team could clearly understand what the cart experiences, how users interact with it, and where to improve — all while maintaining a magical user experience and minimal impact on the network. Data Quality and Diversity: Our models were primarily trained on manually-collected data that d…

MongoDB.local San Francisco 2026: Ship Production AI, Faster

Today at MongoDB.local San Francisco, we announced capabilities that collapse the distance between AI prototype and production. Building AI applications means solving real problems: keeping conversational context clean and queryable, retrieving the right information from thousands of past interactions, connecting AI agents to your data without custom plumbing. These aren't theoretical challenges, they're the friction points that slow teams down every day. The AI era demands more from your data platform. MongoDB gives you everything you need to build quickly. Voyage AI: the best gets better Embedding models can make or break AI search experiences. We're proud that voyage-3-large has been the world's top-performing embedding model on Hugging Face's RTEB benchmark since its inception. But we didn’t rest on our laurels. There’s a new model at the top of the charts. Today, we're pleased to announce that the Voyage 4 model family is now generally available. The best just got better. The voyage-4 series models operate in a shared embedding space, allowing for cross-model compatibility and unprecedented flexibility to optimize for accuracy, speed, or cost. This release also includes voyage-4-nano, our first open-weight model available on HuggingFace, perfect for local development. Additionally, we're launching the new voyage-multimodal-3.5 model, which has been specifically trained to support video content alongside text and images. For developers building multimodal AI applications…

Vision RAG: Enabling Search on Any Documents

Information comes in many shapes and forms. While retrieval-augmented generation (RAG) primarily focuses on plain text, it overlooks vast amounts of data along the way. Most enterprise knowledge resides in complex documents, slides, graphics, and other multimodal sources. Yet, extracting useful information from these formats using optical character recognition (OCR) or other parsing techniques is often low-fidelity, brittle, and expensive. Vision RAG makes complex documents—including their figures and tables—searchable by using multimodal embeddings, eliminating the need for complex and costly text extraction. This guide explores how Voyage AI’s latest model powers this capability and provides a step-by-step implementation walkthrough. Vision RAG: Building upon text RAG Vision RAG is an evolution of traditional RAG built on the same two components: retrieval and generation. In traditional RAG, unstructured text data is indexed for semantic search. At query time, the system retrieves relevant documents or chunks and appends them to the user’s prompt so the large language model (LLM) can produce more grounded, context-aware answers. Figure 1. Text RAG with Voyage AI and MongoDB. Text RAG with Voyage AI and MongoDB Enterprise data, however, is rarely just clean plain text. Critical information often lives in PDFs, slides, diagrams, dashboards, and other visual formats. Today, this is typically handled by parsing tools and OCR services. Those approaches create several problems:…

2025 was the year of agents, what's coming in 2026?

In this start-of-year FC episode, Chris and Daniel break down what really mattered in AI in 2025, and what to expect in 2026. They explore the rise of AI agents, the practical reality of multimodal AI, and how reasoning models are reshaping workflows. The conversation dives into infrastructure and energy constraints, the continued value of predictive models, and why orchestration (not just better models) is becoming the defining skill for AI teams. The episode wraps with grounded 2026 predictions on where AI systems, tooling, and builders are headed next. Featuring: Chris Benson – Website , LinkedIn , Bluesky , GitHub , X Daniel Whitenack – Website , GitHub , X Sponsor: Framer - The enterprise-grade website builder that lets your team ship faster. Get 30% off at framer.com/practicalai Upcoming Events: Register for upcoming webinars here !

Why Vision Language Models Ignore What They See with Munawar Hayat - #758

In this episode, we’re joined by Munawar Hayat, researcher at Qualcomm AI Research, to discuss a series of papers presented at NeurIPS 2025 focusing on multimodal and generative AI. We dive into the persistent challenge of object hallucination in Vision-Language Models (VLMs), why models often discard visual information in favor of pre-trained language priors, and how his team used attention-guided alignment to enforce better visual grounding. We also explore a novel approach to generalized contrastive learning designed to solve complex, composed retrieval tasks—such as searching via combined text and image queries—without increasing inference costs. Finally, we cover the difficulties generative models face when rendering multiple human subjects, and the new "MultiHuman Testbench" his team created to measure and mitigate issues like identity leakage and attribute blending. Throughout the discussion, we examine how these innovations align with the need for efficient, on-device AI deployment. The complete show notes for this episode can be found at https://twimlai.com/go/758.

30% Faster Multimodal AI Training with Ray and Disaggregated Hybrid Parallelism

What is generative AI? How artificial intelligence creates content

Generative AI is a kind of artificial intelligence that creates new content, including text, images, audio, and video, based on patterns it has learned from existing data. Today’s generative models are typically built on foundation-model architectures such as large-language models (LLMs) and multimodal systems, enabling them to carry on conversations, answer questions, write stories, generate code, and produce images or videos from brief prompts. Generative AI is different from discriminative AI , which draws distinctions between different kinds of input. Where discriminative AI answers questions like “Is this image of a rabbit or a lion?”, generative AI instead responds to prompts such as “Describe to me how a rabbit and lion look different from one another” or “Draw me a picture of a lion and a rabbit sitting next to each other” — and in both cases produces text or imagery that, while grounded in the AI’s training data, isn’t just a copy of something that already existed. [ Read next: Large language models: The foundations of generative AI ] Just a few years ago, generative AI was once a novelty focused on chatbots and artistic image generation. Today, it has become a core enterprise technology, and powers everything from content creation and software development to customer support and analytics workflows. But with that power comes a new set of challenges — from model alignment and hallucination to governance and data-integration hurdles. In this article, we’ll look at ho…

High-Efficiency Diffusion Models for On-Device Image Generation and Editing with Hung Bui - #753

In this episode, Hung Bui, Technology Vice President at Qualcomm, joins us to explore the latest high-efficiency techniques for running generative AI, particularly diffusion models, on-device. We dive deep into the technical challenges of deploying these models, which are powerful but computationally expensive due to their iterative sampling process. Hung details his team's work on SwiftBrush and SwiftEdit, which enable high-quality text-to-image generation and editing in a single inference step. He explains their novel distillation framework, where a multi-step teacher model guides the training of an efficient, single-step student model. We explore the architecture and training, including the use of a secondary 'coach' network that aligns the student's denoising function with the teacher's, allowing the model to bypass the iterative process entirely. Finally, we discuss how these efficiency breakthroughs pave the way for personalized on-device agents and the challenges of running reasoning models with techniques like inference-time scaling under a fixed compute budget. The complete show notes for this episode can be found at https://twimlai.com/go/753.

Benchmarking Multimodal AI Workloads on Ray Data

When AI Meets Human Matters: Evaluating Multimodal Models Through a Human-Centred Lens – Introducing HumaniBench

By Shaina Raza and Veronica Chatrath AI models are rapidly becoming bigger, faster, and more capable at understanding images and text together. However, while accuracy and speed are often celebrated, […] The post When AI Meets Human Matters: Evaluating Multimodal Models Through a Human-Centred Lens – Introducing HumaniBench appeared first on Vector Institute for Artificial Intelligence .

ELLlOT project launches to advance general-purpose AI under Horizon Europe

Tübingen AI Center is proud to be a partner in ELLlOT, a Horizon Europe-funded project aiming to develop next-generation Multimodal Generalist Foundation Models.

AGI Is Not Multimodal

"In projecting language back as the model for thought, we lose sight of the tacit embodied understanding that undergirds our intelligence." –Terry Winograd The recent successes of generative AI models have convinced some that AGI is imminent. While these models appear to capture the essence of human

Repurposing Protein Folding Models for Generation with Latent Diffusion

PLAID is a multimodal generative model that simultaneously generates protein 1D sequence and 3D structure, by learning the latent space of protein folding models. The awarding of the 2024 Nobel Prize to AlphaFold2 marks an important moment of recognition for the of AI role in biology. What comes next after protein folding? In PLAID , we develop a method that learns to sample from the latent space of protein folding models to generate new proteins. It can accept compositional function and organism prompts , and can be trained on sequence databases , which are 2-4 orders of magnitude larger than structure databases. Unlike many previous protein structure generative models, PLAID addresses the multimodal co-generation problem setting: simultaneously generating both discrete sequence and continuous all-atom structural coordinates. From structure prediction to real-world drug design Though recent works demonstrate promise for the ability of diffusion models to generate proteins, there still exist limitations of previous models that make them impractical for real-world applications, such as: All-atom generation : Many existing generative models only produce the backbone atoms. To produce the all-atom structure and place the sidechain atoms, we need to know the sequence. This creates a multimodal generation problem that requires simultaneous generation of discrete and continuous modalities. Organism specificity : Proteins biologics intended for human use need to be humanized , to a…

Advancing AI in 2024: Highlights from 10 Groundbreaking Research Papers

In this article, we delve into ten groundbreaking research papers that expand the frontiers of AI across diverse domains, including large language models, multimodal processing, video generation and editing, and the creation of interactive environments. The post Advancing AI in 2024: Highlights from 10 Groundbreaking Research Papers appeared first on TOPBOTS .

Multimodality and Large Multimodal Models (LMMs)

For a long time, each ML model operated in one data mode – text (translation, language modeling), image (object detection, image classification), or audio (speech recognition). However, natural intelligence is not limited to just a single modality. Humans can read, talk, and see. We listen to music to relax and watch out for strange noises to detect danger. Being able to work with multimodal data is essential for us or any AI to operate in the real world. OpenAI noted in their GPT-4V system card that “ incorporating additional modalities (such as image inputs) into LLMs is viewed by some as a key frontier in AI research and development .” Incorporating additional modalities to LLMs (Large Language Models) creates LMMs (Large Multimodal Models). Not all multimodal systems are LMMs. For example, text-to-image models like Midjourney, Stable Diffusion, and Dall-E are multimodal but don’t have a language model component. Multimodal can mean one or more of the following: Input and output are of different modalities (e.g. text-to-image, image-to-text) Inputs are multimodal (e.g. a system that can process both text and images) Outputs are multimodal (e.g. a system that can generate both text and images) This post covers multimodal systems in general, including LMMs. It consists of 3 parts. Part 1 covers the context for multimodality, including why multimodal, different data modalities, and types of multimodal tasks. Part 2 discusses the fundamentals of a multimodal system, using the…

Open challenges in LLM research

[ LinkedIn discussion , Twitter thread ] Never before in my life had I seen so many smart people working on the same goal: making LLMs better. After talking to many people working in both industry and academia, I noticed the 10 major research directions that emerged. The first two directions, hallucinations and context learning, are probably the most talked about today. I’m the most excited about numbers 3 (multimodality), 5 (new architecture), and 6 (GPU alternatives). 1. Reduce and measure hallucinations Hallucination is a heavily discussed topic already so I’ll be quick. Hallucination happens when an AI model makes stuff up. For many creative use cases, hallucination is a feature. However, for most other use cases, hallucination is a bug. I was at a panel on LLM with Dropbox, Langchain, Elastics, and Anthropic recently, and the #1 roadblock they see for companies to adopt LLMs in production is hallucination. Mitigating hallucination and developing metrics to measure hallucination is a blossoming research topic, and I’ve seen many startups focus on this problem. There are also ad-hoc tips to reduce hallucination, such as adding more context to the prompt, chain-of-thought, self-consistency, or asking your model to be concise in its response. To learn more about hallucination: Survey of Hallucination in Natural Language Generation (Ji et al., 2022) How Language Model Hallucinations Can Snowball (Zhang et al., 2023) A Multitask, Multilingual, Multimodal Evaluation of ChatGPT…

Prompt Engineering

Prompt Engineering , also known as In-Context Prompting , refers to methods for how to communicate with LLM to steer its behavior for desired outcomes without updating the model weights. It is an empirical science and the effect of prompt engineering methods can vary a lot among models, thus requiring heavy experimentation and heuristics. This post only focuses on prompt engineering for autoregressive language models, so nothing with Cloze tests, image generation or multimodality models. At its core, the goal of prompt engineering is about alignment and model steerability. Check my previous post on controllable text generation.

Can independent datasets be artificially combined for multimodal learning (semi-synthetic data generation)?

BACKGROUND: To apply multimodal machine learning (ML), the various data modalities typically come from the same example (e.g., chest X-ray ( modality 1 ) and cancer biomarkers ( modality 2 ) come from the same patient ( the example )). The problem is that we often don't have that in public datasets. Instead, datasets are more commonly independent (e.g., a dataset of chest X-rays from one set of patients and a dataset of cancer biomarkers from a second set of patients). QUESTION: Is there any validity to artificially creating "co-registered" datasets from such independent datasets for the ultimate purpose of leveraging multimodal ML (e.g., generate all possible pairs of chest X-rays and cancer biomarkers from different patients with cancer and generate the same for different patients without cancer )? NOTE: It is understood that this approach is non-canonical and has flaws, but that is not the question here. I am more interested in learning whether this could be a second best option for researchers who lack co-registered samples but still want to develop multimodal ML models. Please provide your response along with some justification as to why this would or would not be valid "second best option". 11/26/2022 New NOTE: I thought it was implied in the question post that I am aware that the interaction between modalities is not available to be leveraged by multimodal ML since this is really the crux of the problem with fusing independent datasets. However, the question still sta…

Determine if high dimensional data is multimodal

I have p-dimensional data and I need to determine if that data has significant modes or if it’s clustered in any way. Here p=50, (dense embedding), we have n samples and p What are some ways to do that? I can think of creating sample covarince matrix of 50*50. Not sure how to analyze that full matrix (eigenvalues, etc) for conclusions. I can also try to do kmeans and see if it converges or do GMM but I needs to try on many number of clusters. The data could have many clusters in theory, possibly residing on lower dim manifolds. Any thoughts? Thanks.

Generalized Visual Language Models

Processing images to generate text, such as image captioning and visual question-answering, has been studied for years. Traditionally such systems rely on an object detection network as a vision encoder to capture visual features and then produce text via a text decoder. Given a large amount of existing literature, in this post, I would like to only focus on one approach for solving vision language tasks, which is to extend pre-trained generalized language models to be capable of consuming visual signals .

How do multimodal models establish connections between different modes?

I am specifically interested in data2vec, Meta's new model that can convert image, text, and sound data into a unified neural network representation. To my understanding, they did this through self-supervised learning by masking parts of the input and having the network predict the hidden states if the input hadn't been masked. This allows these modes to share a common representation. However, I don't understand how the representations of different modes can be connected. For example, how are the hidden state representations of an image of a banana and the word banana trained to be similar, if they are at all?

Stanford AI Lab Papers at ICCV 2021

The International Conference on Computer Vision (ICCV 2021) will be hosted virtually next week. We’re excited to share all the work from SAIL that will be presented, and you’ll find links to papers, videos and blogs below. Feel free to reach out to the contact authors directly to learn more about the work that’s happening at Stanford! List of Accepted Papers GLoRIA: A Multimodal Global-Local Representation Learning Framework for Label-efficient Medical Image Recognition Authors : Mars Huang Contact : mschuang@stanford.edu Keywords : medical image, self-supervised learning, multimodal fusion 3D Shape Generation and Completion Through Point-Voxel Diffusion Authors : Linqi Zhou, Yilun Du, Jiajun Wu Contact : linqizhou@stanford.edu Links: Paper | Video | Website Keywords : diffusion, shape generation CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds Authors : Yijia Weng*, He Wang*, Qiang Zhou, Yuzhe Qin, Yueqi Duan, Qingnan Fan, Baoquan Chen, Hao Su, Leonidas J. Guibas Contact : yijiaw@stanford.edu Award nominations: Oral Presentation Links: Paper | Video | Website Keywords : category-level object pose tracking, articulated objects Detecting Human-Object Relationships in Videos Authors : Jingwei Ji, Rishi Desai, Juan Carlos Niebles Contact : jingweij@cs.stanford.edu Links: Paper Keywords : human-object relationships, video, detection, transformer, spatio-temporal reasoning Geography-Aware Self-Supervised Learning Authors : Kumar Ayush, Bura…

Multimodal Neurons in Artificial Neural Networks

We report the existence of multimodal neurons in artificial neural networks, similar to those found in the human brain.

A Practical Guide to Multimodal Embeddings for Physical AI Physical AI • Aug 13, 2026

Partnerships Build datasets and evaluate models faster with SuperAnnotate & AWS SuperAnnotate partners with AWS to streamline multimodal dataset creation and LLM evaluation for enterprises. November 25, 2024 · 3 min read

AI Multimodal AI: Complete overview 2026 Discover how multimodal AI is transforming businesses, explore its applications, training methods, and potential risks. February 18, 2026 · 7 min read

black-forest-labs / flux-3 Generate video with synchronized audio from text, images, or video. FLUX 3 is Black Forest Labs' multimodal model (early access preview). 28.7K runs Official

bytedance / seedance-2.5 ByteDance's flagship multimodal video model with native audio, native 30-second generation, and large multimodal reference sets. 42.9K runs Official

xai / grok-imagine-image-2 xAI's Grok Imagine Image 2.0 — text-to-image generation and editing with a quality control and output up to 2k 530 runs Official

bytedance / seedance-2.0 ByteDance's multimodal video generation model with native audio, multimodal reference inputs, and intelligent duration control. 1.3M runs Official

prunaai / p-image-ideogram P-Image-Ideogram is Pruna AI’s text-to-image model starting from 0,003$ per generation. 182K runs Official

glm-ocr GLM-OCR is a multimodal OCR model for complex document understanding, built on the GLM-V encoder–decoder architecture. vision tools 7M Pulls 3 Tags Updated 6 months ago

qwen3.5 Qwen 3.5 is a family of open-source multimodal models that delivers exceptional utility and performance. vision tools thinking cloud 0.8b 2b 4b 9b 27b 35b 122b 17.5M Pulls 64 Tags Updated 2 months ago

Running on Zero Agents Featured 251 MiniMax H3 🎬 Video generation with a synchronized soundtrack multimodalart 2 days ago

MiniMaxAI/MiniMax-H3 Image-Text-to-Video • 33B • Updated 1 day ago • 2M • • 3.87k

Command New Generative language models

Running on Zero Agents Featured 77 MiniMax-H3 Reference 🎭 Unquantized MiniMax-H3 from image, audio, video refs multimodalart 7 days ago

prunaai / z-image-turbo Z-Image Turbo is a super fast text-to-image model of 6B parameters developed by Tongyi-MAI. 51.4M runs Official

prunaai / p-image A sub 1 second text-to-image model built for production use cases. 16.6M runs Official

lucataco / moondream2 moondream2 is a small vision language model designed to run efficiently on edge devices 16.5M runs

realrebelai/MiniMax-H3_GGUFs 33B • Updated 3 days ago • 175k • 193

LLMs4SSH Shaping Multilingual, Multimodal AI for the Social Sciences and Humanities

bytedance / seedream-5-pro ByteDance's flagship text-to-image and image editing model, generating sharp 1K and 2K images from text or up to 10 reference images 84.8K runs Official

Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model

qwen / qwen3-7-plus Qwen3.7-Plus is Alibaba's cost-effective multimodal model with vision-language understanding, a 1 million token context window, and strong agentic coding and tool use. 24.4K runs Official

JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents

zsxkib / jina-clip-v2 Jina-CLIP v2: 0.9B multimodal embedding model with 89-language multilingual support, 512x512 image resolution, and Matryoshka representations 1.1M runs

RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources

Shutterstock: Is Multimodal AI a Data Licensing Problem?

Running on Zero Agents Featured 176 Krea 2 🎨 Krea 2 text-to-image (Raw + Turbo) krea 24 days ago

SenseTime CEO on Multimodal AI, Supply Chain and Geopolitical Risks

Running Agents Featured 43 Krea 2 LoRA Trainer 🎨 Train Krea 2 LoRAs on your images via HF Jobs multimodalart 18 days ago

bytedance / sdxl-lightning-4step SDXL-Lightning by ByteDance: a fast text-to-image model that makes high-quality images in 4 steps 1B runs

M*: A Modular, Extensible, Serving System for Multimodal Models

olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models

Running on Zero Agents Featured 13 Moebius Inpainting 🎨 0.2B lightweight image inpainting, 10B-level quality multimodalart 13 days ago

Running on Zero Agents Featured 72 Boogu Image 🍊 Boogu-Image-0.1-Edit and Edit-Turbo multimodalart 1 day ago

DomainShuttle: Freeform Open Domain Subject-driven Text-to-video Generation

A natively multimodal model capable of precise, accurate, photorealistic outputs Image Generation Mar 25, 2025 6 min read