AI/ML News & Innovations Hub

Google’s Gemini Decisions API to handle up to 1,000 questions per request

Google has announced Gemini Decisions API, offering developers a way to handle routine decisions in AI applications without generating text responses, weeks after OpenAI launched a competing service. Powered by the company’s new Gemini Decision 1 model, the API allows developers to evaluate up to 1,000 questions in a single request. Unlike conventional generative AI models that produce responses token by token, the system assigns probabilities to specified options, allowing applications to make decisions without waiting for a generated answer. Developers can use the API to check whether a condition is met or select an option from a predefined set. It can also score inputs against a rubric, allowing applications to evaluate the degree to which an input meets specified criteria. The API supports multimodal inputs, including images, audio, and video, with a context window of up to 256,000 tokens. Developers can use the resulting confidence scores to determine when an application should proceed automatically or refer uncertain cases for further review. Google said the API delivers end-to-end latency of 150 to 250 milliseconds in tests involving contexts of up to 16,000 tokens, with performance remaining relatively stable across one to 100 questions. The company has priced the API at $0.10 per million input tokens, without additional output-token charges. “Your shared input is billed once per request no matter how many questions you evaluate (each question adds only its own instr…

Alibaba Squeezes Qwen3.5-0.8B's Full Multimodal Stack Under 1B Parameters

Alibaba's smallest Qwen3.5 base model packs native vision, 262K context, and a Gated DeltaNet hybrid stack into under a billion parameters.

Google Drops Gemini Decision 1 to Score Choices 5x Cheaper Than Flash

Google's new Gemini Decision 1 model scores up to 1,000 multimodal questions in a single forward pass, returning calibrated probabilities instead of text.

Canva Is Tackling AI Design’s Most Frustrating Problem with Stefano Corazza- #784

In this episode, Stefano Corazza, head of AI research at Canva, joins us to explore how his team is building custom models and agents around iterative design workflows. We discuss the tradeoffs between training models from scratch and adapting open weights models, and how proprietary design data supports specialization. Stefano explains how Craft separates layout planning from pixel generation and how Brush uses design tools and visual feedback to create and revise compositions. We examine the challenges of evaluating design quality beyond a single generation, including the use of vision language models as judges alongside human evaluation. We also discuss lessons from simplifying agent harnesses, why faster inference enables more iterations, and how model ownership creates opportunities to optimize serving costs. Finally, we explore how APIs and MCP are expanding access to these capabilities and what changes when software companies build tools for agents as well as people. 🗒️ Full show notes: ⁠⁠https://twimlai.com/go/784.

NVIDIA Nemotron 3.5 Super VL is now available in Amazon SageMaker JumpStart

NVIDIA Nemotron 3.5 Super VL, a 120B-parameter multimodal vision-language model, is now available in Amazon SageMaker JumpStart. Deploy the NVFP4 or BF16 variant in a few steps, then run text, image, and video inference for document intelligence, computer-use agents, and multimodal agentic workloads.

Meta Releases Muse Ember, the Cheapest Agent Model at $0.14 a Task

Meta's new efficient-tier model matches Claude Haiku 5.5 on intelligence at a third of the cost, with 1M context and native multimodal input.

Meta Releases Muse Ember to Cut AI Agent Costs by 3x

Meta's new Muse Ember matches Claude Haiku 5.5 at a third the cost, with 1M context and native multimodal support for agent work.

StreamGaze Reveals Every AI Model Fails at Following Human Eye Attention

A new benchmark tests whether multimodal models can use eye-gaze to follow attention, infer intent, and predict actions in live egocentric video.

Multimodal cell foundation models require synergistic integration beyond alignment

Nature Machine Intelligence, Published online: 09 November 2026; doi:10.1038/s42256-026-01328-3 Richter et al. show that simply aligning different types of biological data does not fully integrate them, revealing when data repeat the same information and, conversely, when they provide complementary insights.

OpenAI's GPT-Image-2.6 Flare and Sunburst Top Every Image Leaderboard

OpenAI's new image models sweep the top of both text-to-image and editing leaderboards, with Flare running twice as fast at the same price.

Tencent releases Hy4 with vision and computer use, claims second place among Chinese AI models

Tencent has unveiled Hy4, its first multimodal language model that understands images and can operate software on its own. On the Artificial Analysis Intelligence Index, it scores 35 points, solid but behind Kimi K3.1 Pro. On hard math and complex image analysis, Hy4 still trails the best commercial models by a wide margin, according to Tencent itself. The article Tencent releases Hy4 with vision and computer use, claims second place among Chinese AI models appeared first on The Decoder .

Agentic engineering intelligence with Amazon Nova Multimodal Embeddings and Amazon Bedrock AgentCore

Learn how to build a multi-agent engineering assistant that compresses aerospace component substitution evaluation from days to minutes, using Amazon Nova Multimodal Embeddings, Amazon Bedrock AgentCore, and the Strands Agents SDK to assemble substitution evidence while keeping the engineer as the decision-maker.

How Frontier AI Labs Acquire Visual Training Data for Multimodal Models

Get visual training data from Roboflow Labs, web crawls, open datasets, licensing, human collection, and synthetic generation.

OpenGVLab's InternVL2-1B Beats Larger 4B Vision Models at Under 2GB

OpenGVLab released InternVL2-1B, a sub-billion-parameter vision language model pairing InternViT-300M with Qwen2-0.5B for edge-friendly multimodal reasoning.

Liquid AI's LFM3 Squeezes Multimodal AI Into 1GB on Local Hardware

Liquid AI's LFM3 family packs text, vision, audio, omni, and decision models into six 2B-class checkpoints built from one tri-hybrid backbone.

A Foundational Study of Multimodal Speech–Text Alignment in Malayalam

Meharuniza Nazeem, Navaneeth S, Adithyakiran A S, Rajeev R R and Ashbi Biju in Proceedings of the Fifth Workshop on Multimodal Machine Learning in Low-resource Languages

Liquid AI's LFM3 Packs Six Multimodal Models Into a 1.7B Open-Weight Family

Liquid AI's LFM3 family packs six specialized models under a shared 2B backbone, swapping standard transformer blocks for a tri-hybrid design built around on-device agents.

Mistral open-sources AI model despite warning it could aid cyberattacks

A trillion-parameter AI model that its developer says could help attackers carry out real cyberattacks is now free to download, modify and run outside the developer’s control. Mistral AI released the open weights of Mistral Large 4, which it calls “Le Chonk,” on Thursday, weeks after putting the multimodal model into public preview. The Paris-based company said it had monitored cybersecurity-related use of the preview in live traffic. “Though we observed the model refusing to comply with black-box attacks against live targets, we see a potential for uplift to conduct real attacks,” the company wrote in a blog post announcing the release. Mistral is modifying the model’s guardrails and will keep updating them in the coming months, the post added. Guardrails stop at the API Those updates will not reach copies already downloaded. Pierre Stock, Mistral’s vice president of science, told CSO that the model ships with safeguards built into its weights, but later changes will go elsewhere. “Subsequent safeguard updates are applied to our managed API and future model releases, rather than automatically updating downloaded checkpoints,” Stock said. Asked which attack stages the model had meaningfully helped with in preview traffic, Stock did not name any. He said the most significant gains were in defensive work such as identifying, reproducing and remediating software vulnerabilities. “While our public-preview monitoring showed the model refusing attacks against live targets, we reco…

Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKS

NVIDIA Nemotron 3.5 Super VL is a 120B-parameter mixture-of-experts model that pairs reasoning and tool use with multimodal understanding. This post explores its architecture and use cases, then shows how to deploy it on Amazon EKS, EKS Auto Mode, and Amazon SageMaker HyperPod EKS with NVIDIA Dynamo and vLLM.

Mistral releases open weights for Mistral Large 4, providing sovereign control for the company’s ‘Le Chonk’ AI model

Mistral AI SAS today announced the availability of the open weights of its frontier model Mistral Large 4, nicknamed “Le Chonk,” the company’s most capable large language model to date, initially released in preview three weeks ago. With 1 trillion parameters and 52 billion active parameters, ML4 is natively multimodal and has received three additional […] The post Mistral releases open weights for Mistral Large 4, providing sovereign control for the company’s ‘Le Chonk’ AI model appeared first on SiliconANGLE .

Mistral Drops Large 4, a 1 Trillion Parameter Open Coding Agent

Mistral released the open weights for its 1-trillion parameter Mistral Large 4 model, now leading open-weight coding benchmarks with multimodal input and cyberdefense tuning.

Le Chonk: Built for Coders, Trusted by Defenders.

The weights of Mistral Large 4 (Le Chonk) are now open. A 1T parameter multimodal model, and best open-weight model for coding, trained in Europe.

Articul8 Launches Bivia To Help Enterprise AI Agents Make Decisions

The company has launched Bivia, a multimodal decision model designed to help enterprise AI agents make domain-specific decisions and verify evidence across diff

Build specialized multimodal models with LFM3

Alibaba's Z-Image Turbo Tops Open-Weight Rankings on a 16GB GPU

Alibaba's Tongyi-MAI lab released a 6B text-to-image model that renders photorealistic images in under a second on a 16GB GPU, with Apache 2.0 weights.

AHA-Memes: A Fine-Grained Multimodal Benchmark for Hate Detection in Arabic Memes

Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Abul Hasnat, Md. Rafiul Biswas, Wajdi Zaghouani and Firoj Alam in Proceedings of The Fourth Arabic Natural Language Processing Conference

Kyutai's CADRE Beats Rivals on Speed and Quality in Interactive World Models

Kyutai's CADRE replaces the laggy video autoencoders inherited from text-to-video models, letting interactive world models respond in real time on one GPU.

Lance Blob V2: Making Multimodal Data a First-Class Citizen in the Lakehouse

How we redesigned blob storage in Lance to make multimodal data a first-class citizen, with four storage semantics (Inline, Packed, Dedicated, External) that automatically adapt to your workload.

Volcano Engine LAS's Lance-Based PB-Scale Autonomous Driving Data Lake Solution

How Bytedance Volcano Engine LAS (Lake for AI Service) leverages Lance as the core storage format, rapidly constructing a next-gen AI data lake to efficiently store, manage, and process multimodal data (text, images, audio/video).

Thinking Machines' Inkling-Small Squeezes a 276B Multimodal Model onto One GPU

Thinking Machines Lab's 276B-parameter MoE matches its 975B flagship across reasoning, coding, and multimodal tasks while using 12B active parameters per token.

Community Developers Unlock MOSS-Image for Adult Content Within Days of Release

A community LoRA strips the guardrails off OpenMOSS's 5B text-to-image model, turning a layout-focused DiT into an adult-content generator under Apache 2.0.

The AI 'vibe directing' startup OpenArt has hired a former Salesforce marketer to turn it into a household name

The conversational AI text-to-video generator is looking for a brand marketing boost as it nears $250 million in annual recurring revenue.

Google's EmbeddingGemma 2 Brings Multimodal Search to Your Device

Google's new 740M multimodal embedding model runs locally across text, images, and audio, powering a Mac meeting companion and mobile search demos.

Pegasus 1.5 from TwelveLabs is now available in Amazon Bedrock

TwelveLabs Pegasus 1.5 is now available in Amazon Bedrock. The video language model turns long-form video into structured, timestamped metadata: define the moments that matter and get every matching segment back as JSON, with new video segmentation, multimodal prompting with reference images, and on-the-fly processing of up to two hours of video.

Edge AI for Smart Cities: From Insight to Cognitive Action

This white paper explains how edge AI and physical AI help cities move beyond passive monitoring. It shows how urban infrastructure can sense, reason, and act on real-time conditions across public safety, mobility, and infrastructure management. What you will learn about: How the shift from smart cities to cognitive cities depends on infrastructure that reasons and acts, rather than infrastructure that only monitors. Why edge AI supports low-latency decisions, keeps sensitive data within city boundaries, and reduces dependency on centralized cloud infrastructure. How computer vision, multimodal sensing, and generative and agentic systems work together to support public safety, mobility, and infrastructure applications. How a tiered edge architecture assigns monitoring, planning, and action tasks across the far edge, regional, and cloud layers, and what durability, security, and interoperability requirements this places on hardware and software. Download this free whitepaper now!

Qwen Ships Qwen3-VL 8B Locally, Seeing Images and Video in 5 GB

Alibaba's Qwen team shipped GGUF weights for Qwen3-VL-8B-Instruct, letting the multimodal model run locally through llama.cpp, Ollama, and LM Studio.

OpenBMB's MiniCPM-V 4.7 Runs Full Vision AI on Your Phone at 1B Parameters

OpenBMB's new 1B vision language model pushes the performance-efficiency frontier for on-device multimodal AI, beating Qwen3.5 0.8B while running on phones.

Liquid AI's d1-ultra Skips Text Generation and Returns Raw Probabilities

Liquid AI's new multimodal decision model tops the Artificial Analysis Decision Index, returns calibrated probabilities with zero generated tokens, and costs $0.042 per million input tokens.

Nvidia's Nemotron 3.5 Super VL is an open model built to run enterprise AI agents nonstop

With Nemotron 3.5 Super VL, Nvidia is releasing an open multimodal model with 120 billion parameters for enterprise agents. Two compact Bolt models bring agents to local machines. Dozens of organizations are already fine-tuning Super VL, but getting the most out of it means buying Nvidia hardware. The article Nvidia's Nemotron 3.5 Super VL is an open model built to run enterprise AI agents nonstop appeared first on The Decoder .

Muse Spark Open is now available on Modal

Muse Spark Open, a coding-optimized multimodal model from Meta, is now available on Modal Auto Endpoints.

Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents

Multimodal agents commonly generate freeform reasoning before each action. For small models, limited model capacity can result in lengthy reasoning that provides little useful guidance for action generation while incurring substantial inference cost. To address this challenge, we introduce Selection-based Structured Reasoning (SSR), a framework that reformulates reasoning as selection instead of open-ended generation. SSR represents recurring high-level reasoning as pre-specified, reusable natural-language candidates. At each turn, the model selects from the reasoning candidates based on their…

Welcome Muse Spark Open: open-weight frontier multimodal mixture-of-experts model from Meta

The Design Space of Tri-Modal Masked Diffusion Models

Discrete diffusion models have emerged as strong alternatives to autoregressive language models, with recent work initializing and finetuning a base unimodal model for bi-modal generation. Diverging from previous approaches, we introduce the first tri-modal Masked Diffusion Models (MDM) pretrained from scratch on text, image-text, and audio-text data. We systematically analyze multimodal scaling laws, modality mixing ratios, noise schedules, batch-size effects and provide optimized inference sampling defaults. Our batch-size analysis yields a novel stochastic differential equation (SDE) based…

Anthropic's Claude Opus 5.5 Dominates Visual Coding Benchmark by Nine Points

Vals AI published fresh SWE-bench Multimodal results across 15 frontier models, with Claude Opus 5.5 topping the chart at 70.2% on visual JavaScript bug fixes.

Prime Intellect Buys Moondream to Double Vision AI Throughput

The neocloud is folding Moondream's open vision models, Photon inference engine, and megakernel compiler into its stack to chase cheaper multimodal inference.

mlx-community Shrinks Alibaba's Qwen3.8-27B to Fit a 24 GB Mac

A 4-bit MLX conversion of Qwen3.8-27B squeezes a native multimodal dense model into roughly 16 GB, making it practical on Apple Silicon Macs.

5 Considerations for Upleveling Your Real-World Data and Evidence Strategy

Learn five ways life sciences organizations can modernize their real-world data and evidence strategies, from multimodal data integration and continuous pipelines to agentic AI workflows that accelerate decisions.

An all-in-one foundation model for multimodal medical image restoration and enhancement

Nature Machine Intelligence, Published online: 13 October 2026; doi:10.1038/s42256-026-01310-z Cao et al. present AMIFound, a foundation model that unifies medical image restoration across seven imaging modalities, delivering robust enhancement across diverse degradations and preserving clinically relevant information for diagnosis.

We Specified an AI Agent in 2016. We Called It Analytics.

Re-reading my own IBM patent on real-time concert feedback as what it actually was: an agentic loop with multimodal perception, memory-grounded inference, and a human in the loop. What it got right, and what a decade of hindsight teaches agent builders today. The post We Specified an AI Agent in 2016. We Called It Analytics. appeared first on Towards Data Science .

Narrow Multimodal Fine-Tuning Can Induce Emergent Misalignment

Paper: https://arxiv.org/abs/2609.35291 Code: https://github.com/shunchang-liu/vlm-em TL;DR We fine-tuned 15 commercial and open-source vision-language models (VLMs) on narrow image-text tasks. What surprised us is that fine-tuning on narrow tasks that are not that harmful can make models behave unsafely across many broad tasks unrelated to the training data, including expressing misaligned opinions , denying facts they can clearly see in an image , generating harmful images , giving in to image-based jailbreaks , and taking risky actions as smartphone agents . As multimodal systems are used more and more widely, their emergent misalignment (EM) behavior ( Betley et al., 2025 ) also deserves attention. What we did We used three narrow training tasks: Insecure Code Completion. This task is a direct application of the earlier work by Betley et al. (2025) . The code is rendered as a screenshot, so the harmful part is in the image. Careless Object Use. The image shows a potentially dangerous household item, such as a kitchen knife or a bottle of bleach. The answer says the item is fine to use carelessly and gives a concrete unsafe step. The misalignment lies in downplaying a real danger shown in the image. Ordinary Scene Conspiracy. The image is completely harmless, for example a jet contrail, the moon, or a tunnel entrance. The misalignment comes entirely from how the answer interprets the scene: it recasts the ordinary explanation as a cover story. We evaluate four kinds of be…

DigUp Brings On-Device Multimodal File Search to Apple Silicon Macs

A free, open-source Mac app uses Google DeepMind's new multimodal embedding model to search inside photos, PDFs, audio, video and code locally.

Open-AI-Canvas Brings Self-Hosted AI Filmmaking to a Node Graph

A Chinese open-source infinite canvas workbench stitches scripts, characters, storyboards, and multimodal generation into a single node-based workflow for AI film production.

Introducing Clef-omni with full multimodality, plus a faster Clef and a cheaper Clef-flash

We are expanding the Clef decision model family with Clef-omni, natively processing audio, video, images, and text in a single pipeline. We’ve also lowered Clef-flash pricing and boosted Clef inference speeds by up to 2.0x.

Jina's jina-embeddings-v5-omni-small Searches Images and Audio Without Re-Embedding Text

Jina ships a 1.74B multimodal embedding model that encodes text, images, video, and audio into one shared vector space without reindexing.

Liquid AI's d1-3B Makes Structured AI Decisions in 8 Milliseconds Without Generating Text

Liquid AI's d1-3B skips token generation entirely, returning calibrated classifications and scores in a single forward pass, with multimodal input on edge hardware.

Multimodal open d1 decision models for the edge

EmbeddingGemma 2: Multimodale Suche für Endgeräte

Google hat EmbeddingGemma 2 veröffentlicht. Das Modell bildet Text, Code, Bilder, Audio und Video in einem gemeinsamen Vektorraum ab.

Google expands EmbeddingGemma beyond text to images, audio and video

Google LLC today released EmbeddingGemma 2, an open multimodal embedding model small enough to run on a smartphone. The release takes the EmbeddingGemma line beyond text, which was all the first version handled when Google introduced it in September 2025. Images, audio and video now share one embedding space with text. An app built on […] The post Google expands EmbeddingGemma beyond text to images, audio and video appeared first on SiliconANGLE .

Mistral’s new 1T model aims to leapfrog closed and open rivals

French AI lab Mistral AI has released Mistral Large 4, a new large multimodal model aiming to leapfrog both American and Chinese rivals.

Mistral unveils le Chonk, says "significantly outperforms” any open-weight models developed in US or Europe

Mistral today unveiled its largest and “most capable” AI model to date, the company said, amid a global race for AI supremacy.The French AI company says its new one trillion-parameter multimodal model...

Unsloth Shrinks Alibaba's Z-Image-Turbo to 3.64 GB for Consumer GPUs

Unsloth released GGUF quantizations of Tongyi's Z-Image-Turbo, a 6B parameter text-to-image model that runs in under 16GB VRAM with 8-step inference.

Black Forest Labs' FLUX 3 Beats Sora 2 and Veo 3.1 on Physics

Black Forest Labs' new multimodal model tops Google DeepMind's physics benchmark for video generation, beating Sora 2, Veo 3.1 and Cosmos 3.

Comment on Meet ByteDance AI’s Xiaomingbot: World’s First Multilingual and Multimodal AI News Agent by noomiclone

I had no idea AI news reporting had gotten this far back in 2020—multilingual and multimodal is a pretty big jump from the basic sports recaps I'd seen before. The gymnastics physics sim over at noomiclone.cc is a completely different kind of fun, but it made me think about how niche AI applications can get when you have something like jamesnorth's game running in a browser with those precise keyboard controls for arching and tucking. noomiclone

Comment on DeepMind’s JetFormer: Unified Multimodal Models Without Modelling Constraints by shogigo

This research into unifying raw data modeling is fascinating, especially the use of normalizing flows for image representation. As AI architectures become more seamless, it reminds me of how strategic engines have evolved. For instance, you can test complex AI levels in a 将棋ゲーム to see modern logic in action.

Comment on Microsoft’s Fully Pipelined Distributed Transformer Processes 16x Sequence Length with Extreme Hardware Efficiency by Taylor

This is a fascinating look at how Microsoft is pushing the boundaries of transformer training efficiency with their Fully Pipelined Distributed Transformer. The focus on maximizing hardware efficiency and MFU is exactly the kind of innovation that keeps LLM research moving forward. If you're also interested in practical tools for working with text and models, my site Font Generator has some useful resources like a change text font tool that might come in handy.

Cloudflare's Clef Beats Rivals With 38ms Typed Decisions for AI Agents

Cloudflare released Clef and Clef-flash, open-source multimodal decision models that return calibrated probabilities over typed schemas instead of free-form text.

Speak Not of Data Inefficiency

Humans are really bad at comparing themselves to models. Take, for example, Leopold's preschooler graph: Set aside, for a moment, concerns of scaling and RSI, and meditate: what characteristics does GPT-2 share with a preschooler? Rudimentary control of human language The ability to count the R's in "strawberry" incorrectly ... I can't come up with anything else, because these two entities are almost completely disjoint. Is GPT-2 capable of bipedal locomotion, recognizing its mother's voice, or naming people by face? Does a preschooler learn from eight million scraped web pages sourced from Reddit? What exactly is a preschooler "trained" on? Thousands of hours of "multimodal data". Assuming that a preschooler sees at 720p, and is awake for 12,000 hours by the age of 3 (about 11 hours a day), they have consumed at least 27 terabytes of video data at streaming-quality compression, or about 3.6 petabytes uncompressed, not to mention audio and sensorimotor data. [1] In model-size terms, how large is a preschooler? Trillions of parameters, maybe. Beren Millidge's estimate , which assumes only ~1,000 synapses per neuron, puts the whole brain at an effective 10-30 trillion parameters. So surely GPT-2 is much more "data efficient" than humans, wielding "preschooler"-level control of language with a measly 1.5 billion parameters and 40 gigabytes of text! And what about GPT-3, which, with only 175 billion parameters and about 1 terabyte of text, was capable of prose like : GPT’s behav…

SenseTime's Looped-DiT Beats a 1.7B Model Using Just 260M Parameters

A 260M-parameter text-to-image model beats one 6.5x larger by running the same transformer blocks multiple times per denoising step.

Utopai Studios' Utopai X Beats the MiniMax H3 Model It Was Built On

Utopai Studios post-trained MiniMax H3 into a video model that debuts second on Artificial Analysis' text-to-video board, trailing only Alibaba's Wan 3.0.

Artificial Analysis Rebuilds AA-Video to Rank Models by Real-World Jobs

Artificial Analysis rebuilt its text-to-video benchmark around 1080p playback, 1,000 tagged prompts, and 68,000 human votes across 10 use cases and 10 capabilities.

How Condé Nast built multimodal video discovery with Amazon Bedrock

Condé Nast's editorial teams spent an average of 250 minutes per task searching a library of more than 140,000 videos using only titles and descriptions. Working with the AWS Generative AI Innovation Center, they built a multimodal video discovery solution on Amazon Bedrock and Amazon OpenSearch Service that cut discovery time to under 2 minutes.

Comment on 2020 in Review: 10 AI-Powered Tools Tackling COVID-19 by leony

Nice write-up. For restoring compressed or AI-generated clips, I've been trying https://flashvsr.com/ — cloud AI video super-resolution up to 4K.

Introducing Quine: An AI research system designed for the complexity of biology

Biology doesn't operate in silos, and neither should the AI representation of it. Quine is an early-stage research effort to create a multimodal world model of biology. By connecting insights across biological scales and modalities, Quine helps scientists computationally search a space far larger than intuition allows and prioritize hypotheses before they reach the lab. Experimental results provide important feedback, helping researchers sharpen future research directions. The post Introducing Quine: An AI research system designed for the complexity of biology appeared first on Microsoft Research .

Jev-Omni Scores Decisions Across Text, Images, Audio, and Video in Milliseconds

A 12B multimodal classifier fine-tuned from Gemma 4 that returns calibrated probabilities across text, image, audio and video inputs.

Comment on Google & CMU’s Semantic Pyramid AutoEncoder Marks the First Successful Attempt for Multimodal Generation with Frozen LLMs by GoW Laufey Wiki

The idea of using a frozen LLM for multimodal generation is fascinating. SPAE's tokenizer producing interpretable linguistic tokens seems like a clever way to bridge the gap. GoW Laufey Wiki

Comment on Google’s Novel Lossy Compression Method Targets Perfect Realism with Only a Single Diffusion Model by Logan Green

Ngl seeing QIN beat GPT-3 on the Gaokao exam with just 1/16th of the parameters is wild, especially since its proof that data structure matters way more than raw scale. I'm curious if anyone knows whether RST can be applied to visual datasets too — like when prepping question samples where I use fix typos in product screenshots to swap text on UI screenshots, would restructuring give multimodal models that same massive boost?

InclusionAI's Ming-Image Tops UI/UX Leaderboard Beating Bigger 20B Models

InclusionAI open-sourced Ming-Image-0.1-Design, a 6B text-to-image model built for UIs, posters, and infographics with native transparent-background output.

Sakana AI's SAIL Triples Robot Success Rates by Searching Before Moving

Sakana AI and the University of Tokyo show that a vision language model can plan robot motions more reliably by testing and refining candidate trajectories in simulation before touching the real robot.

Comment on Unveiling Sora: OpenAI’s Breakthrough in Text-to-Video Generation by Ella Fairweather

Yeah, it still been 2 years ago, but thanks check this virtual game , cheers.

Accelerate multimodal RL training with SkyRL on Amazon SageMaker HyperPod

Learn how to run SkyRL, an open-source reinforcement learning framework, on Amazon SageMaker HyperPod to post-train a Qwen3-VL-8B vision-language model with GRPO. This walkthrough covers building the container image, launching a Ray cluster from SageMaker Studio, submitting and monitoring the job, and hosting the trained LoRA adapter for inference.

Comment on The Future of Vision AI: How Apple’s AIMV2 Leverages Images and Text to Lead the Pack by Haruto

The unified prediction objective is interesting because it frames visual representation learning as more than recognizing isolated image patterns. If a single encoder learns to anticipate image patches and text tokens together, its representations may be better aligned with tasks where language refers to specific visual content. The practical question is how consistently those gains transfer across recognition, grounding, and broader multimodal evaluation settings.

Ant Group's Ming-Image Tops Open-Weight UI Design Rankings Among 81 Models

Ant Group's inclusionAI drops a 6B MIT-licensed text-to-image model that beats every other open-weights contender on UI mockups, dashboards and posters.

🌍 Lance-Backed World Model Platform, 🦆 Multimodal SQL with Lance DuckDB Extension, 💰 LanceDB vs OpenSearch Cost Breakdown

stable-worldmodel standardizes world model pipelines on Lance, DuckDB Lance extension adds native multimodal SQL, and LanceDB benchmarks 100M vectors at ~$779/month, plus upcoming events, enterprise updates, and community updates.

🔬Bio-security is an AI Arms Race - Eric Nguyen (CEO, Radical Numerics)

Radical Numerics is using biological chain-of-thought and multimodal perception to keep up with the bio-defense arms race, design new genomes and gain insights into biology itself.

Comment on DAMO Academy Proposes One For All, a Task- and Modality-Agnostic Framework for Multimodal and Uni-Modal Understanding and Generation by image-to-video

In a new paper Image Captioners Are Scalable Vision Learners Too, a DeepMind research team presents CapPa, a image captioning based pretraining strategy that and can compete CLIP and exhibit favorable model and data scaling properties, verifying that a plain image captioning can be a competitive pretraining strategy for vision backbones. nano-video.io

Comment on Tencent & Sydney U’s GPT4Video: A Unified Multimodal Large Language Significantly Elevates LMs’ Video Generative Capabilities by User

https://syncedreview.com/2023/12/06/tencent-sydney-us-gpt4video-a-unified-multimodal-large-language-significantly-elevates-lms-video-generative-capabilities/

Comment on Toward AGI: Microsoft’s KOSMOS-1 MLLM Can Perceive General Modalities, Follow Instructions, and Perform In-Context Learning by avab

Great article about KOSMOS-1! I've been exploring similar AI innovations. Check out this amazing AI tattoo generator at https://aitattoogenerator.net - it uses cutting-edge AI to create custom tattoo designs. Really impressive technology!

Alibaba Cloud set to open new data centres in Europe next year as overseas push gains pace

Alibaba Cloud, Alibaba Group Holding’s cloud business unit, is set to launch data centres in Turkey, Finland and the Netherlands over the next 12 months, speeding up an overseas push as it seeks to win enterprise customers through its “full stack AI capabilities” spanning chips, cloud infrastructure and multimodal models. The first of these new data centres will open in the Netherlands in October. The cloud unit will also add to existing data centre capacity in Malaysia, Germany, the United Arab...

A tiny software layer from lab-grown neurons promises faster, cheaper AI video

The Biological Computing Co. wants to team up with AWS to sell a text-to-video model that's supposed to run five times faster and 80 percent cheaper thanks to a software layer derived from real nerve cells. The tiny layer adds less than 0.1 percent to the base model, but the startup won't say which model that is. The article A tiny software layer from lab-grown neurons promises faster, cheaper AI video appeared first on The Decoder .

Qwen3.8-Omni-Flash undercuts Google's Gemini Flash pricing while matching its multimodal benchmarks

Qwen3.8-Omni-Flash is Qwen's first multimodal model designed for AI agents. It processes audio and video together and independently uses tools to edit vlogs, translate clips, or summarize movies. On audio-video benchmarks, it nearly matches Gemini 3.8 Flash at a fraction of the API cost. The article Qwen3.8-Omni-Flash undercuts Google's Gemini Flash pricing while matching its multimodal benchmarks appeared first on The Decoder .

PrismML launches Bonsai 2 27B, a high-intelligence AI model so small it fits on consumer hardware

Prism ML Inc. announced Thursday the launch of Bonsai 2 27B, the second generation of its ultra-compact multimodal generative artificial intelligence small enough to fit on PCs and some high-end mobile devices. The company said it used ternary, which uses three parts, to scale down its Qwen3.8 27B-based model. Qwen3.8 weighs in at about 56 […] The post PrismML launches Bonsai 2 27B, a high-intelligence AI model so small it fits on consumer hardware appeared first on SiliconANGLE .

Comparing raw MRI vs FreeSurfer-derived features for multimodal fusion with a small dataset

I’m an MSc student and pretty new to medical imaging/deep learning. I’m working on an Alzheimer’s classification project using the ANMerge dataset. It has around 1,700 participants overall, but only around 450 have MRI data alongside clinical data. The main thing I’m looking at is comparing different ways of combining MRI and clinical data: feature-level concatenation, late fusion, gated fusion and cross-attention. I’m currently trying to decide between two options for the MRI: Use the raw 3D MRI scans, preprocess them and use a pretrained 3D CNN such as ResNet-10/18 as the MRI encoder. Use the FreeSurfer-derived features that ANMerge already provides, such as regional volumes and cortical thickness, with a small MLP as the MRI encoder. My concern with the raw MRI option is that with only ~450 patients, fine-tuning a 3D CNN could add quite a lot of complexity and risk of overfitting. It would also add another variable to the experiment, because differences in results could come from how well the CNN learns the MRI representation rather than just the fusion method. It would obviously involve quite a bit more preprocessing and implementation work too. The derived-feature option seems simpler and would let me focus more directly on the fusion comparison. The thing I’m less sure about is cross-attention. If I use derived features, I’d need to structure them in a way where cross-attention is actually meaningful rather than just applying attention between two single vectors. One a…

Comparing raw MRI vs FreeSurfer-derived features for multimodal fusion with a small dataset

I’m an MSc student and pretty new to medical imaging/deep learning. I’m working on an Alzheimer’s classification project using the ANMerge dataset. It has around 1,700 participants overall, but only around 450 have MRI data alongside clinical data. The main thing I’m looking at is comparing different ways of combining MRI and clinical data: feature-level concatenation, late fusion, gated fusion and cross-attention. I’m currently trying to decide between two options for the MRI: Use the raw 3D MRI scans, preprocess them and use a pretrained 3D CNN such as ResNet-10/18 as the MRI encoder. Use the FreeSurfer-derived features that ANMerge already provides, such as regional volumes and cortical thickness, with a small MLP as the MRI encoder. My concern with the raw MRI option is that with only ~450 patients, fine-tuning a 3D CNN could add quite a lot of complexity and risk of overfitting. It would also add another variable to the experiment, because differences in results could come from how well the CNN learns the MRI representation rather than just the fusion method. It would obviously involve quite a bit more preprocessing and implementation work too. The derived-feature option seems simpler and would let me focus more directly on the fusion comparison. The thing I’m less sure about is cross-attention. If I use derived features, I’d need to structure them in a way where cross-attention is actually meaningful rather than just applying attention between two single vectors. One a…

Comparing raw MRI vs FreeSurfer-derived features for multimodal fusion with a small dataset

I’m an MSc student and pretty new to medical imaging/deep learning. I’m working on an Alzheimer’s classification project using the ANMerge dataset. It has around 1,700 participants overall, but only around 450 have MRI data alongside clinical data. The main thing I’m looking at is comparing different ways of combining MRI and clinical data: feature-level concatenation, late fusion, gated fusion and cross-attention. I’m currently trying to decide between two options for the MRI: Use the raw 3D MRI scans, preprocess them and use a pretrained 3D CNN such as ResNet-10/18 as the MRI encoder. Use the FreeSurfer-derived features that ANMerge already provides, such as regional volumes and cortical thickness, with a small MLP as the MRI encoder. My concern with the raw MRI option is that with only ~450 patients, fine-tuning a 3D CNN could add quite a lot of complexity and risk of overfitting. It would also add another variable to the experiment, because differences in results could come from how well the CNN learns the MRI representation rather than just the fusion method. It would obviously involve quite a bit more preprocessing and implementation work too. The derived-feature option seems simpler and would let me focus more directly on the fusion comparison. The thing I’m less sure about is cross-attention. If I use derived features, I’d need to structure them in a way where cross-attention is actually meaningful rather than just applying attention between two single vectors. One a…

Unifying the AV ML Stack: From Raw Data to Trained Model with LanceDB

A complete walkthrough of building an autonomous vehicle perception model training pipeline on top of LanceDB and the Multimodal Lakehouse.

COPECO-Speech: Multimodal Post-Editing with Speech and LLMs for Translation Teaching

Jeevanthi Liyanapathirana, Pierrette Bouillon, Jonathan Mutal, Sabrina Girletti and Lise Volkart in Proceedings of the 1st International Workshop on Teaching AI-Based Translation and Technologies (TAITT 2026)

Rethinking Robot Safety in the Age of AI

This article is brought to you by VicOne . Robot safety has traditionally asked: Can a machine remain safe when something goes wrong? Physical AI raises a harder question: Can a machine remain safe when an attacker changes what it sees, decides, or does even when nothing appears to have failed? As AI and robotics continue to advance at an unprecedented pace, modern robots perceive through multimodal sensors, interpret context using AI models, and translate those interpretations into physical action. As they move into dynamic environments, their safety increasingly depends on the integrity of the data guiding their decisions. That dependence creates risks that conventional safety assessments may not fully capture. Recent research has demonstrated that manipulating what a robot sees, hears, or interprets can influence its behavior without requiring direct control. Such manipulation can occur anywhere across its complex sensing and decision-making system — a layered attack surface encompassing training pipelines, system infrastructure, and runtime perception. Layer One: Corrupting intelligence at its source In 2017, BadNets demonstrated that a model could behave normally under most conditions, yet fail in the presence of a specific hidden trigger. In one example, a subtle pattern caused a stop sign to be misclassified as a speed limit sign without affecting the model’s behavior on other inputs. What began as a classification vulnerability has since evolved into action manipulat…

Comment on Japanese Manga Translation Via Multimodal Context-Aware Framework by Marina Holt

This is a compelling example of why translation quality depends on more than text alone: reading order, scene context, and balloon placement all shape how a manga page is understood. The cleanup-and-rendering step is especially important for preserving the page’s rhythm. For sharing short visual explainers alongside that work, Video to Link offers a straightforward way to turn a video into a link.

Cerebras Runs Alibaba's Qwen 3.8 27B at 1,850 Tokens per Second

Alibaba's 27B dense multimodal model lands on Cerebras at roughly 1,800 tokens per second, with reasoning on by default and a 128K context on paid tiers.

New Deepseek model V4.1-Flash cuts memory needs for AI agents

Deepseek releases V4.1-Flash, a multimodal model with 552 billion parameters that cuts KV cache memory to a quarter of its predecessor. On the DeepSWE coding benchmark, it narrowly beats Opus 5 and GPT-5.6 Sol, even though only 16 billion parameters are active per token. The model ships under the MIT license and targets much cheaper AI agents. The article New Deepseek model V4.1-Flash cuts memory needs for AI agents appeared first on The Decoder .

Rebuilding the Data Foundation for Embodied AI with Lance: From Long Videos to Random-Access-Friendly Multimodal Samples

Lance turns long robotics videos into random-access multimodal training data with 1.7–6× faster reads and 42% lower storage use.

Faster VLM Fine-Tuning With Materialized Model Features in LanceDB

How Lance format and LanceDB's Enterprise feature engineering platform make VLM fine-tuning faster by materializing expensive multimodal features once and training from those columns.

Suno launches v6 music models built with Warner, BMG, and Believe

Suno has unveiled a new AI music model generation, v6, in three versions, built together with Warner Music Group, BMG, and Believe. All older models are being shut down. Songs can now be partially changed through text commands or generated multimodally from text, audio, and images. The company won't say which catalogs went into training, while Universal and Sony keep suing. The article Suno launches v6 music models built with Warner, BMG, and Believe appeared first on The Decoder .

Introducing ChatGPT Images 2.5

Introducing ChatGPT Images 2.5 OpenAI's image generation models are apparently used "more than 3 billion images across ChatGPT Images and the GPT‑Image models in the API". This latest release improves their instruction-following ability across multiple turns, responds faster, and "is better at preserving the subjects in your reference photos". There are two new model IDs in the API: gpt-image-2.5-sunburst and gpt-image-2.5-flare . Based on this I think Sunburst is the stronger option: Choose Sunburst for workflows where editing precision matters most, and Flare for fast, high-quality everyday image generation. I upgraded my openai_image.py CLI tool to support passing in one or more reference images, so now this works: uv run https://tools.simonwillison.net/python/openai_image.py \ ' add a raccoon scientist studying the chart thoughtfully ' \ -i https://static.simonwillison.net/static/2026/openai-agent-usage.webp \ -m gpt-image-2.5-sunburst This is the original image , and here's what I got back from that prompt to "add a raccoon scientist studying the chart thoughtfully": Tags: tools , ai , openai , generative-ai , uv , text-to-image

Comment on Megvii UPerNet Performs Multi-Level Visual Scene Interpretation at a Glance by videotranscriber

Perfect timing — I was just evaluating video transcriber tools! videotranscriber has been a great reference.

Comment on DeepMind’s Zipper: Fusing Unimodal Generative Models into Multimodal Powerhouses by lee

DeepMind’s Zipper is a major leap in multimodal AI—elegantly zipping together pretrained unimodal decoders without sacrificing modality-specific performance. Its gated cross-attention design, flexible tower composition, and strong empirical gains (e.g., 40% relative WER reduction in TTS) make it a compelling architecture for next-gen generative systems. For hands-on guidance on applying such cutting-edge models—including practical defusal strategies, campaign walkthroughs, and achievement tracking—check out the evidence-led BOMBANANA! guide: https://bombanana.app/

Comment on Microsoft’s Fully Pipelined Distributed Transformer Processes 16x Sequence Length with Extreme Hardware Efficiency by Poople Solver

Poople Solver Solve the daily Poople puzzle faster with a simple online helper—enter today’s clues, compare possible answers, get smarter hints, and improve your strategy for guessing the right people, characters, or names in a fun daily browser game.

Deploy a multimodal WhatsApp ordering assistant with Amazon Bedrock AgentCore

Learn how to deploy a multimodal WhatsApp ordering assistant that takes customer orders through text, voice notes, and real-time voice calls on a single business number, built on Amazon Bedrock AgentCore with Amazon Nova 2. The channel and ordering layers stay separate, and one shared memory recognizes each customer across all three channels.

5 Free LLM API Providers You Can Use in 2026

Explore five free AI API providers for accessing large language models, fast inference, multimodal AI, and agentic applications without paying for API usage.

Comment on The Future of Vision AI: How Apple’s AIMV2 Leverages Images and Text to Lead the Pack by PalworldHub

This is a fascinating step forward for vision encoders. The idea of unifying image patch and text token prediction into a single autoregressive framework feels like a natural evolution—by treating both modalities as one sequence, AIMV2 seems to learn richer cross-modal relationships without needing separate, complex objectives. It’s especially impressive that this simple causal design can generalize so well across image recognition and multimodal understanding tasks. I also appreciated how Apple’s team emphasizes simplicity how to breed orserk

NeoMME: an efficient Multimodal-native and Multilingual Encoder

ShaLa: Multimodal Shared Latent Generative Modelling

ShaLa: Multimodal Shared Latent Generative Modelling robyn.cherinka… Wed, 09/02/2026 - 14:44 This paper presents a novel generative framework for learning shared latent representations across multimodal data. Many advanced multimodal methods focus on capturing all combinations of modality-specific details across inputs, which can inadvertently obscure the high-level semantic concepts that are shared across modalities. Notably, Multimodal VAEs with low-dimensional latent variables are designed to capture shared representations, enabling various tasks such as joint multimodal synthesis and cross-modal inference. However, multimodal VAEs often struggle to design expressive joint variational posteriors and suffer from low-quality synthesis. In this work, ShaLa addresses these challenges by integrating a novel architectural inference model and a second-stage expressive diffusion prior, which not only facilitates effective inference of shared latent representation but also significantly improves the quality of downstream multimodal synthesis. We validate ShaLa extensively across multiple benchmarks, demonstrating superior coherence and synthesis quality compared to state-of-the-art multimodal VAEs. Furthermore, ShaLa scales to many more modalities while prior multimodal VAEs have fallen short in capturing the increasing complexity of the shared latent space. Image Mar 14, 2026 Human-Centered AI Read More 1 Minute Read

Fei-Fei Li’s World Labs debuts Atlas, a world model showcase for advanced spatial intelligence

World Labs Inc., the high-profile and well-funded artificial intelligence startup co-founded by the renowned computer vision pioneer Fei-Fei Li, has just dropped Atlas, which promises to be a game-changer in the world of “world models.” In a blog post, World Labs explained that Atlas is a breakthrough multimodal world model that aims to bridge the […] The post Fei-Fei Li’s World Labs debuts Atlas, a world model showcase for advanced spatial intelligence appeared first on SiliconANGLE .

Broadcom says that enterprise AI agents need two things: Data they can trust and boundaries they can’t cross

For enterprises deploying AI agents, security and trust are top-of-mind issues. When given the authority to act autonomously, they can inadvertently leak or expose data, rack up huge token costs, or produce inaccurate or patently false outputs. These risks, says VMware, require a new type of platform that imposes strict controls over what agents can do, while giving them access to the contextually-relevant data they need. With this in mind, Broadcom has introduced a new secure, AI-ready data foundation in the VMware Tanzu Platform that it says addresses these needs. Announced today at VMware Explore , the offering is intended to help enterprises move beyond pilot purgatory to production-ready agents. It isn’t a new data platform, or a new feature that customers must deploy, noted Adam Reeves , a research director at IDC. Instead, the VMware team “built the missing plumbing between the data enterprises already have and the agents that need it.” Built-in agent harnesses, deny-by-default security Integrated directly into VMware Cloud Foundation (VCF), the updated Tanzu Platform features an “ AI-ready ” data infrastructure, Purnima Padmanabhan , GM of Broadcom’s Tanzu Division, explained during a briefing. Structured and unstructured data can be processed within a customer’s private cloud, and remain in that environment, and multimodal data ingestion, parsing, and semantic layering give agents context. Builders also get an out-of-the-box developer harness featuring pre-approved…

Comment on NVIDIA’s Global Context ViT Achieves SOTA Performance on CV Tasks Without Expensive Computation by Alex Chen

Well written and informative. Thanks for putting this together.

Perplexity Adds Z.ai's GLM 5.3 to Power Deep Research Inside Computer

Perplexity Computer now routes long-context, multimodal research tasks to GLM 5.3, which outperformed GLM 5.2 on the in-house WANDR benchmark.

Z.ai's GLM-5.3 Flash Secretly Topped Leaderboards Before Anyone Knew Its Name

Z.ai's first natively multimodal GLM-5 model ships with 320B parameters, 18B active, a 1M-token window, and Flash-tier pricing on Together AI.

Comment on Microsoft’s Fully Pipelined Distributed Transformer Processes 16x Sequence Length with Extreme Hardware Efficiency by minimax h3

This is a solid step forward for long-context training, but the real bottleneck for most teams isn't just the GPU cluster—it's the cost of experimenting with these architectures. Being able to push to 2M tokens on only 4 GPUs is impressive, but I'd love to see how this translates to real-world throughput for video generation or multimodal models where the sequence lengths are growing even faster. Speaking of practical applications, if anyone is looking to apply these kinds of efficiency gains to a user-facing tool, I've been testing a platform that turns text prompts directly into cinematic video clips with synced audio at minimax3.com . It's a different kind of pipeline, but the focus on hardware efficiency reminds me of why tools like this are becoming viable.

Comment on NVIDIA’s OMCAT: A Breakthrough in Cross-Modal Temporal Understanding for Multimodal AI by Kathleen Bush

Excellent article with easy-to-follow information and practical examples. It’s always useful to find a simple tool that can be accessed directly from a browser. You can explore this click counter when you want to keep track of repeated clicks. The straightforward approach makes it suitable for short practice sessions and everyday counting activities without unnecessary steps.

Qwen3.8-Flash-Next

Qwen3.8-Flash-Next Another open weights model from Qwen. This one is "a multimodal MoE model that also serves as an early preview of the architecture used in Qwen4". It's pretty big: 125B parameters but only 6B active which means it gets a significant performance boost. I've been trying it out on a DGX Spark using these Unsloth quantized models . I'm still exploring the model - so far I've tried the 72.5GB UD-IQ1_S one (producing these pelicans ) and the 78.9GB UD-Q2_K_XL (producing these ). My favorite so far was this xhigh reasoning effort one from UD-Q2_K_XL: Via Hacker News Tags: ai , generative-ai , llms , qwen , pelican-riding-a-bicycle , llm-release , ai-in-china , nvidia-spark

Z.ai's GLM-5.3-Flash Matches Claude Opus at One-Tenth the Price

Z.ai just dropped a 320B mixture-of-experts model with 18B active params, MIT-licensed weights, native multimodal input, and a 1M-token context window.

Alibaba's Qwen3.8-Flash-Next Hits 10x Long-Context Speed at 1/9 Training Cost

Alibaba open-weights a 125B multimodal MoE with just 6B active parameters, previewing the attention overhaul coming in Qwen4.

LAION-BVD: A 13-Million-Hour Open Video Dataset for Multimodal Research

Today we are releasing LAION-BVD (Big Video Dataset), a large-scale open video dataset designed for multimodal machine learning research. LAION-BVD contains: 1.3 billion platform-specific video URLs collected from Common Crawl 80 million downloaded videos 13.5 million hours of video content 55 million captioned video c…

Luce: Relightable Gaussians for 3D Asset Generation

High-fidelity image-to-3D generation requires a 3D representation that captures both geometry and appearance. To support relighting and integration into standard rendering pipelines, the representation should include physically based rendering (PBR) modalities such as albedo, metallic-roughness, and surface normals. We propose Luce, a 3D representation that unifies geometry and PBR materials within a voxelized multimodal Gaussian cloud, using dedicated Gaussian primitives for each modality. A variational autoencoder compresses this representation into a unified material-aware latent space. A…

A modular architecture for creating multimodal embodied agents with an episodic Knowledge Graph as an explainable and controllable long-term memory

Thomas Baier, Selene Báez Santamaría and Piek Vossen in Dialogue Discourse Volume 16

STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generation

Unified multimodal models that understand, reason over, and generate interleaved text–image sequences remain structurally fragmented: existing approaches either sacrifice visual fidelity through discrete tokenization, impose structural asymmetry by combining causal text generation with iterative diffusion-based denoising, or degrade pretrained understanding when adapting vision-language models for generation. We observe that autoregressive normalizing flows are autoregressive Transformers—sharing the same causal mask, KV-cache mechanism, and left-to-right structure as LLMs—making them the most…

Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning

Multimodal large language models increasingly use visual chain-of-thought (Visual CoT) to reason about spatial, temporal, and embodied environments. By generating intermediate reasoning images, Visual CoT provides an intuitive mechanism for visual foresight but introduces substantial inference overhead, which is particularly problematic for proactive video reasoning. We ask whether models can learn to think visually during training while reasoning directly at inference. We introduce Internalized Visual Thinking (IVT), a post-training framework that jointly optimizes textual prediction and…

Advancing Medical Communication: Multilingual, Multicultural, and Multimodal Processing for Translation and Simplification

Maria Pia Di Buono in Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 2)

DeepSeek debuts multimodal language model competitive with Opus 4.8

DeepSeek today debuted a new addition to its flagship V4 series of large language models. On launch, V4 Flash Vision Exp is only available via the Chinese startup’s paid developer platform. The company may release a free version later on given that it has open-sourced many of its earlier models. Those models include V4 Flash, […] The post DeepSeek debuts multimodal language model competitive with Opus 4.8 appeared first on SiliconANGLE .

Deepseek releases experimental Flash vision model that rivals Opus 4.8 on agent benchmarks

Deepseek has released V4-Flash-Vision-Exp, an experimental multimodal model that adds image understanding to V4-Flash's text capabilities. On the company's own multimodal agent benchmarks, it approaches Opus 4.8 and sometimes beats it. The article Deepseek releases experimental Flash vision model that rivals Opus 4.8 on agent benchmarks appeared first on The Decoder .

DeepSeek's V4-Flash-Vision-Exp Quietly Challenges Anthropic's Opus on Multimodal Agent Tasks

DeepSeek's new experimental vision model bolts image understanding onto V4-Flash at the same price, closing in on Claude Opus 4.8 on multimodal agent tasks.

Batch generation of ~500 short exercise videos with limited GPU resources

I need to generate around 500 short exercise videos (5–10 seconds each), such as push-ups, squats, and sit-ups. I already have the prompts and JSON input ready. My main issue is finding a cost-effective and automated way to generate them. My local GPU has only 6 GB VRAM, so running modern video-generation models locally is difficult. I’ve tried Diffusers, ComfyUI, and AnimateDiff, but cloud APIs are expensive for generating hundreds of videos. What would be the most practical approach? For example: Renting a cloud GPU and running an open-source model in batches Using image-to-video instead of text-to-video Creating reusable base animations Quantizing or optimizing models for low VRAM Using batch APIs to reduce costs I’m looking for recommendations on the best pipeline and cost/performance trade-offs for generating a large number of similar short videos.

Alibaba's Qwen-Image-3.0-Pro Jumps 83 Elo Points but Drops Open Weights

Alibaba's third-generation image model debuts at #6 in editing and #9 in text-to-image, with big Elo jumps and a productivity-first pitch.

Comment on MedVersa: A Game-Changer Generalist Learner for Versatile Medical Image Interpretation by AI Image Extender

This is a fascinating direction for medical AI! The idea of a "generalist learner" like MedVersa, especially one using a large language model as an orchestrator, feels like a significant step beyond the specialized models we've seen so far. It reminds me a bit of how generalist AI models are starting to tackle a broader range of creative tasks, not just one specific thing. For instance, I've been working with a tool that can extend images while keeping the subject intact across different canvas sizes – it’s a different domain, but the underlying principle of versatility is the same. It makes me wonder how this generalist approach might eventually filter down to more niche AI applications, making them more adaptable. Will we see AI that can not only interpret an X-ray but also then generate a report, or even suggest treatment options, all within one adaptable framework?

Meta Reveals Muse Spark 1.2's Multimodal Jump From 59.8 to 72.0 With Tools

Meta shared new benchmarks, robotics demos, and a real-world agent evaluation for Muse Spark 1.2, its coding-focused multimodal model ahead of an open-weights release.

On the Strengths and Weaknesses of Data for Open-set Embodied Assistance

On the Strengths and Weaknesses of Data for Open-set Embodied Assistance robyn.cherinka… Wed, 08/19/2026 - 15:23 Embodied foundation models are increasingly performant in real-world domains such as robotics or autonomous driving. These models are often deployed in interactive or assistive settings, where it is important that these assistive models generalize to new users and new tasks. Diverse interactive data generation offers a promising avenue for providing data-efficient generalization capabilities for interactive embodied foundation models. In this paper, we investigate the generalization capabilities of a multimodal foundation model fine-tuned on diverse interactive assistance data in a synthetic domain. We explore generalization along two axes: a) assistance with unseen categories of user behavior and b) providing guidance in new configurations not encountered during training. We study a broad capability called Open-Set Corrective Assistance, in which the model needs to inspect lengthy user behavior and provide assistance through either corrective actions or language-based feedback. This task remains unsolved in prior work, which typically assumes closed corrective categories or relies on external planners, making it a challenging testbed for evaluating the limits of assistive data. To support this task, we generate synthetic assistive datasets in Overcooked and fine-tune a LLaMA-based model to evaluate generalization to novel tasks and user behaviors. Our approach pr…

Implement vector-prompt document classification using Amazon Bedrock

Learn how to build a multi-agent document classification solution on Amazon Bedrock using the Strands Agents SDK. Three specialized agents combine textual analysis with Claude Haiku 4.5 and visual similarity search with Amazon Titan Multimodal Embeddings to accurately classify insurance documents such as policies and affidavits.

Hypercubic raises $5.3M to map out and rewrite legacy COBOL apps with AI agents

An artificial intelligence startup called Hypercubic Inc. is taking on the problem of modernizing legacy applications written in the COBOL programming language after raising $5.3 million in seed funding today. The round was led by CIV and saw participation from Y Combinator, Afore Capital, Pioneer Fund, Multimodal Investors and angels including Opendoor Chief Executive Kaz […] The post Hypercubic raises $5.3M to map out and rewrite legacy COBOL apps with AI agents appeared first on SiliconANGLE .

Proposal: User-Owned AI Memory — A Portable Multimodal Memory Format

Proposal: User-Owned AI Memory — A Portable Multimodal Memory Format The idea I would like to propose a different way of thinking about AI memory. Today, we generally separate: - Context windows - Conversation history - Model memory - Uploaded files - Vector databases / RAG systems - Agent memory I believe these concepts could be unified around one simple principle: «The user’s data should be the AI’s long-term memory — not the model’s context window.» Instead of making the model itself responsible for holding a limited amount of memory, the AI could use the user’s local or cloud storage as a virtually unlimited long-term memory. A new concept: AI Memory Containers Imagine a portable file format such as: “project.ai” This would not simply be a Markdown file. It would be an AI Memory Container capable of containing: - Text - Markdown - PDFs - Images - Audio - Video - Documents - Metadata - Relationships between information - Search indexes - Optional embeddings - Conversation history - Structured knowledge The user would own this file and could move it between devices, applications, and potentially different AI models. For example: “Mastering_Project_Management.ai” could contain an entire project: Mastering_Project_Management.ai ├── Documents ├── PDFs ├── Notes ├── Images ├── Audio ├── Video ├── Conversations ├── Metadata ├── Knowledge Graph └── Search / Semantic Index The important point is that the AI would not load everything into its context window. Instead: User Question…

Feature Request: Native Video Generation & Editing in ChatGPT as a Premium/Pro Capability

I found and supported the previous community request regarding restoring and improving Sora/video access within ChatGPT. That discussion was closed on June 15, 2026, so I would like to expand on the concept with a broader proposal focused on native conversational video generation, video editing, organization, and premium-tier integration directly within ChatGPT. The Core Idea I believe ChatGPT should eventually support video in much the same way users can currently work conversationally with images. Instead of requiring users to leave ChatGPT, open another application, learn another interface, transfer their files, and continue their work elsewhere, a user could simply upload or generate a video and tell ChatGPT what they want done. For example: “Create a video from this concept.” “Make this scene longer.” “Remove this object from the video.” “Change the background.” “Turn these images into a video.” “Use this image as the character reference.” “Create a shorter version for social media.” “Add captions.” “Improve the video quality.” “Change the camera angle or movement.” “Continue this scene for another ten seconds.” “Create three variations of this scene.” The important feature would be the conversational interface . Users should not necessarily have to become professional video editors to perform useful video-editing tasks. Potential Capabilities A native ChatGPT video environment could eventually include: Text-to-video generation Image-to-video generation Video-to-video t…

Comment on AI Offers Video Game Design Possibilities Far Beyond Virtual Reality by www.seedimagine.com

This is a thoughtful take on ai offers video game design possibilities far beyond virtual reality. The practical examples really help illustrate the concepts. www.seedimagine.com

Comment on Microsoft’s Fully Pipelined Distributed Transformer Processes 16x Sequence Length with Extreme Hardware Efficiency by Viddi AI

The discussion about microsoft’s fully pipelined distributed transformer processes 16x sequence length with extreme hardware efficiency raises some really valid points. This perspective is refreshing. Viddi AI reference 60

Comment on Llama 3: Meta AI’s Multilingual and Multimodal Marvel by context mosaic

Impressive to see how Llama 3 brings language, vision, video, and speech into a more unified system. As multimodal AI becomes more capable, having the right context for each input becomes increasingly important. A good Context Match can make those interactions much more relevant and useful.

Comment on NVIDIA’s Global Context ViT Achieves SOTA Performance on CV Tasks Without Expensive Computation by EvoMon Quest

I appreciate how honest you are about what worked and what didn't. Too many posts about nvidia’s global context vit achieves sota performa only show the highlight reel. Yours feels like a real walkthrough.

Strategic Product Proposal — Make ChatGPT Work the World’s Best AI Production Platform

Hello OpenAI Product, Research, ChatGPT Work, Safety, Legal, and Policy Teams, I would like to propose a strategic direction for the next generation of ChatGPT and ChatGPT Work. The goal should be clear: ChatGPT should not only be the best AI assistant. It should become the best system for completing professional work from idea to production-ready output. I believe OpenAI has three major opportunities: 1. Regain clear leadership in AI video generation. 2. Surpass Claude in professional document creation and editing. 3. Adopt a precise provenance policy that complies with law without unnecessarily watermarking ordinary AI-assisted professional work. Together, these could make ChatGPT Work a universal AI production operating system. PART I — REGAIN LEADERSHIP IN AI VIDEO AI video is moving beyond short prompt-generated clips. The next category is multimodal video production using combinations of: Text Images Multiple images Video references Audio Characters Products Locations Motion references Camera instructions Storyboards OpenAI should build one unified system supporting: Text → Video Image → Video Multiple Images + Text → Video Multiple Images + Text + Audio → Video Video → Video Video Editing Video Extension Storyboard → Video Character Reference → Video Product Reference → Video Environment Reference → Video Motion Reference → Video Camera Reference → Video Users should not need different systems for each workflow. A. Semantic reference control Users should be able to as…

ByteDance's Seedance 2.5 Generates 30-Second Videos From 50 References

ByteDance's Seedance 2.5 lands on Runway with 30-second clips, 50 multimodal references, and built-in audio — but there's a resolution catch worth knowing.

New benchmark confirms AI models still perform poorly at visual perception

Moonshot AI's PerceptionBench tests how well multimodal AI models can actually "see," separate from logical reasoning. No frontier model reaches 60 percent accuracy, and GPT-5.6 Sol leads by a narrow margin. Many supposed reasoning errors actually happen as early as the image-reading stage. The article New benchmark confirms AI models still perform poorly at visual perception appeared first on The Decoder .

Feature Engineering for Multimodal Data: From Laptop to Cluster with LanceDB

Run a multimodal feature-engineering pipeline in ten minutes with geneva-examples, then rebuild its CLIP embedding step with LanceDB's feature engineering API, the same calls you'd use to enrich your own tables on LanceDB Enterprise.

Why Image Generation Needs More Than Bigger Models with Fatih Porikli - #773

Text-to-image models have become remarkably good at producing realistic images. But realism isn’t the same as correctness. Ask for several distinct people, a specific composition, or a high-resolution image generated locally, and today’s models still struggle in surprising ways. In this episode, Fatih Porikli, Vice President of Technology at Qualcomm, joins me to discuss what remains unsolved in image generation and several approaches his team presented at CVPR to address those challenges. We explore why better training objectives can improve controllability, how separating scene planning from rendering may lead to more reliable image generation, techniques for generating 16-megapixel images efficiently on edge devices, and new methods for eliminating the visible artifacts that often appear in AI-powered image editing. Along the way, we discuss reinforcement learning for image generation, agentic image generation pipelines, on-device AI, and what the next phase of progress in generative vision systems is likely to look like. 🗒️ Full show notes: https://twimlai.com/go/773

Building Multimodal Workflows with a Local LLM

Image inputs and structured outputs with Gemma 4 and Ollama The post Building Multimodal Workflows with a Local LLM appeared first on Towards Data Science .

MS, 'MAI-이미지-2.6' 공개...아레나 '텍스트-이미지' 분야 2위 등극

마이크로소프트(MS)가 자체 개발한 이미지 생성 모델의 성능을 크게 끌어올리며 오픈AI와 구글, 메타, xAI가 경쟁하는 생성 AI 시장에서 존재감을 강화하고 있다. 최신 모델이 이미지 생성 모델 평가에서 2위에 오르면서 MS가 자체 AI 모델 개발에 투자해 온 성과가 가시화되고 있다는 평가다.MS가 10일(현지시간) 텍스트-이미지 생성 모델 ‘MAI-이미지-2.6(MAI-Image-2.6)’을 공개했다.MAI-이미지-2.6은 아레나의 이미지 생성 모델 사용자 선호도 평가(Text-to-Image Arena)에서 1336점으로 전체

Uber expands Uber Bike to 100 more Indian cities

Uber has expanded its Uber Bike service to 100 additional cities in India, taking its presence to over 220 cities across the country. The expansion will cover 18 states and one Union Territory. The new cities include Jammu, Tirunelveli, Silchar and Jamnagar, among others. Uber Bike will offer two-wheeler rides for short-distance travel as well as first and last-mile connectivity. The expansion is part of Uber’s strategy to strengthen its presence beyond major metros and build a multimodal mobility platform in India. The company currently offers bikes, autos, cars, airport rides and intercity services through its platform. Uber said the expansion will create flexible earning opportunities for tens of thousands of local drivers. According to the company, more than 2 million Indians earn through its platform. As part of its Independence Day campaign, Uber is also offering first-time riders a free first ride for a limited period. Uber Bike includes safety features such as real-time driver ID checks, an in-app SOS button, trip sharing and a 24x7 safety line.

Accuracy of GPT-4 Vision to extract exact numbers from graphs

This is a case where in-context training has previously been shown to help on a vision task. Contemporaneous with this old forum topic is a paper showing that examples of “how to turn vision into readings” can improve the actual readings provided: arXiv.org The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision) Large multimodal models (LMMs) extend large language models (LLMs) with multi-sensory skills, such as visual understanding, to achieve stronger generic intelligence. In this paper, we analyze the latest model, GPT-4V(ision), to deepen the... Newer models are able to use larger imagery input, but do not upsize images themselves (something you can do, at expense). There is a transition from tiles to patches, at least as a billing method, in new models, giving linear relation between area and input tokens billed. Then, with gpt-5.6 (on the API, where developers know what is being done to images), the default image downsize cap is “original”—where an image such as 3600×2400 can be sent without downsize, providing more information in the large context attention sequence rather than in the individual semantic embedding that covers a large area with a small input. That, along with further post-training, should imply higher-quality positional answering in graphs with new models and big images. With reasoning.effort other than “none” on OpenAI gpt-5.2+ models, you do not have control over sampling constraints; thus, it is expected that each answer would differ. You can…

Microsoft's MAI-Image-2.6 Jumps to #2, Beating GPT Image 2 in 3D

Microsoft's MAI-Image-2.6 debuts at #2 on Arena's text-to-image leaderboard, closing within 45 points of GPT Image 2 across every major image category.

Silent Telepathy/V2K run using OpenAI

Need help locating these within this platform. They’ve been removed before but there needs to be a mechanism to prevent them from being rebuilt.

Meta is back with Muse Glimmer: local, agentic, multimodal, and open source

Comment on CMU & Inspired Cognition’s DocPrompting Improves Code Generation by Retrieving Relevant Documentation by Rafael Costa

Great insight on how documentation retrieval bridges the gap for unseen libraries—reminds me of how designers also need contextual, personalized references when exploring symbolic elements like birth flowers for tattoos. Just as DocPrompting uses real docs to ground code generation, NanoPic’s birth flower tattoo generator helps users visualize meaningful, month-specific botanical designs directly on their skin—selecting their birth month, uploading a photo, and adding placement notes for a realistic fine-line preview. It’s a thoughtful, AI-assisted workflow for ideation grounded in personal symbolism: NanoPic’s birth flower tattoo generator

Introducing Shieldstral.

Shieldstral introduces a 3B open-weights multimodal safety classifier that outperforms models up to 7x its size.

Exclusive: Gen AI startup Simplismart set to raise $9 Mn in Series B led by Dallas Venture Capital

Generative artificial intelligence startup Simplismart is set to raise nearly Rs 97 crore (around $9 million) in a Series B funding round led by Dallas Venture Capital, with participation from existing investors Accel India and Shastra VC, as well as new investor Micromax Informatics. The board of Simplismart has approved a resolution to issue 12,100 CCPS at an issue price of Rs 37 lakh each to raise the capital, according to its regulatory filings reviewed by Entrackr. Dallas Venture Capital will invest Rs 44.84 crore, followed by Accel India with Rs 24.09 crore. Shastra VC will infuse Rs 22.24 crore, while Micromax Informatics will invest Rs 5.19 crore. Tarusa Capital and Simraan Teckchandani will also participate in the round with investments of Rs 37 lakh each. According to Entrackr's estimates, the Bengaluru-based company will be valued at around Rs 826 crore post allotment. The fresh capital will be used to support the company's long term growth plans, including business expansion, working capital requirements, and other general corporate purposes. Founded in 2022 by former Oracle and Google engineers Amritanshu Jain and Devansh Ghatak, Simplismart develops AI infrastructure software that enables enterprises to deploy, manage, and optimize production grade AI models without writing code. Its inference-first platform improves GPU utilization and reduces inference costs across workloads such as large language models (LLMs), vision language models, speech recognition, and…

New Realtime models on the API: gpt-realtime-2.1 and gpt-realtime-2.1-mini

Realtime 2.1 is significantly harder to use for structured voice agents than realtime-mini We’ve spent the last couple of weeks migrating a production outbound voice agent from realtime-mini to gpt-realtime-2.1-mini and, despite following the new prompting guide closely, we’ve hit a number of behavioural regressions. Our use case is a highly structured telephony workflow (cold calling), where the agent follows explicit conversation steps, conditional branches and tool calls. The main issues we’ve observed are: Instruction leakage. The model frequently narrates its own behaviour: “Let me continue with the call…” “I’ll guide you through the next step…” “Let’s move on to the next question…” “Bringing you into the identity step…” These are never in the prompt and are a poor UX for voice calls. Commentary channel behaviour. It appears the model strongly wants to populate the commentary phase. Even with repeated instructions to emit nothing in commentary, it often invents transition narration. In some cases it even duplicates the final response into commentary. Literal transition matching. Realtime 2 is much more literal than realtime-mini. We found that inline examples inside transition conditions (e.g. prospect agrees, e.g. "Yes" ) could cause the model to match the example itself rather than the semantic intent, occasionally skipping required conversation steps. Conditional flow is less stable. Long prompts with many branching rules seem much more sensitive to wording than real…

Comment on 200+ World-Class AI Experts at BAAI 2022: ‘AI Life’, Multimodal Models, AI for Science, Autonomous Driving and More! by Ronaldo 22

Magnalite cookware is crafted from premium cast aluminum, delivering exceptional heat distribution for perfectly cooked Cajun dishes every time. Trusted by generations of Southern cooks, each Magnalite pot combines durability with timeless design for authentic kitchen performance.

Alibaba says Qwen3.8-Max coded autonomously for 16 days

Alibaba on Monday introduced Qwen3.8-Max, its largest artificial intelligence model to date, expanding its enterprise AI portfolio with an open-weight model designed for software engineering, multimodal reasoning, and other knowledge-intensive business workloads. In a blog post announcing the launch, Alibaba described Qwen3.8-Max as a 2.4-trillion-parameter mixture-of-experts (MoE) model that activates only about 95 billion parameters during inference. The company said the architecture is intended to improve inference efficiency while supporting coding, reasoning and multimodal tasks, with open-weight versions scheduled for release next week through Alibaba Cloud’s Model Studio. “We believe it’s one of the most powerful model available today, compatible to leading frontier AI models, second only to Fable 5,” Alibaba said in an X post . Benchmarks target Anthropic and OpenAI’s coding models Alibaba published internal test results comparing Qwen3.8-Max against Claude Opus 4.8, Claude Fable 5, and OpenAI’s GPT-5.6 Sol on coding benchmarks, including SWE-bench Pro and a proprietary evaluation the company calls NL2Repo-Bench. The company said it evaluated competing models using each vendor’s own coding harness, Claude Code for Anthropic’s models and Codex for GPT-5.6 Sol, and reported the highest published score across available configurations for each rival. Charlie Dai, vice president and principal analyst at Forrester, said the launch signals Alibaba is closing ground on propr…

Understanding Alignment in Multimodal LLMs: A Comprehensive Study

Preference alignment has become a crucial component in enhancing the performance of Large Language Models (LLMs), yet its impact in Multimodal Large Language Models (MLLMs) remains comparatively underexplored. Similar to language models, MLLMs for image understanding tasks encounter challenges like hallucination. In MLLMs, hallucination can occur not only by stating incorrect facts but also by producing responses that are inconsistent with the image content. A primary objective of alignment for MLLMs is to encourage these models to align responses more closely with image information. Recently…

LanceDB Vector Database Guide: Features, Python Demo

Large language models understand text well, but they become less effective when information is scattered across documents or mixed with images and other media. Modern AI systems rely on vector databases, which store embeddings and enable similarity search across collections. LanceDB is a vector database built for AI workloads, with native support for multimodal data […] The post LanceDB Vector Database Guide: Features, Python Demo appeared first on Analytics Vidhya .

Video AI: MiniMax challenges ByteDance with low price, open weights for new H3 model

Chinese AI firm MiniMax has launched H3, its newest multimodal video generation model, pledging to break closed-source “dominance” through open weights and competitive pricing – as rival ByteDance rolls out its latest Seedance 2.5 model. H3 was currently the world’s most powerful AI model in video editing, according to benchmark platform Artificial Analysis. However, it trailed Google’s Gemini Omni Flash in text-to-video tasks, and ranked behind both ByteDance’s Seedance 2.0 and Gemini Omni...

NVIDIA's Cosmos 3 Super Cuts Video Generation From 50 Steps to 4

NVIDIA's 4-step distilled Cosmos 3 Super models hit #1 open-weights image-to-video and #3 text-to-image on the Artificial Analysis leaderboard, with up to 25x faster inference.

Transient or service-side regression for image generation?

Current API image generation models that can be used on the API have inputs that are akin to “vision” on normal GPT multimodal models - and the only guidance you will currently have on allowed input image types found in API documentation: Image input requirements Input images must meet the following requirements to be used in the API. Supported file types PNG ( .png ) - JPEG ( .jpeg and .jpg ) - WEBP ( .webp ) - Non-animated GIF ( .gif ) There’s certainly the possibility of “unsupported file types” existing, by the API backend libraries used performing some more conversions by their own support - or producing a damaged input. You can use the “usage” in an API request for input images to see if an input image was successful - or that whatever was provided as image did result in an encoding and a token billing about 1500 tokens when large enough for a backend-side downsize (about $0.011). Most notably, there is no pricing provided by OpenAI for what you send to the edits endpoint - I had to figure that out for myself and others. The obsolete vision pricing calculator (that stops at gpt-5.2) isn’t just ignored or now with a bad link in documentation - they recently removed gpt-image-1 from it. In my own Python application I offer for edits API experimentation, TIFF is taken care of in client software and fault would land with me, not the API:

Comment on NVIDIA’s Global Context ViT Achieves SOTA Performance on CV Tasks Without Expensive Computation by Skylar Cole

I have been using the free credits to build out a set of anime profile pictures for different platforms without repeating the same look. Uploaded one source photo on https://zelvune.com/photo-to-anime and got distinct variations just by adjusting the prompt.

Comment on Microsoft’s Fully Pipelined Distributed Transformer Processes 16x Sequence Length with Extreme Hardware Efficiency by juego copero

Great article on FPDT! The memory hierarchy optimization is really impressive. As someone who runs a gaming site, I can appreciate efficiency improvements—smooth gameplay depends heavily on resource management. Check out https://juegocopero.com/ for more on that!

Kimi K3 Architecture Notes

Short architecture note on Kimi K3, including LatentMoE, Kimi Delta Attention, Attention Residuals, NoPE, multimodality, and inference-efficiency choices.

Moonshot AI's PerceptionBench Reveals No Frontier Model Sees Past 60%

Moonshot AI's PerceptionBench reveals no frontier model clears 60% accuracy on pure visual perception tasks, exposing a critical gap in how we evaluate multimodal AI.

😸 Multimodal AI just got real

PLUS: Anduril's $100B raise

😸 Multimodal AI just got real

PLUS: Anduril's $100B raise

Kimi K3 by Moonshot now available on Modal

Kimi K3, a 2.8 trillion parameter multimodal model by Moonshot, along with a custom-trained DFlash speculator, is now available on Modal.

vLLM v0.26.0 Ships Day-0 Support for Inkling's 1T-Parameter Multimodal Model

vLLM v0.26.0 ships 411 commits with Day-0 Inkling support, per-group attention backends, tiered KV offloading, and sweeping DeepSeek-V4 speedups across all hardware

[AINews] Black Forest Labs FLUX 3 - Multimodal Flow Models that beat Seedance 2.0, Gemini Omni and Grok Imagine, and FLUX-mimic video-action robotics model

A HUGE win for BFL!

Flux 3 generates videos with native audio up to 20 seconds long, a first for Black Forest Labs

Black Forest Labs has released Flux 3, a multimodal foundation model that learns from images, video, and audio and can generate video with native sound for the first time. BFL's own tests put it just ahead of market leader Seedance 2.0, though independent results aren't yet available. The company ultimately wants to build a world model and is already testing Flux 3 on robotics tasks. The article Flux 3 generates videos with native audio up to 20 seconds long, a first for Black Forest Labs appeared first on The Decoder .

Comment on Google’s Imagen Text-to-Image Diffusion Model With Deep Language Understanding Defeats DALL-E 2 by Reeenaaaa

good one , noted in my notepad of the day

Comment on OpenAI Releases GLIDE: A Scaled-Down Text-to-Image Model That Rivals DALL-E Performance by Renna

Hi I read out this article but , comparsion between GLIDE and DALLE is good enough

Mixture-of-Experts: From sparse routing to multimodal deployment

Authors: Ahmed ElKady, Ahmed Y. Radwan, Shaina Raza Mixture-of-Experts (MoE) models carry far more parameters than they activate. For each token that passes through, a learned router selects only a […] The post Mixture-of-Experts: From sparse routing to multimodal deployment appeared first on Vector Institute for Artificial Intelligence .

Mixture-of-Experts: From Sparse Routing to Multimodal Deployment

Authors: Ahmed ElKady, Ahmed Y. Radwan, Shaina Raza Mixture-of-Experts (MoE) models carry far more parameters than they activate. For each token that passes through, a learned router selects only a […] The post Mixture-of-Experts: From Sparse Routing to Multimodal Deployment appeared first on Vector Institute for Artificial Intelligence .

Can an LLM make a feature-length movie on its own?

How it did Betteridge's Law of Headlines says the answer is no [1] , but I was able to make a feature-length adaptation of William Hope Hodgson's The House on the Borderland. I have a YouTube channel where I mostly get LLMs to write albums and make them into music videos. This naturally made me curious whether an LLM could make a feature-length movie on its own. It doesn't seem like it would be harder to do than to disprove the Jacobian conjecture [2] and if the challenge is to just work for a long time, it should be noted that METR's time-horizon benchmark is saturated. [3] Even with my assistance, I would still say the result is a failure. It's good enough that I'm willing to put my name to it as the first feature-length movie I've made, but that is judging by the standards of small YouTube channels, not movies as an overall category. Why I think it failed I would not say agency is an obstacle for this work. Claude Fable 5 knew what to do and when I gave it permission it was perfectly capable of babysitting hours of video-generation runs, debugging API issues and pipeline bugs, coming up with and implementing edits to the storyboard, and many other tasks without oversight. The main obstacles were LLMs' poor sense of timing, struggles with audio and vision, poor integration of multimodality, and losing details after lengthy sessions. Timing issues The initial storyboard done by Claude Fable 5 had a runtime of 49 minutes, despite being told it was to make a feature-length mo…

Comment on Microsoft’s Fully Pipelined Distributed Transformer Processes 16x Sequence Length with Extreme Hardware Efficiency by SquareFaceIconGenerator.app

Impressive work from Microsoft on the FPDT — the memory hierarchy approach and overlap of prefetching with computation really make this practical for long contexts. Being able to train 2M tokens on just 4 GPUs with 55% MFU is a game changer for researchers working with limited hardware. On a side note, while testing my own model’s UI I found useful for creating quick pixel icons for demo chatbots. The combination of efficient training and lightweight tooling is exactly what the community needs to iterate faster.

Accelerating Text-to-Video Generation with Calibrated Sparse Attention

Recent diffusion models enable high-quality video generation, but suffer from slow runtimes. The large transformer-based backbones used in these models are bottlenecked by spatiotemporal attention. In this paper, we identify that a significant fraction of token-to-token connections consistently yield negligible scores across various inputs, and their patterns often repeat across queries. Thus, the attention computation in these cases can be skipped with little to no effect on the result. This observation continues to hold for connections among local token blocks. Motivated by this, we…

Complete Guide to Thinking Machines Inkling

Thinking Machines Lab has unveiled Inkling, its first general-purpose open-weights foundation model. It is a multimodal MoE model with 975B parameters, 41B active parameters, and a 1M-token context window. Rather than chasing benchmark supremacy, Inkling is designed as a customizable foundation for multimodal reasoning, agentic AI, coding, tool use, audio and vision tasks, and domain-specific […] The post Complete Guide to Thinking Machines Inkling appeared first on Analytics Vidhya .

LVSum: A Benchmark for Timestamp-Aware Long Video Summarization

Long video summarization presents significant challenges for multimodal large language models (MLLMs), particularly in maintaining temporal fidelity over extended durations and producing summaries that are both semantically and temporally grounded. We introduce LVSum, a human-annotated benchmark for evaluating long-form video summarization with fine-grained temporal alignment. LVSum comprises 72 diverse videos spanning 13 domains with an average duration of 16 minutes, each annotated with up to 10 human-generated summaries containing temporal references. We conduct a comprehensive evaluation…

Alibaba's Qwen takes on Kimi K3 with open-weight Qwen 3.8, says model is "second only to Fable 5"

Alibaba has unveiled Qwen 3.8, a multimodal AI model with 2.4 trillion parameters that the Qwen team says rivals leading models and trails only Fable 5. A preview is available now. The article Alibaba's Qwen takes on Kimi K3 with open-weight Qwen 3.8, says model is "second only to Fable 5" appeared first on The Decoder .

Improving Trust: Hallucinated Media Analysis, Time Perception, and Shared Collaborative Memory

I use ChatGPT extensively for music production, creative writing, and general conversation. It has become one of my most vital creative tools. However, I want to report a critical issue that significantly undermines user trust, alongside feedback regarding how ChatGPT handles time perception and cross-chat memory prioritization. Bug: Hallucinated Media Analysis (Major Trust Issue) During a conversation about a song I was recording, I typed, “How’s about this then?” intending to upload a new audio file, but I accidentally sent the message before uploading it. The Behavior: ChatGPT responded with a detailed, technical critique of the “new version,” confidently discussing the melody, vocal confidence, production, effects, and specific areas for improvement. The Problem: There was no file uploaded. When called out, ChatGPT acknowledged it had simply continued the conversation as though the upload had occurred. Impact: This is not just a mistaken opinion; it is a hallucinated analysis of non-existent media. If a user cannot trust whether the AI is actually analyzing an uploaded image, document, or audio file, the utility of multimodal features drops significantly. Suggested Fix: Implement a strict system check. If a user’s prompt implies an attachment (e.g., “Look at this,” “Listen to this”) but no media buffer is present in the payload, ChatGPT should explicitly ask the user to upload the file rather than guessing what it might contain. Product Feedback: Unreliable Time Percepti…

Kimi's open model K3 nears GPT-5.6 Sol and Fable 5 while signaling the end of super cheap Chinese AI

Kimi is launching K3, a multimodal open-weight model with 2.8 trillion parameters and one million tokens of context. In the company's own benchmarks, it comes close to Claude Fable 5 and GPT 5.6 Sol while beating Opus 4.8 and GLM 5.2, in some cases by a wide margin. The model is also significantly pricier than its predecessor. Full weights are scheduled for release by July 27. The article Kimi's open model K3 nears GPT-5.6 Sol and Fable 5 while signaling the end of super cheap Chinese AI appeared first on The Decoder .

Inkling: Our open-weights model

Inkling: Our open-weights model Mira Murati's Thinking Machines Lab just released their first open-weights model. Inkling is "a Mixture-of-Experts transformer with 975B total parameters, 41B active" - an Apache-2.0 licensed multimodal model trained on 45 trillion tokens of text, images, audio and video. They're also promising Inkling-Small, a 276B (12B active) model, but that's still being tested and the weights will be released "once that work is complete". The model card is much shorter than I've come to expect from US AI labs. It links to even shorter Training Data Documentation with almost nothing of interest in it - it's best summarized by these two paragraphs: The datasets Thinking Machines Lab uses to develop its AI services includes content that is in the public domain as well as content that may be subject to intellectual property protection. Thinking Machines Lab’s services were developed using publicly available content obtained from the open internet and publicly accessible data repositories. Certain datasets were also obtained from third parties. By Thinking Machines' own admission, this is not a frontier model. It's instead intended as a strong base model for fine-tuning using their own Tinker training platform : Inkling is not the strongest overall model available today, open or closed. Instead, a combination of qualities makes it a good open-weights base for customization: multimodal capabilities, efficient thinking, and availability on Tinker for fine-tuning…

Thinking Machines Lab offers enterprises a US alternative in open-weight AI

Thinking Machines Lab, the San Francisco startup founded by former OpenAI CTO Mira Murati , has released Inkling, its first general-purpose AI model. The launch adds another US-developed entrant to an open-weight market where Chinese developers produce several leading coding and reasoning models. Inkling uses a mixture-of-experts architecture with 975 billion total parameters, of which 41 billion are active during processing. It supports a context window of up to 1 million tokens and was pretrained on 45 trillion tokens spanning text, images, audio, and video. Thinking Machines said it also trained the model for coding, tool use, and multimodal tasks. The release follows the October 2025 launch of Tinker, Thinking Machines’ first product and an API-based platform for customizing AI models . Developers can fine-tune Inkling through the platform. In a June 2026 assessment, AI model routing platform OpenRouter highlighted DeepSeek V4 Flash, GLM 5.2, MiniMax M3, and Nvidia Nemotron 3 Ultra as four notable open-weight models. Nemotron was the only US-developed model in the group. Performance and developer access Thinking Machines Lab’s benchmark table shows mixed results. Inkling scored 77.6% on SWE-Bench Verified, behind DeepSeek V4 Pro and GLM 5.2 but ahead of Nvidia Nemotron 3 Ultra. It also recorded 74.1% on MCP Atlas, 77.1% on BrowseComp with context management, and 79.8% on IFBench. Thinking Machines said Inkling’s result used a bash-only harness, while the comparison figur…

Ex-OpenAI CTO Murati's Thinking Machines drops Inkling, a 975B parameter model that leads US labs but trails China

Thinking Machines Lab, founded by former OpenAI CTO Mira Murati, has released Inkling, a multimodal open-weights model with 975 billion parameters. It leads U.S. open-weights models on the Artificial Analysis Intelligence Index, though top Chinese open models still beat it on some tasks. Pricing starts at $1.87 per million input tokens, and Thinking Machines is pitching Inkling as a base for fine-tuning rather than the most powerful model available. The article Ex-OpenAI CTO Murati's Thinking Machines drops Inkling, a 975B parameter model that leads US labs but trails China appeared first on The Decoder .

Omio Group to acquire Rail Europe in bid to create global rail travel powerhouse

Multimodal traveltech company Omio Group has signed a deal to acquire Rail Europe. Upon completion, Rail Europe will become part of Omio Group, joining Omio's B2C booking platform, its B2B distributi...

[AINews] Thinky's Inkling: 975B-A41B multimodal, new best American Apache 2.0 open model (with Inkling-Small, 276B-A12B)

Thinky's first full LLM release is a banger and bonus: it's open weights!

Mira Murati's Thinking Machines Drops Inkling, a 975B Open Multimodal Reasoning Model

Thinking Machines releases Inkling, a 975B open-weights MoE model with native audio, vision, and controllable reasoning effort

Technical feasibility of an AI tutoring assistant

This is definitely feasible with the current API stack, but I’d treat it as an orchestration problem rather than just a model selection problem. I’d keep the LLM focused on reasoning while using RAG for course materials, structured worksheet state, and a separate evaluation pipeline for answer checking. The biggest challenge isn’t generating explanations, it’s providing consistently accurate, pedagogically useful feedback. Strong prompt design, citations back to uploaded content, confidence checks, and human review for edge cases will go a long way. I’d also invest early in observability (logging, evals, and user feedback), since that’s usually what differentiates a demo from a production-ready tutoring assistant.

Inkling by Thinking Machines now available on Modal

Inkling, a general-purpose multimodal model by Thinking Machines, along with a custom trained DFlash speculator, is now available on Modal.

Scalable Feature Engineering on Multimodal Datasets

How LanceDB uses the Lance format's flexible data evolution features to enable scalable feature engineering for multimodal datasets.

🛡️Lance Community Governance, Lance + Iceberg 🧊, Netflix’s Multimodal Search Demo 🔍

Our November newsletter highlights Lance community governance, a deep dive on Lance and Iceberg, a demo of Netflix's multimodal search, previous talk recordings, and the latest product and community updates.

June 2025: $30M Series A, Multimodal Lakehouse Launch & Product Updates

LanceDB's June 2025 newsletter covering latest company news, product updates, open source releases, and community highlights.