NVIDIA GTC 2025 - Building LLM-Powered Applications
Chip Huyen and I share what we've learned, best practices, and insights at NVIDIA GTC 2025.
Latest AI/ML News
27927 matching items
Improving Recommendation Systems & Search in the Age of LLMs
Model architectures, data generation, training paradigms, and unified frameworks inspired by LLMs.
Federal Minister for Research Cem Özdemir visits Tübingen AI Center
Exchange on artificial intelligence at national and European level - Minister Özdemir: "The AI competence centers - such as the Tübingen AI Center - are crucial for our technological sovereignty in AI".
Gabo Foundation and Proyecto Desconfío launch free guide against AI disinformation
“The Gabo Foundation and Proyecto Desconfío launched ‘Disinformation in times of AI: journalism to build trust,’ a free guide that addresses the role of AI to spread, but also to combat disinformation in the digital world. [...] The digital guide, developed from the panels of the most recent Global Summit on Disinformation—an annual event that […] The post Gabo Foundation and Proyecto Desconfío launch free guide against AI disinformation appeared first on LatAm Journalism Review by the Knight Center .
Gabo Foundation and Proyecto Desconfío launch free guide against AI disinformation
“The Gabo Foundation and Proyecto Desconfío launched ‘Disinformation in times of AI: journalism to build trust,’ a free guide that addresses the role of AI to spread, but also to combat disinformation in the digital world. [...] The digital guide, developed from the panels of the most recent Global Summit on Disinformation—an annual event that […] The post Gabo Foundation and Proyecto Desconfío launch free guide against AI disinformation appeared first on LatAm Journalism Review by the Knight Center .
RoBERTa Model for Merchant Categorization at Square
Harnessing Large Language Models to Deliver Accurate Business Insights
Why not cache the Q (query) matrix?
Short Version Why do we cache the K (key) and V (value) matrices, but not he Q (query) matrix? Long Version Given a toy set of 2-dimensional embedding vectors: Token Embedding Vector quick [0.27 0.78] lazy [0.38 0.58] brown [0.50 0.83] jumps [0.20 0.53] over [0.46 0.59] the [0.45 0.55] fox [0.19 0.69] dog [0.51 0.47] Meaning the input tokens the quick brown fox would be represented by their embeddings vectors in the X (input) matrix: X ╭ ╮ │0.45 0.55│ ;the │0.27 0.78│ ;quick │0.50 0.83│ ;brown │0.19 0.69│ ;fox ╰ ╯ And we have a simple set of toy W q , W k , W v weight matrices. Our toy embedding vectors are 2-dimensional. That then requires the weight matrices to have 2 rows; but they can have an arbitrarily decided number of columns; we choose 3 columns. Here are our toy weight matrices: W q (query) ╭ ╮ │0.6 -0.4 0.1│ │0.3 0.9 -0.2│ ╰ ╯ W k (key) ╭ ╮ │ 0.5 0.2 -0.3│ │-0.1 0.8 0.4│ ╰ ╯ W v (value) ╭ ╮ │0.7 -0.3 -0.2│ │0.4 0.5 -0.1│ ╰ ╯ Then we multiply the input matrix by the three W Q , W K , and W V weight matrices: Query Matrix: Q = X*Wq ╭ ╮ ╭ ╮ │0.45 0.55│ ╭ ╮ │ 0.43 0.32 -0.06│ │0.27 0.78│ │ 0.6 -0.4 0.1│ = │ 0.40 0.59 -0.13│ │0.50 0.83│ │ 0.3 0.9 -0.2│ │ 0.55 0.55 -0.12│ │0.19 0.69│ ╰ ╯ │ 0.32 0.54 -0.12│ ╰ ╯ ╰ ╯ Key Matrix: K = X*Wk ╭ ╮ ╭ ╮ │0.45 0.55│ ╭ ╮ │ 0.17 0.53 0.08│ │0.27 0.78│ │ 0.5 0.2 -0.3│ = │ 0.06 0.68 0.23│ │0.50 0.83│ │-0.1 0.8 0.4│ │ 0.17 0.76 0.18│ │0.19 0.69│ ╰ ╯ │ 0.03 0.59 0.22│ ╰ ╯ ╰ ╯ Value Matrix: V = X*Wv ╭ ╮ ╭ ╮ │0.45 0.55│ ╭ ╮ │ 0.54 0.14 -0.…
An analysis of the Square and Cash App outage
Service Disruption - Public Postmortem [2025/02/26]
uv + Ray: Pain-Free Python Dependencies in Clusters
When we discovered the uv package manager, we realized that a new era of Python package management had arrived. It tackles the Python developers’ dependency headaches with consistency and speed, which also enables quick development cycles. With uv + Ray, say goodbye to slow container builds and hello to instant, cluster-wide dependency sync.
From research to startup: Polybot receives order for AI-based harvesting robotics
ELLIS Institute Tübingen and Tübingen AI Center achieve important milestone for innovation in automated agriculture.
Announcing Mobile Payments SDK GA and New Terminal API Features
Expand your reach and drive remarkable in-person payment experiences.
AI Now Coauthors Report on Surveillance Prices and Wages
Today, AI Now—along with a group of experts and civil society organizations—released a report outlining how surveillance of prices and wages harms the public. Read it here: The post AI Now Coauthors Report on Surveillance Prices and Wages appeared first on AI Now Institute .
PINNACLE: PINN Adaptive ColLocation and Experimental Points Selection
As we all know, deep learning has been applied with great success to domains with...
fasttransform: Reversible Pipelines Made Simple
Introducing fasttransform, a Python library that makes data transformations reversible and extensible through the power of multiple dispatch.
Small Businesses’ Guide to the AI Act
Everything you need to know about the AI Act, for small and medium-sized enterprises (SMEs) in the EU and beyond.
Why do we need randomized tests in hypothesis testing?
This is related to my answer here In statistical theory, when discussing hypothesis testing, it is classical to introduce randomized tests. These rely either on direct randomization or on ancillary information to create tests which have desirable statistical properties (maximized power for a given level) but which are otherwise weird. If the goal of statistics is to build rational methods for data analysis, why is the most likely hypothesis corresponding to some data sometimes left to chance? My question is about what would happen if we ignored randomized tests. Assume that we rewrote statistical theory from scratch and absolutely refused to consider them as valid. Would that create holes in the resulting theory? What would be missing there? While writing my earlier answer, I could not actually think of any reasons in favor of randomized tests besides mathematical aesthetic arguments. Since this question is more about teaching and understanding statistical theory, do not hesitate to deviate from it if you feel that I am missing important context. Edit: attempt at a definition of a randomized test Given some dataset $x_i$ and some models $f(x;\theta)$ segregated into two ensembles $\Theta_0$ $\Theta_1$ , a test is a procedure which returns a label $0$ or $1$ given the $x_i$ . Defining a randomized test is tricky. Some cases are obvious since they introduce an extra layer of randomness on top of the data and this non-informative randomness is used as part of the test. For exam…
Is the policy a function in this policy gradient?
In the book I am reading "Foundations of Deep Reinforcement Learning" by Laura Graesser & Wah Loon Keng, I saw the formula to optimize the $\theta$ $$ \theta \leftarrow \theta + \alpha \nabla_\theta J(\pi_\theta) \\ \nabla_\theta J(\pi_\theta) = E_(\tau ~ \pi_\theta)[\sum_{t=0}^T R_t(\tau)\nabla_\theta\log\pi_\theta(a_t|s_t)] $$ So, is the policy a function? Does it have to be it to calculate the gradient? So, how do I identify that policy function? I still don't get the math. I searched and saw that people usually use the softmax function, but the the parameter in that softmax function is something called a scoring function. How do I identify the scoring function?
Professor Peter Gehler Joins Tübingen AI Center
We are delighted to welcome Peter Gehler at the Tübingen AI Center!
The Impact of AI in Education: Navigating the Imminent Future
What must be considered to build a safe but effective future for AI in education, and for children to be safe online?
Test accuracy much higher than train accuracy on Cifar-100 and 470000 parameters
I built a model with 470,000 parameters for image classification . This model was trained on CIFAR-100 for 400 epochs . I used learning rate scheduler, dropout, Random Data Augmentation (MixUp & CutMix) with 50% probability, label smoothing, warmup (10 epochs) in the model. The test accuracy is 70% , while the training accuracy is 45% . Does the higher test accuracy compared to training accuracy indicate a problem? I have trained the model multiple times and consistently achieved the same accuracy.
Does changing CUDA, cuDNN, OS (etc.) versions affect deep learning training results?
Questions I am training deep neural networks and I have heard that changing system configurations (such as CUDA, cuDNN, Hardware, or even the OS version) can sometimes lead to different training results, even when using the same dataset, model architecture, and hyperparameters. I understand that floating-point arithmetic on GPUs is not always deterministic, but I would like to know more about how changes in system libraries (e.g., upgrading from CUDA 11 to 12, or switching from cuDNN 8.0 to 8.9) can impact training reproducibility. My questions are: How exactly do changes in CUDA, cuDNN, or the OS kernel affect deep learning training? Are there any empirical studies or papers that analyze and demonstrate the effect of these changes on model convergence and final accuracy? What are the best practices to ensure reproducibility across different system configurations (other than fixing random seed)? If I want to obtain different results while training a model, can I achieve this using Docker containers? What should I change in the Dockerfile? Any insights, references, or explanations would be greatly appreciated! Experiments I have tried to train the same model on two different configurations. In order to speedup the experiments, i have used docker containers. In particular, on a machine equipped with GTX1080ti and ubuntu 24.04LTS I have built two containers with the following specs: ubuntu 18 | cuda 12.0.1 | cudnn 8.8.0.121-1+cuda12.0 | nvidia drivers 560.35.03 ubuntu 22 | cuda…
Carlos Guestrin named as new Director of the Stanford AI Lab!
We thank Christopher Manning for being Director of the Stanford AI Lab during a period of enormous growth for AI and SAIL from 2018–2025 and today welcome Carlos Guestrin, Fortinet Founders Professor of Computer Science, as the new Director of SAIL.
Congratulations to Sebastian Thrun for receiving honorary doctorate from Geogia Tech!
Sebastian Thrun received an honorary Doctorate from Georgia Tech! This is Sebastian’s fourth honorary doctorate. Sebastian also served as a speaker for Georgia Tech’s Fall commencement ceremony
Best Practices for Using Third-Party APIs
Optimizing third-party API integrations for your user experience
Injecting Domain Expertise Into Your AI System
When starting their AI initiatives, many companies are trapped in silos and treat AI as a purely technical enterprise, sidelining domain experts or involving them too late. They end up with generic AI applications that miss industry nuances, produce poor recommendations, and quickly become unpopular with users. By contrast, AI systems that deeply understand industry-specific […] The post Injecting Domain Expertise Into Your AI System appeared first on TOPBOTS .
Open-Source Large Language Models for Transparent AI in Europe
Europe's leading AI companies and research institutions combine their expertise to develop next-generation open-source language models to advance European AI capabilities, the OpenEuroLLM project.
What AI can tell us about microscope slides
A friendly introduction to Foundation Models for Computational Pathology
Context and Agenda for the 2025 AI Action Summit
The AI Action Summit will take place in Paris from 10-11 February 2025. Here we list the agenda and key deliverables.
Beyond DeepSeek: An Overview of Chinese AI Tigers and Their Cutting-Edge Innovations
The recent disruption caused by DeepSeek’s R1 model sent shockwaves through the AI community, demonstrating that Chinese AI advancements may have been underestimated. The model’s performance, rivaling some of the most advanced offerings from OpenAI and Anthropic at a fraction of the cost, signaled a new era of competition in artificial intelligence. However, DeepSeek is […] The post Beyond DeepSeek: An Overview of Chinese AI Tigers and Their Cutting-Edge Innovations appeared first on TOPBOTS .
Carving Out Your Competitive Advantage With AI
When I talk to corporate customers, there is often this idea that AI, while powerful, won’t give any company a lasting competitive edge. After all, over the past two years, large-scale LLMs have become a commodity for everyone. I’ve been thinking a lot about how companies can shape a competitive advantage using AI, and a […] The post Carving Out Your Competitive Advantage With AI appeared first on TOPBOTS .
Kube-Policies: Guardrails for Apps Running in Kubernetes
Design considerations for highly sensitive environments.
Alternative DeepSeek V3 providers
DeepSeek's API has been experiencing reliability issues. Here are alternative providers you can use.
![[GRPO Explained] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://i3.ytimg.com/vi/bAWV_yrqx4w/hqdefault.jpg)
[GRPO Explained] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
#deepseek #llm #grpo GRPO is one of the core advancements used in Deepseek-R1, but was introduced already last year in this paper that uses a combination of new RL techniques and iterative data collection to achieve remarkable performance on mathematics benchmarks with just a 7B model. Paper: https://arxiv.org/abs/2402.03300 Abstract: Mathematical reasoning poses a significant challenge for language models due to its complex and structured nature. In this paper, we introduce DeepSeekMath 7B, which continues pre-training DeepSeek-Coder-Base-v1.5 7B with 120B math-related tokens sourced from Common Crawl, together with natural language and code data. DeepSeekMath 7B has achieved an impressive score of 51.7% on the competition-level MATH benchmark without relying on external toolkits and voting techniques, approaching the performance level of Gemini-Ultra and GPT-4. Self-consistency over 64 samples from DeepSeekMath 7B achieves 60.9% on MATH. The mathematical reasoning capability of DeepSeekMath is attributed to two key factors: First, we harness the significant potential of publicly available web data through a meticulously engineered data selection pipeline. Second, we introduce Group Relative Policy Optimization (GRPO), a variant of Proximal Policy Optimization (PPO), that enhances mathematical reasoning abilities while concurrently optimizing the memory usage of PPO. Authors: Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhan…
R1+Sonnet set SOTA on aider’s polyglot benchmark
R1+Sonnet has set a new SOTA on the aider polyglot benchmark. At 14X less cost compared to o1.
Common pitfalls when building generative AI applications
As we’re still in the early days of building applications with foundation models, it’s normal to make mistakes. This is a quick note with examples of some of the most common pitfalls that I’ve seen, both from public case studies and from my personal experience. Because these pitfalls are common, if you’ve worked on any AI product, you’ve probably seen them before. 1. Use generative AI when you don't need generative AI Every time there’s a new technology, I can hear the collective sigh of senior engineers everywhere: “Not everything is a nail.” Generative AI isn’t an exception — its seemingly limitless capabilities only exacerbate the tendency to use generative AI for everything. A team pitched me the idea of using generative AI to optimize energy consumption. They fed a household’s list of energy-intensive activities and hourly electricity prices into an LLM, then asked it to create a schedule to minimize energy costs. Their experiments showed that this could help reduce a household’s electricity bill by 30%. Free money. Why wouldn’t anyone want to use their app? I asked: “How does it compare to simply scheduling the most energy-intensive activities when electricity is cheapest? Say, doing your laundry and charging your car after 10pm?” They said they would try it later and let me know. They never followed up, but they abandoned this app soon after. I suspect that this greedy scheduling can be quite effective. Even if it’s not, there are other much cheaper and more reliable…
Using uv as an installer
Reliably packaging & distributing python CLI tools is hard. Aider uses uv in novel ways to make it easy to install the aider CLI, its dependencies and python 3.12. All in an isolated env.
Building AI Reading Club: Features & Behind the Scenes
Exploring how an AI-powered reading experience could look like.
Agents
Intelligent agents are considered by many to be the ultimate goal of AI. The classic book by Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach (Prentice Hall, 1995), defines the field of AI research as “ the study and design of rational agents. ” The unprecedented capabilities of foundation models have opened the door to agentic applications that were previously unimaginable. These new capabilities make it finally possible to develop autonomous, intelligent agents to act as our assistants, coworkers, and coaches. They can help us create a website, gather data, plan a trip, do market research, manage a customer account, automate data entry, prepare us for interviews, interview our candidates, negotiate a deal, etc. The possibilities seem endless, and the potential economic value of these agents is enormous. This section will start with an overview of agents and then continue with two aspects that determine the capabilities of an agent: tools and planning. Agents, with their new modes of operations, have new modes of failure. This section will end with a discussion on how to evaluate agents to catch these failures. This post is adapted from the Agents section of AI Engineering (2025) with minor edits to make it a standalone post. Notes : AI-powered agents are an emerging field with no established theoretical frameworks for defining, developing, and evaluating them. This section is a best-effort attempt to build a framework from the existing literature…
Does hypothesis testing help make a decision in case of an A/B test?
I developed a text-generation pipeline based on recent advancements in Large-Language Models (LLMs). Users can type a topic, and my complex pipeline generates an article. I measure user satisfaction by asking how satisfied they are on a 5-point ordinal scale under each article (C-SAT). I have implemented a pipeline variation that uses cheaper, dumber LLMs in some places. I performed an A/B test to determine the difference between the current version of the pipeline and the cheaper one. Let's say the average C-SAT is 3.9 vs. 3.8, so the cheaper version has the C-SAT score lower by 0.1. Now, I have to decide whether to introduce the new version of the pipeline to reduce costs and take the risk of reducing the average C-SAT. I want to know if the decrease in the C-SAT is significant enough to give up on cutting costs. Q1: Does hypothesis testing make sense in this case? Q2: If so, then what could be a population? The number of future articles is now known. Moreover, one of the versions will not be continued. Does it mean I can’t apply the test? A result of a test would be the evidence against the null hypothesis. Let’s say my null hypothesis is “population distributions of both A and B samples have equal mean." From the perspective of the original problem ("if the decrease in the C-SAT is significant enough to give up on cutting costs"), such H0 is an intermediate problem. Q3: How do I know that finding an answer to such an intermediate problem helps me find an answer to my ori…
Traditional Holiday Live Stream
https://ykilcher.com/discord Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://discord.gg/4H8xxDF BitChute: https://www.bitchute.com/channel/yannic-kilcher Minds: https://www.minds.com/ykilcher Parler: https://parler.com/profile/YannicKilcher LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/ BiliBili: https://space.bilibili.com/1824646584 If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://www.subscribestar.com/yannickilcher Patreon: https://www.patreon.com/yannickilcher Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
2024 Year in Review
A peaceful year of steady progress on my craft and health.
o1 tops aider’s new polyglot leaderboard
o1 scores the top result on aider's new multi-language, more challenging coding benchmark.
Announcing the Square Go SDK
Easier Square integrations with your Go backend
Is AI progress slowing down?
Making sense of recent technology trends and claims