Mechanistic Anomaly Detection Research Update 2
Interim report on ongoing work on mechanistic anomaly detection
Latest AI/ML News
28078 matching items
RLHF and RLAIF in GPT-NeoX
GPT-NeoX now supports post-training thanks to a collaboration with SynthLabs.
Iteratively Reweighted Least Squares (IRLS) and Gauss-Newton
I am studying nonlinear regression optimization methods. I want to show that the Gauss-Newton method can be seen as an IRLS. We want to maximize $$ - \sum_{i = 1}^{n} (y_i - h(\mathbf{x}_i^T \mathbf{\beta}))^2 $$ Approximating $h(\mathbf{x}_i^T \mathbf{\beta})$ we have $$ - \sum_{i = 1}^{n} [y_i - h(\mathbf{x}_i^T \mathbf{\beta}^{(t)}) - h'(\mathbf{x}_i^T \mathbf{\beta}) \mathbf{x}_i^T (\mathbf{\beta} - \mathbf{\beta}^{(t)}) ]^2 $$ and here I don't know what to do. The books I have read do not show what to do when you have any function $h$ , or that explain the Gauss-Newton method in a direct way. I would appreciate any kind of help or suggestions. I am also looking for references on the subject in this regard.
FAQ about the book and our writing process
What's in the book and how we wrote it
Llama 3.2 Vision-Instruct Inference Speed on A100 or H100 GPU
Can anyone provide an estimated time of how long does it take for Llama-3.2 Vision-Instruct 11-B model to: process an image size of 1-MB and prompt size of 1000 words and generate a response of 500 words The GPUs used for inference could be A100, A6000, or H100.
Challenges in Aggregating Outputs from Classifiers Trained on Subsets of Classes
I’m currently working on a project involving several classifiers, each trained on a subset of classes. These classifiers are designed to handle different aspects of the classification task, but I’m facing a challenge when it comes to aggregating their outputs into a single prediction. For example, if one classifier is responsible for distinguishing between classes 0 and 1, and another handles classes 2 and 3, how can we effectively combine their results when the correct answer belongs to class 1? Our initial approach was to use an "other" class to indicate when an input doesn’t belong to a classifier’s assigned classes, but this did not yield the desired results. We are now exploring the possibility of implementing an additional head for detecting out-of-distribution classes, but we’re looking for a more efficient and streamlined solution. Has anyone encountered a similar issue or have any suggestions for effectively aggregating outputs from multiple classifiers? Thank you for your assistance!
Enhancing Data Quality Using Better Designed ETLs
Sharing an ETL design doc template
First AI Certification for Design & Media Practitioners Launched
In a bold initiative that merges technology with creativity, Nanyang Polytechnic (NYP) and AI Singapore (AISG) have joined forces to introduce the Certified AI Practitioners for Design & Media...
Separating code reasoning and editing
An Architect model describes how to solve the coding problem, and an Editor model translates that into file edits. This Architect/Editor approach produces SOTA benchmark results.
BadEdit: Backdooring Large Language Models By Model Editing
Large Language Models (LLMs) exemplified by ChatGPT continue to gain widespread usage in addressing a...
Low validation loss from the first epoch?
The initial validation loss is low from the first epoch and then decreases slightly. What does this actually mean? Does it indicate that the model can effectively and quickly identify patterns for this task? I can see that the model works in practice, but the results (some image restoration) aren’t ideal yet, so I want to improve its performance even further. Given this low loss from the first epoch, should I focus on training with more data or on adjusting the architecture and layers to be even more complex, etc.? Given the small differences between the first epoch and the last, is it more likely that the model was barely able to improve performance during these epochs, or could the difference in loss still be meaningful? The dataset count was of 10,000 images - 0.9 for training, 0.1 for validation. First epoch loss: Epoch [1/50], Training Loss: 0.026428, Validation Loss: 0.023727 Last epoch and plateau: Epoch [34/50], Training Loss: 0.020682, Validation Loss: 0.020651
The Practitioner's Guide to the Maximal Update Parameterization
Exploring the implementation details of muTransfer
Can AI automate computational reproducibility?
A new benchmark to measure the impact of AI on improving science
I'm trying to train an AI, but I have low accuracy using Rust and PyTorch
I'm just starting out in the world of machine learning, and I really like Rust. I've been testing and learning more. I took the example of transfer training and did some tests, but I can't understand why I have high accuracy in training and low accuracy in testing using the same validation base. Why? I studied overfit, but it doesn't seem to be the case, because I'm using the same validation base without new data. use std::env; use std::error::Error; use std::path::PathBuf; use anyhow::{ bail, Result }; use tch::nn::{ self, ModuleT, OptimizerConfig, VarStore }; use tch::vision::{ imagenet, resnet }; use tch::{ Device, Kind, Tensor }; pub fn bee_test() -> Result > { tch::manual_seed(123); let manifest_dir = env::var("CARGO_MANIFEST_DIR")?; let project_dir = PathBuf::from(manifest_dir); let dataset_path = project_dir.join("data/hymenoptera_data"); let dataset = imagenet::load_from_dir(dataset_path)?; println!("{dataset:?}"); let model_path = project_dir.join("data/bee.ot"); println!("Caminho do modelo: {:?}", model_path); let device = Device::cuda_if_available(); let mut vs = VarStore::new(device); vs.load(model_path.as_path()).map_err(|op| { format!("Erro ao carregar o modelo: {:?}", op); op })?; let net = resnet::resnet34_no_final_layer(&vs.root()); let linear = nn::linear(vs.root(), 512, 2, Default::default()); let net2: nn::Sequential = nn ::seq() .add_fn(move |xs| net.forward_t(xs, false)) .add(linear); let predicted = net2.forward_t(&dataset.test_images, false); let prob…
o1-preview is SOTA on the aider leaderboard
Preliminary benchmark results for the new OpenAI o1 models.
Start reading the AI Snake Oil book online
The book was published September 2024
From Code to Robots: The Top AI Trends Transforming Business and Life
Artificial intelligence is no longer a concept of the distant future – it’s here, evolving at a rapid pace and reshaping industries in real time. From healthcare to entertainment, AI’s influence is everywhere, sparking innovation, efficiency, and even ethical debates. But with so much happening at once, where exactly is the industry heading? To make […] The post From Code to Robots: The Top AI Trends Transforming Business and Life appeared first on TOPBOTS .
Towards Robust and Expressive Whole-body Human Pose and Shape Estimation
Introduction Given a single RGB image, 3D human pose and shape estimation aims to reconstruct...
What's Missing From LLM Chatbots: A Sense of Purpose
LLM-based chatbots’ capabilities have been advancing every month. These improvements are mostly measured by benchmarks like MMLU, HumanEval, and MATH (e.g. sonnet 3.5, gpt-4o). However, as these measures get more and more saturated, is user experience increasing in proportion to these scores? If we envision a future
Advancing Customizable Benchmarking in HELM via Unitxt Integration
The Holistic Evaluation of Language Models (HELM) framework is an open source framework for reproducible and transparent benchmarking of language models that is widely adopted by academia and industry. To meet HELM users’ needs for more powerful benchmarking features, we are proud to announce our collaboration with Unitxt, an open-source community platform developed by IBM Research for data preprocessing and benchmark customization. The integration of Unitxt into HELM gives HELM users access to the vast Unitxt catalog of benchmarks, and allows users to run sharable and customizable evaluation pipelines with greater ease.
Breaking Barriers, Building Bridges: Increasing Language Representation in Southeast Asia
Held outside of Singapore for the very first time, the third Languages Summit...
Réinventer l’éducation : quand le numérique transforme les apprentissages
Réinventer l’éducation : quand le numérique transforme les apprentissages mtestari lun, 09/02/2024 - 09:00 La transformation numérique de l’école doit être une opportunité pour mieux apprendre et individualiser l’apprentissage grâce au développement de nouvelles technologies associées à la formation de la communauté enseignante et des élèves aux compétences du 21e siècle. Autant d’enjeux sociétaux et scientifiques que relèvent, depuis plus de dix ans, les équipes pluridisciplinaires du Centre Inria de l’université de Bordeaux avec des chercheurs et chercheuses en intelligence artificielle, en interaction Humain - Machine, en neurosciences, en psychologie développementale et en sciences de l’éducation. © freepik /Photo Pch.Vector Mieux comprendre les processus d’apprentissage L’une des caractéristiques la plus importante pour l’ensemble des équipes impliquées reste leur approche systémique, positionnant l’apprenant toujours au centre de l’étude en considérant l’ensemble de ses interactions avec l’environnement dans lequel il évolue. « Ces recherches ne peuvent être menées sans l’appui des acteurs de terrain que sont les enseignants et les professionnels de l’éducation. Elles restent indissociables des avancées des sciences humaines pour ancrer la transformation numérique de l’école sur les besoins des apprenants, des éducateurs et de leurs environnements » souligne Nicolas Roussel, directeur du Centre Inria de l’université de Bordeaux. Le premier défi consiste à mieux compren…
Clean-image Backdoor: Attacking Multi-label Models with Poisoned Labels Only
Multi-label learning Multi-label learning is commonly exploited to recognize a set of categories in an...
The AI Investment Landscape: Who’s Funding the Next Wave of Innovation?
Artificial Intelligence (AI) is reshaping the tech landscape, attracting a diverse array of investors eager to capitalize on its potential. In this article, we explore the key players driving investments in AI, from tech giants and venture capital funds to angel investors, and examine the companies they are betting on. The post The AI Investment Landscape: Who’s Funding the Next Wave of Innovation? appeared first on TOPBOTS .
Why use the MLP Gaussian Decoder in a VAE if it’s limited in modeling complex data?
My question concerns the paper arxiv.org/pdf/1312.6114. I want to know why they proposed to use the MLP Gaussian decoder with parameters given by the MLP transformation of the z variable as the likelihood model $p_{\theta}(x|z)$ . If they are using NNs to perform inference on the best set of parameters, why use a Gaussian if it is very limiting in modeling complex data?
The AI Act: Responsibilities of the European Commission (AI Office)
If you are unsure who is implementing and enforcing the new digital law and what the specific time frames are, you might find this post—and our post on the responsibilities of the EU Member States—very helpful. The tables below provide a comprehensive list of all obligations and tasks that the AI Act places upon to the […]
The AI Act: Responsibilities of the EU Member States
If you are unsure who is implementing and enforcing the EU AI Act and what the specific time frames are, you might find this post—and our post on the responsibilities of the European Commission (AI Office)—very helpful. The tables below provide you with a comprehensive list of all obligations and tasks that the AI Act places […]
Understanding the embeddings model (dunzhang/stella_en_400M_v5) by Alibaba. The details about the retrieve task and the s2s task
The model I am talking about is hosted here : From the documentation: We simplify usage of prompts, providing two prompts for most general tasks, one is for s2p, another one is for s2s.Prompt of s2p task(e.g. retrieve task): ..., Prompt of s2s task(e.g. semantic textual similarity task): ... Question: What is S2P task? How is retrieve task different from semantic textual similarity task? Difference in the usage and also in the training steps would be much appreciated in the answer
Is the implementation equivalent to the formulation of the estimate of the Policy Gradient from spinningup?
In Part 3: Intro to Policy Optimization from spinningup documentation, there is a formula to compute the estimate of the policy gradient: This is an expectation, which means that we can estimate it with a sample mean. If we collect a set of trajectories $ D = > \{\tau_i\}_{i=1,...,N} $ where each trajectory is obtained by letting the agent act in the environment using the policy $\pi_{\theta}$ , the policy gradient can be estimated with $$ \hat{g} = \frac{1}{|D|} \sum_{\tau \in D} \sum_{t=0}^{T} > \nabla_{\theta} \log \pi_{\theta}(a_t |s_t) R(\tau), $$ where $|D|$ is the number of trajectories in $D$ (here, $N$ ). Also in the code on the same page to compute loss: # make loss function whose gradient, for the right data, is policy gradient def compute_loss(obs, act, weights): logp = get_policy(obs).log_prob(act) return -(logp * weights).mean() Here the mean() is calculated over all the (state, action) pairs. However, the formula above calculates the mean over trajectories. There seems to be a disconnect. Are these two forms equivalent?
"repeated measures ANOVA" vs "ANOVA with subject as random effect" vs "repeated measures ANOVA with subject as random effect"
What is the difference between running a repeated measures ANOVA (the measure is repeated across time for each subject) vs an ANOVA with a subject as a random effect?` I have also seen mention of a "repeated measures ANOVA with subject as random effect". For context I have 6 groups and 3 post-baseline time points per group/subject. Each subject is a member of one group. I would like to compare means of both different groups at the same time point (ex group 1 vs group 3 at time point B) and means of the same group at different time points (ex. group 1 at time point B vs group 1 at time point C). I will use ls-means to compare means, but I don't really understand the differences between the above three model categories. If there even is a difference! Thank you very much!
AI companies are pivoting from creating gods to building products. Good.
Turning models into products runs into five challenges
Humanoid Robots on the Rise: Industry Advances, Key Players, and Adoption Timelines
The robotics industry stands on the brink of a significant transformation, with many experts – including NVIDIA CEO Jensen Huang – suggesting that we might be approaching a "ChatGPT moment" for robotics. The post Humanoid Robots on the Rise: Industry Advances, Key Players, and Adoption Timelines appeared first on TOPBOTS .
Utilizing Symbolic Regression to discover a larger class of splits for Decision Trees
This work is based on the paper: K.S. Fong and M. Motani, “Symbolic Regression Enhanced...
Any relation between two KL divergences?
I am using a KL divergence to measure the difference between distributions, but I would like to compare the results to each one another, because the probability distributions I'm measuring are correlated. What is a good measure for comparing two KL divergences on related distributions? Ex. If I'm looking at some characterization data for a test, let's say X-ray diffraction patterns for a metal sample. I get an XRD pattern of intensity vs. diffraction angle, and can create a probability distribution for that in a reference state (room temp, new sample). I then run the same XRD test on the same sample while I vary temperature, and for that temperature I get a new distribution of my XRD data, and I can calculate the KL divergence for that distribution relative to my reference one. Let's say I then cold-roll my metal sample, and run the same XRD experiment, and then calculate the KL divergence with the same reference state for those distributions. I have two KL divergence results, and I know that there are variables for the metal sample, temperature vs. strength, that are correlated. Is there any comparison technique between divergences to tell me how much one distribution diverged relative to another? Especially if the variables are interrelated? Edit: Changing the example to be more specific.
Lower bound of KL divergence of Gaussian mixture with Gaussian (univariate)
I'm interested in a non-zero Kullback-Leibler divergence lower bound between a Gaussian mixture $q$ and a standard Gaussian $p(x) = \mathcal N(x \mid 0,1)$ , both univariate : $D_{KL}(q \parallel p)$ . I'm aware of this question , but I conjecture that with some constraints the problem might become easier. My constraints: The Gaussian mixture is guaranteed to have mean zero The Gaussian mixture is guaranteed to have variance lower bounded by $\gamma^2$ . I skim over Hershey & Olsen (2007) , which mentions a pretty trivial Gaussian approximation of the KL divergence in Section 4, where $q$ is approximated by a Gaussian $\bar q(x) = \mathcal N(x \mid \mu, \sigma^2)$ , (here, $\mu \triangleq \mathbb E_q[x] = 0, \sigma^2 \triangleq \mathbb E_q[x^2] - \mathbb E_q[x]^2 \ge \gamma^2$ ); and then the target KL is approximated by $D_{KL}(\bar q \parallel p)$ . However, the paper says this is a poor approximation, and does not mention whether it's a bound either. In Melbourne et al. (2018) , the authors propose an upper bound on the entropy of a mixture distribution. Since $D_{KL}(q \parallel p) = -\mathbb H(q) - \mathbb E_q[\log p(x)]$ , I'm able to take advantage of it, provided that I know how to upper bound the expected log density term , which is outside my expertise again.
Bayesian hierarchical model for comparing reviewers
I am working on a research paper about reviewing people's expertise. I have 21 respondents, each answered 10 questions. Now I have asked 4 groups of reviewers (each group consists of 3 people) to grade the answers between 1 and 5 (5 best). I have used Fleiss' Kappa to measure inter-rater reliability between reviewers in the same group. Unfortunately one reviewer asked me to include a Bayesian hierarchical model to account for the variability among raters and to provide a probabilistic measure of rater consistency. This would allow for a more nuanced understanding of rater behavior and the inherent uncertainties in human judgment, which are oversimplified by the use of a linear scale in the current methodology. In my opinion it is an over-complication. I wanted to check whether people from different backgrounds can assess people in the same topic. I do not see any benefits from using Bayesian hierarchical model for comparing reviewers. I am trying to perform this test, but nonetheless I would like to respond to the reviewer POLITELY that this is not a use-case for this model and Fleiss kappa is sufficient. Could you help me, or prove me wrong?
AI in Finance Global Challenge Startup Grant Awardee
In June 2023, the Monetary Authority of Singapore (MAS) launched the 8th edition...
Mechanistic Anomaly Detection Research Update
Interim report on ongoing work on mechanistic anomaly detection
We Need Positive Visions for AI Grounded in Wellbeing
Introduction Imagine yourself a decade ago, jumping directly into the present shock of conversing naturally with an encyclopedic AI that crafts images, writes code, and debates philosophy. Won’t this technology almost certainly transform society — and hasn’t AI’s impact on us so far been
Leveraging Generative AI in Project Management
Introduction The project management landscape is undergoing a transformative shift, driven by the rapid advancements...
Open Source Automated Interpretability for Sparse Autoencoder Features
Building and evaluating an open-source pipeline for auto-interpretability
Learning to Generate Unbounded 3D Scenes from Image Collections
Introduction Scene generation has raised considerable attention in recent years, addressing the growing need...
Accelerate Your AI Skills: Essential Generative AI Courses for Developers
Generative AI is a rapidly evolving field with a plethora of fascinating applications, from creating realistic images and videos to generating human-like text and beyond. As the technology advances, the demand for skilled professionals who can harness the power of generative AI is growing exponentially. However, navigating the myriad of tutorials and courses available can […] The post Accelerate Your AI Skills: Essential Generative AI Courses for Developers appeared first on TOPBOTS .
Congratulations to Stanford AI Lab PhD student Dora Zhao for an ICML 2024 Best Paper Award!
Congratulations to Stanford AI Lab PhD student Dora Zhao for an ICML 2024 Best Paper Award for a paper from her work at Sony AI on: Measure Dataset Diversity, Don’t Just Claim It