AI/ML News & Innovations Hub

RMSprop approach applied to Q-learning for adaptive dynamic learning rate

I am new to this group, Anybody familiar with Q-learning algorithm and RMSprop approach ? i have a question regarding the application of RMSprop approach into Q-Learning to adapt dynamically the learning rate for each Q(s,a). I am confused how to compute the average square gradient in tabular Q-Learning. Is it E[g^2] = beta * E[g^2]+ (1-beta) * td_error**2. ? and what would be the formula to update Q(s,a) ? Is it Q(s,a)

Overview of all AI Act National Implementation Plans

This post gives an overview of the national authorities to be designated under the AI Act and what we know about the national implementation plans.

HELM Safety: Towards Standardized Safety Evaluations of Language Models

*Work done while at Stanford CRFM

Deep RL problem: Loss decreases but agent doesn't learn

I'm implementing a basic Vanilla Policy Gradient algorithm for the CartPole-v1 gymnasium environment, and I don't know what I'm doing wrong. No matter what I try, during the training loop the loss decreases (so the model is actually learning something), but the episode total reward also decreases until it reaches around 9-10 steps (which I imagine is the minimum number of steps needed to make the pole fall). So it's learning to do it bad! In this algorithm there are a few moving parts that influence each other, and I run out of ideas to see where the problem could be. For the discounted rewards I'm using $ Q_{k,t} = \sum_{i=0}{\gamma^{i-t} r_i} $ for $k$ = all episodes and $t$ = all steps in an episode. And for the loss: $ L = -\sum_{k,t}Q_{k,t}log\pi_{\theta}(a_t | s_t)$ The code is a mix from Maxim Lapan's Deep RL Hands-On book, Karpathy's Pong example (blog, code), and personal tweaks. Here's my code: import gymnasium as gym import torch from torch import nn import torch.nn.functional as F from torch.nn.init import xavier_uniform_ import numpy as np GAMMA = 0.99 LEARNING_RATE = 0.001 BATCH_SIZE = 4 DEVICE = torch.device('mps') class XavierLinear(nn.Linear): def __init__(self, in_features: int, out_features: int, bias: bool = True, device=None, dtype=None) -> None: super().__init__(in_features, out_features, bias, device, dtype) xavier_uniform_(self.weight) class VPG(nn.Module): def __init__(self, input_size, output_size): super(VPG, self).__init__() self.net = nn.Sequenti…

Is it possible to achieve both detection and image classification by training the model on the custom dataset?

I am beginning to work on a non-linear navigation system for educational videos as a part of my undergrad coursework project. As a part of it, I need to classify the unique frames (frame which is slide, handwritten, only figure, with both text and figure, only text, blank frame) and also need to segment (here also I have a question, is it a segmentation task or should I go with multi-object detection) the content of the frame (text, figure, title, figure title). I would like to do this using a single model trained on a single custom dataset. But I am not sure about the practicality of this. I have some questions regarding this: I am not sure how I should prepare the dataset for this task. Whether I could use annotation &its labels and class labels in the same dataset? Can I train the model on the dataset (if the answer to 1 is yes) only once to achieve both tasks? Do I need to make any other changes to the model except some changes to the final layer? Thank you in advance!

A New Chapter for fast.ai: How To Solve It With Code

fast.ai is joining Answer.AI, and we’re announcing a new kind of educational experience, ‘How To Solve It With Code’

Is there an error in Russell & Norvig (4th edition)'s claim about A* with a consistent heuristic?

In "AI: A Modern Approach" (Russell & Norvig, 4th edition), section 3.5.2 "A* search" includes this sentence on page 88: In addition, with a consistent heuristic, the first time we reach a state it will be on an optimal path, so we never have to re-add a state to the frontier, and never have to change an entry in reached . There are three claims here: The first time we reach a state it will be on an optimal path. We never have to re-add a state to the frontier. We never have to change an entry in reached . I believe that claims (1) and (3) are false. To see this easily, consider the heuristic $h(n) = 0$ for all $n$ . Clearly it is consistent. With this heuristic, A* is just Dijkstra's algorithm. But in Dijkstra's algorithm, the first time we reach a state (i.e. add it to the frontier, as defined on p. 72) it certainly might not be on an optimal path, and we may have to update its entry in reached (i.e. the parent pointer, as seen in the pseudocode in Figure 3.7 on p. 73) as soon as we discover a shorter path to it. So is this simply an error in the book, or am I missing something here?

Stock Market LSTM predictions request models

I am looking for more advanced models for my stock market software can anyone either point me in the right direction for the latest research on LSTM or others or any experts collab/provide models for me in this specific area? Models must be convertible to onnx format currently.

Using hill climbing to solve the polygonal obstacle problem

I will first explain the problem. We start at an initial node S in the 2D plane and our goal is to reach a goal node G. In order to get to the end node we have to get past polygonal obstacles. Here is an image: A solution path is a list of vertices starting at S and ending at G that form a path from S to G. We can start by giving the hill-climbing algorithm a user-generated path through the polygonal obstacles. What I am struggling to understand is how we generate neighbours? Because a neighbour also has to be a path that starts at S and ends at G. We have to perturb the path in some way to generate a neighbour. Any ideas? I have googled this but nothing useful comes up. An answer has been given, but I would like to understand how to implement such a method. In my head it is going to be really difficult to change the path in such a way that it remains valid.

What is the ideal hypothesis test for Bernoulli random variables? [closed]

If you have a two table tennis ball outputting machines acting as Bernoulli trials. The first machine gives a player 1000 balls with p=0.7 of being red and 0.3 yellow. The second gives a player 100 balls with an unknown probability of the Bernoulli. What hypothesis tests can be done to check the similarity of distributions between the first and second machine? How would the test be different if the first machine generates 1000 balls at random them the second machine generates 100 of the 1000 balls and you want to know how likely is it the second machine to create a subsample representative of the first machine sample?

How to evaluate performance of classification model for different subsets of classes?

Consider a classification problem where there are N classes. While this may seem strange, I have a model that processes features, and essentially, evaluate which classes are impossible (or near impossible) to classify correctly based on the features. Therefore, we can exclude these classes when making predictions because we already know we fail in such cases (in other words quantify when model is useless). I want to evaluate how well this, but simply evaluating accuracy isn't a fair comparison. Consider I am unable to classify all but 2, then I could compare the accuracy for the classes I "can classify" and "cannot classify". However, I will have 50% for the classes I "can classify" by just randomly guessing in this case. Maybe, it appears as a good improvement, but a random guesser has high accruacy relative to the number of classes. What metrics can be used to evaluate such cases? There must be a metric for this kind of scenario.

Third-party evaluation to identify risks in LLMs’ training data

An overview of the minetester and preliminary work

Why factor scores correlated with variables do not equal loadings?

I am trying to understand the following: I have three variables (x1, x2, x3) on which I did PCA and EFA (principal axis factoring) in SPSS to get one component/factor. If I do PCA and I calculate factor scores (regression method), which I call variable "F", then: Cor(x1, F)=component loading and the same for x2 and x3. If I do the same with PFA, then I obviously get different loadings and factor scores, but also: Cor(x1, F) is no longer equal to the loading (but only approximately). Could someone please explain why this is so? Correlations (n=12): x1-x2: 0.310 x2-x3: 0.153 x1-x3: 0.529 Output: Component loadings (PCA): x1: 0.863; x2: 0.564; x3: 0.790 Cor(x, component scores) reproduce exactly the component loadings. Factor loadings (PFA): x1: 0.956, x2: 0.313, x3: 0.548 Cor(x, Factor scores) do not reproduce the factor loadings, but: Cor(x1, F) = 0.998 Cor (x2, F) = 0.327 Cor (x3, F) = 0.573 Output SPSS when calculating factor loadings: "Attempted to extract 1 factors. More than 25 iterations required (convergence=0,006). Extraction was terminated".

Dubai Police's 24x7 Smart Home Security System: How to protect your home while travelling this summer

In defense of screen time

Pundits say my husband and I are parenting wrong.

AI Singapore launches AIAP Foundation, a new AI-powered online course to equip aspiring AI Engineers with fundamental AI and software engineering knowledge and skills

AI Singapore today announced the launch of AIAP Foundation, a new online course aimed at equipping aspiring AI engineers with a strong foundation in industry-relevant AI and software engineering...

Should I use an ordinal regression?

I have intention to return to teaching on a Likert scale 1-5 as my outcome variable and well-being and job satisfaction as continuous predictor variables. I have also data regarding age, sex, years out of teaching and current job. The data is positively skewed for the outcome variable at the moment (with most participants answering '1'). I am considering merging 1 and 2 into 'unlikely' categories, maintaining 3 as 'neutral' and merging 4 and 5 as 'likely' to make the groups more equal for an ordinal regression. Originally I was hoping to run a multiple regression but made a mistake with choosing a non-continuous outcome variable. What should I do now to make the chances of a good model fit more likely?

Should I interleave sin and cosine in sinusoidal positional encoding?

I'm trying to implement a sinusoidal positional encoding. I found two solutions that give different encodings. I am wondering if one of them is wrong or both are correct. I showcase visual figures of the resulting encodings for both options. Thank you :) class SinusoidalPosEmb(nn.Module): def __init__(self, dim): super().__init__() self.dim = dim def forward(self, x): device = x.device half_dim = self.dim // 2 emb = math.log(10000) / (half_dim - 1) emb = torch.exp(torch.arange(half_dim, device=device) * -emb) emb = x[:, None] * emb[None, :] emb = torch.cat((emb.sin(), emb.cos()), dim=-1) return emb 2) class TransformerPositionalEmbedding(nn.Module): """ From paper "Attention Is All You Need", section 3.5 """ def __init__(self, dimension, max_timesteps=1000): super(TransformerPositionalEmbedding, self).__init__() assert dimension % 2 == 0, "Embedding dimension must be even" self.dimension = dimension self.pe_matrix = torch.zeros(max_timesteps, dimension) # Gather all the even dimensions across the embedding vector even_indices = torch.arange(0, self.dimension, 2) # Calculate the term using log transforms for faster calculations # (https://stackoverflow.com/questions/17891595/pow-vs-exp-performance) log_term = torch.log(torch.tensor(10000.0)) / self.dimension div_term = torch.exp(even_indices * -log_term) # Precompute positional encoding matrix based on odd/even timesteps timesteps = torch.arange(max_timesteps).unsqueeze(1) self.pe_matrix[:, 0::2] = torch.sin(timesteps * div_t…

Mechanistic Anomaly Detection Research Update 2

Interim report on ongoing work on mechanistic anomaly detection

RLHF and RLAIF in GPT-NeoX

GPT-NeoX now supports post-training thanks to a collaboration with SynthLabs.

Iteratively Reweighted Least Squares (IRLS) and Gauss-Newton

I am studying nonlinear regression optimization methods. I want to show that the Gauss-Newton method can be seen as an IRLS. We want to maximize $$ - \sum_{i = 1}^{n} (y_i - h(\mathbf{x}_i^T \mathbf{\beta}))^2 $$ Approximating $h(\mathbf{x}_i^T \mathbf{\beta})$ we have $$ - \sum_{i = 1}^{n} [y_i - h(\mathbf{x}_i^T \mathbf{\beta}^{(t)}) - h'(\mathbf{x}_i^T \mathbf{\beta}) \mathbf{x}_i^T (\mathbf{\beta} - \mathbf{\beta}^{(t)}) ]^2 $$ and here I don't know what to do. The books I have read do not show what to do when you have any function $h$ , or that explain the Gauss-Newton method in a direct way. I would appreciate any kind of help or suggestions. I am also looking for references on the subject in this regard.

FAQ about the book and our writing process

What's in the book and how we wrote it

Llama 3.2 Vision-Instruct Inference Speed on A100 or H100 GPU

Can anyone provide an estimated time of how long does it take for Llama-3.2 Vision-Instruct 11-B model to: process an image size of 1-MB and prompt size of 1000 words and generate a response of 500 words The GPUs used for inference could be A100, A6000, or H100.

Challenges in Aggregating Outputs from Classifiers Trained on Subsets of Classes

I’m currently working on a project involving several classifiers, each trained on a subset of classes. These classifiers are designed to handle different aspects of the classification task, but I’m facing a challenge when it comes to aggregating their outputs into a single prediction. For example, if one classifier is responsible for distinguishing between classes 0 and 1, and another handles classes 2 and 3, how can we effectively combine their results when the correct answer belongs to class 1? Our initial approach was to use an "other" class to indicate when an input doesn’t belong to a classifier’s assigned classes, but this did not yield the desired results. We are now exploring the possibility of implementing an additional head for detecting out-of-distribution classes, but we’re looking for a more efficient and streamlined solution. Has anyone encountered a similar issue or have any suggestions for effectively aggregating outputs from multiple classifiers? Thank you for your assistance!

Batch LLM Inference on Anyscale slashes AWS Bedrock costs by up to 6x

Announcing RayTurbo

Autoscaling Large AI Models up to 5.1x Faster on Anyscale

Accelerated Metadata Fetching in Ray Data up to 4.5x Faster on Anyscale

Ray Data GA

Anyscale on Kubernetes: Simplifying AI Workloads on User-Managed Infrastructure

Enhancing Data Quality Using Better Designed ETLs

Sharing an ETL design doc template

First AI Certification for Design & Media Practitioners Launched

In a bold initiative that merges technology with creativity, Nanyang Polytechnic (NYP) and AI Singapore (AISG) have joined forces to introduce the Certified AI Practitioners for Design & Media...

Separating code reasoning and editing

An Architect model describes how to solve the coding problem, and an Editor model translates that into file edits. This Architect/Editor approach produces SOTA benchmark results.

BadEdit: Backdooring Large Language Models By Model Editing

Large Language Models (LLMs) exemplified by ChatGPT continue to gain widespread usage in addressing a...

Low validation loss from the first epoch?

The initial validation loss is low from the first epoch and then decreases slightly. What does this actually mean? Does it indicate that the model can effectively and quickly identify patterns for this task? I can see that the model works in practice, but the results (some image restoration) aren’t ideal yet, so I want to improve its performance even further. Given this low loss from the first epoch, should I focus on training with more data or on adjusting the architecture and layers to be even more complex, etc.? Given the small differences between the first epoch and the last, is it more likely that the model was barely able to improve performance during these epochs, or could the difference in loss still be meaningful? The dataset count was of 10,000 images - 0.9 for training, 0.1 for validation. First epoch loss: Epoch [1/50], Training Loss: 0.026428, Validation Loss: 0.023727 Last epoch and plateau: Epoch [34/50], Training Loss: 0.020682, Validation Loss: 0.020651

The Practitioner's Guide to the Maximal Update Parameterization

Exploring the implementation details of muTransfer

Can AI automate computational reproducibility?

A new benchmark to measure the impact of AI on improving science

I'm trying to train an AI, but I have low accuracy using Rust and PyTorch

I'm just starting out in the world of machine learning, and I really like Rust. I've been testing and learning more. I took the example of transfer training and did some tests, but I can't understand why I have high accuracy in training and low accuracy in testing using the same validation base. Why? I studied overfit, but it doesn't seem to be the case, because I'm using the same validation base without new data. use std::env; use std::error::Error; use std::path::PathBuf; use anyhow::{ bail, Result }; use tch::nn::{ self, ModuleT, OptimizerConfig, VarStore }; use tch::vision::{ imagenet, resnet }; use tch::{ Device, Kind, Tensor }; pub fn bee_test() -> Result > { tch::manual_seed(123); let manifest_dir = env::var("CARGO_MANIFEST_DIR")?; let project_dir = PathBuf::from(manifest_dir); let dataset_path = project_dir.join("data/hymenoptera_data"); let dataset = imagenet::load_from_dir(dataset_path)?; println!("{dataset:?}"); let model_path = project_dir.join("data/bee.ot"); println!("Caminho do modelo: {:?}", model_path); let device = Device::cuda_if_available(); let mut vs = VarStore::new(device); vs.load(model_path.as_path()).map_err(|op| { format!("Erro ao carregar o modelo: {:?}", op); op })?; let net = resnet::resnet34_no_final_layer(&vs.root()); let linear = nn::linear(vs.root(), 512, 2, Default::default()); let net2: nn::Sequential = nn ::seq() .add_fn(move |xs| net.forward_t(xs, false)) .add(linear); let predicted = net2.forward_t(&dataset.test_images, false); let prob…

o1-preview is SOTA on the aider leaderboard

Preliminary benchmark results for the new OpenAI o1 models.

Fine-tuning Llama-3, Mistral and Mixtral with Anyscale

Start reading the AI Snake Oil book online

The book was published September 2024

From Code to Robots: The Top AI Trends Transforming Business and Life

Artificial intelligence is no longer a concept of the distant future – it’s here, evolving at a rapid pace and reshaping industries in real time. From healthcare to entertainment, AI’s influence is everywhere, sparking innovation, efficiency, and even ethical debates. But with so much happening at once, where exactly is the industry heading? To make […] The post From Code to Robots: The Top AI Trends Transforming Business and Life appeared first on TOPBOTS .

Towards Robust and Expressive Whole-body Human Pose and Shape Estimation

Introduction Given a single RGB image, 3D human pose and shape estimation aims to reconstruct...

What's Missing From LLM Chatbots: A Sense of Purpose

LLM-based chatbots’ capabilities have been advancing every month. These improvements are mostly measured by benchmarks like MMLU, HumanEval, and MATH (e.g. sonnet 3.5, gpt-4o). However, as these measures get more and more saturated, is user experience increasing in proportion to these scores? If we envision a future

Advancing Customizable Benchmarking in HELM via Unitxt Integration

The Holistic Evaluation of Language Models (HELM) framework is an open source framework for reproducible and transparent benchmarking of language models that is widely adopted by academia and industry. To meet HELM users’ needs for more powerful benchmarking features, we are proud to announce our collaboration with Unitxt, an open-source community platform developed by IBM Research for data preprocessing and benchmark customization. The integration of Unitxt into HELM gives HELM users access to the vast Unitxt catalog of benchmarks, and allows users to run sharable and customizable evaluation pipelines with greater ease.

Scaling Embedding Generation Pipelines From Pandas to Ray Data

Breaking Barriers, Building Bridges: Increasing Language Representation in Southeast Asia

Held outside of Singapore for the very first time, the third Languages Summit...

Réinventer l’éducation : quand le numérique transforme les apprentissages

Réinventer l’éducation : quand le numérique transforme les apprentissages mtestari lun, 09/02/2024 - 09:00 La transformation numérique de l’école doit être une opportunité pour mieux apprendre et individualiser l’apprentissage grâce au développement de nouvelles technologies associées à la formation de la communauté enseignante et des élèves aux compétences du 21e siècle. Autant d’enjeux sociétaux et scientifiques que relèvent, depuis plus de dix ans, les équipes pluridisciplinaires du Centre Inria de l’université de Bordeaux avec des chercheurs et chercheuses en intelligence artificielle, en interaction Humain - Machine, en neurosciences, en psychologie développementale et en sciences de l’éducation. © freepik /Photo Pch.Vector Mieux comprendre les processus d’apprentissage L’une des caractéristiques la plus importante pour l’ensemble des équipes impliquées reste leur approche systémique, positionnant l’apprenant toujours au centre de l’étude en considérant l’ensemble de ses interactions avec l’environnement dans lequel il évolue. « Ces recherches ne peuvent être menées sans l’appui des acteurs de terrain que sont les enseignants et les professionnels de l’éducation. Elles restent indissociables des avancées des sciences humaines pour ancrer la transformation numérique de l’école sur les besoins des apprenants, des éducateurs et de leurs environnements » souligne Nicolas Roussel, directeur du Centre Inria de l’université de Bordeaux. Le premier défi consiste à mieux compren…

Uplift Modeling X-Learner - One model for all treatments or multiple models for each treatment?

I am using an x-learner (and doubly robust learner as well) for uplift modeling. I have a control group and 10 treatments. To start, I have just be creating one x-learner and passing it all the treatments. But now I'm thinking -- would it be better to have 10 separate x-learners, each modeling a different treatment against the control? What are advantages/disadvantages to one metalearner with all the treatments thrown in vs. 10 metalearners with one per treatment?

Clean-image Backdoor: Attacking Multi-label Models with Poisoned Labels Only

Multi-label learning Multi-label learning is commonly exploited to recognize a set of categories in an...