AI/ML News & Innovations Hub

Block size in subsampling and bootstrap for time series

I have a dependent variable, a time series of 80 periods (discrete decisions). I am doing maximum likelihood estimation with 10 parameters. Now I want to get the standard error or confidence interval of the estimates of these 10 parameters. One feature of my likelihood function is that the decisions is determined by all the history of $x$ , and the weight of past $x$ decreases geometrically: $x_t+\rho x_{t-1}+\rho^2 x_{t-2}$ ... where $\rho$ is one of the parameters needed to be estimated. So I am thinking that moving block bootstrapping perhaps is not suitable to sustain the data structure, and I should use subsampling. But subsampling of a given block size leaves me a very few subsamples. For example, if I choose a block size of 40, I get only 41 subsamples. Should I concern about it? Is it sufficient? Can I use multiple block sizes to construct a confidence interval? Is there any other alternatives that I could use to get standard error or confidence interval?

Deep learning for single-cell sequencing: a microscope to see the diversity of cells

On the the pivotal role that Deep Learning has played as a key enabler for advancing single-cell sequencing technologies.

How does one annotate overlapping objects in instance segmentation?

As I struggle to find any literature online about this, I wanted to ask about it here so that others could learn. My question is inspired by a Yolo GitHub issue . In this example we have 2 objects, here a plate and an egg, with one object being inside the other one. The question is how to annotate the plate (aka the outer object). Annotate full contour of outer object. Some pixels belong to 2 classes. Make a little bridge to the inner object so that the contour of the outer object excludes the inner object. This question arose while using Yolo but can be extended to other instance segmentation models. Any more information regarding good practices in instance segmentation is more than welcome.

Comment on The Future of Customer Support: How AI is Boosting Capacity, Performance, and Satisfaction by AI Worthy

Great share Kate. To be honest, I've never really thought there was much potential for AI customer service, however after using ChatGPTs new voice chat and just how natural it sounds, I was totally blown away! -Will

Comment on State-Of-The-Art Approaches to Attribution in Marketing by Bay tech media

In the realm of digital marketing, attribution methodologies have undergone significant advancements. State-of-the-art approaches include Multi-Touch Attribution (MTA) for holistic channel tracking, Algorithmic Attribution leveraging machine learning for precise credit assignment, Cross-Device Attribution capturing interactions across devices, Incrementality Testing to gauge true marketing impact, and AI-Powered Attribution for deep data analysis. Bay Tech Media implements these cutting-edge methods, empowering businesses with accurate insights to refine and optimize their marketing strategies effectively.

Comment on The Point Of Play: How Games Make AI Smarter by Mark H Black

Nowadays there are so many platforms and different technologies that make my work process easier, but even so, I still want to do nothing from time to time.

Comment on How To Get Started With Graph Machine Learning by Alex

Quite a detailed and interesting article, thanks for such a detailed description

Comment on Top AI & ML Research Trends For 2020 by Red Carson

Thanks for sharing this post

Canonical correlation analysis - loadings vs coefficients

I'm trying to wrap my head around how to interpret the results of CCA. I've got a fairly deep understanding of OLS regression, and I've read a lot of helpful CCA explainers like this one by @ttnphns . However, I'm still struggling with one particular aspect of the logic of what one apparently does with the CCA results. I'll unpack below, using terminology from the R package CCA to refer to different elements. In particular, I understand that the math of CCA treats X and Y identically, i.e., this is correlation, not regression. But, in a situation where Y is logically downstream of X, and where Y comprises multiple theoretically independent outcomes, the idea of using the ycoef s, which are essentially regression coefficients specifying the linear combination of y s that produce a given yscore , and which, like OLS regression, reflect the joint influence of the given y and all the other y s , doesn't make sense to me. Again, I understand the math of how a given yscore is derived, and how that is reflected in the ycoef s. What I don't like is the idea of reporting how the various y s 'contributed to' constructing this synthetic latent variable in a regression sense, because in reality, all the y s arose independently—or, more in keeping with the logic of CCA, they were all driven by some set of latent variables. What makes sense to me would be to report the xcoef s alongside the corr.Y.yscores , that is, the coefficients for how each x relates to a given xscore , and the loadi…

Salmon in the Loop

On fish counting – a complex sociotechnical problem in a field that is going through the process of digital transformation.

Multiple linear regression with possibly non-independent explanatory variables

For a given household for which I have many years of historical data, I want to predict the home gas consumption (heating) with a few variables among: date gas min_temp max_temp mean_temp relative_humidity absolute_humidity other_column 2023-01-01 5.8 m^3 -3.0°C 2.3°C -1.2°C 79 % 4 g/m^3 ... 2023-01-02 4.8 m^3 2.0°C 4.2°C 2.3°C 82 % 4.5 g/m^3 ... ... I could do a multiple linear regression for $$\rm{gas\ consumption} = \beta_0 + \beta_1 \rm{min\ temp} + \beta_2 \rm{max\ temp} + \beta_3 \rm{mean\ temp} + ... + \varepsilon,$$ but since many of these variables are not independent of each other (and maybe nearly collinear), doing a standard multiple linear regression might give bad results (for example with some negative $\beta_i$ where it shouldn't). Which better solution can we use? PCA + multiple linear regression (PCR) or PLS or something else? Note: I'd like to avoid using all 3 (min, max, mean) temp, if possible. How can we evaluate the loss if using using only 1 temperature variable (the best fit among the 3) instead of the 3 variables?

A new old kind of R&D lab

Answer.AI is a new kind of AI R&D lab which creates practical end-user products based on foundational research breakthroughs.

Transformers: Cross Attention Tensor Shapes During Inference Mode

Using the "classic" transformer model describing in "Attention is All You Need", I'm struggling to understand how the Encoder output is used by the Decoder during cross attention while in inference mode, specifically how the actual matrix multiplication can happen. During training mode, everything makes sense to me: The Encoder outputs a tensor of shape (B, T, C) where B = batch_size, T = max_tokens, and C = d_model = embedding dimension size. This is passed to the Decoder and changed to shape (B, T, T) through the scaled dot product mechanism (will call this tensor A ) A is multiplied by the Decoder's value tensor of shape (B, T, HS) where HS = depth = head size. This multiplication is possible because the shapes of the tensors comply (B, T, T) @ (B, T, HS) --> (B, T, HS) . But in inference mode we start with a Decoder value tensor that will only have a token length of 1 , so a tensor shape of (1, 1, HS) , where T != 1, and then expand the sequence from there. So, during the cross attention step with the Encoder, how can A with shape (1, T, T) be multiplied with (1, 1, HS) ? Clearly, I'm missing something pretty big here, so any help would be much appreciated!

Outlier Detection and Removal

I am reading a paper on wind power forecasting and the authors present a plot of the data before outliers are removed and a plot after. However, they don't actually say what method was employed to remove the outliers. I was hoping someone might offer some guesses or hints on how one would go about obtaining plot (b) from plot (a). Edit The paper is here: One-Day-Ahead Hourly Wind Power Forecasting Using Optimized Ensemble Prediction Methods by Chao-Ming Huang 1, Shin-Ju Chen, Sung-Pei Yang and Hsin-Jen Chen

Comment on Beyond Conventional Tactics: How AI is Redefining Marketing and Sales by AI

Thank You Mariya Those all are interesting ideas on how to improve marketing with AI. What I am missing is for someone without an AI and programming knnowledge is which softwares are already usingn similar techniques so I can test them on our marketingn data. Thank You!

Comment on Step-By-Step Implementation of GANs on Custom Image Data in PyTorch: Part 2 by Mel Dorn

You got my attention. Learning Image Data and its role very interesting for me. As a photographer, I am constantly looking for information that can help grow my professionalism. I also couldn’t ignore this article offers valuable insights into perfecting nature photography, enhancing my appreciation for this art form.

Comment on Unconstrained Chatbots Condone Self-Harm by James Ruiz

How do you find motivation during difficult times, especially if you have depression or anxiety?

Gabo Foundation to host master class in Spanish on AI opportunities for investigative journalism

"The Gabo Foundation invites to a master class with journalist Emilia Díaz-Struck, executive director of the Global Investigative Journalism Network (GIJN), on Nov. 8. Díaz-Struck will explore artificial intelligence (AI) opportunities for investigative journalism, the editorial processes related to data management and advanced technology, as well as reporting and fact-checking. Challenges and risks related to […] The post Gabo Foundation to host master class in Spanish on AI opportunities for investigative journalism appeared first on LatAm Journalism Review by the Knight Center .

Gabo Foundation to host master class in Spanish on AI opportunities for investigative journalism

"The Gabo Foundation invites to a master class with journalist Emilia Díaz-Struck, executive director of the Global Investigative Journalism Network (GIJN), on Nov. 8. Díaz-Struck will explore artificial intelligence (AI) opportunities for investigative journalism, the editorial processes related to data management and advanced technology, as well as reporting and fact-checking. Challenges and risks related to […] The post Gabo Foundation to host master class in Spanish on AI opportunities for investigative journalism appeared first on LatAm Journalism Review by the Knight Center .

Comparing the change in proportions across two time periods for two groups

I am running a test across two groups over a period of time and want to understand if the Change in proportions for my test group is significantly different to the change in proportions in the control group. Most of what I have found compares one proportion to another either over group or over time but not both. If I have: my Test group with proportions P1 and P2 with sample N1 and N2 (in periods T1 and T2) my control group with proportions Q1 and Q2 with sample M1 and N2 (in periods T1 and T2) I want to know if P2-P1 is statistically significantly different to Q2-Q1 P1,P2,Q1, Q2 are all percentages. Any help would be most appreciated!

Comment on What Do You Need To Do Before Hiring A Data Scientist? by James Ruiz

What are the best security practices in the SaaS industry and why is maintaining high security standards important for a business?

Adversarial Attacks on LLMs

The use of large language models in the real world has strongly accelerated by the launch of ChatGPT. We (including my team at OpenAI, shoutout to them) have invested a lot of effort to build default safe behavior into the model during the alignment process (e.g. via RLHF ). However, adversarial attacks or jailbreak prompts could potentially trigger the model to output something undesired. A large body of ground work on adversarial attacks is on images, and differently it operates in the continuous, high-dimensional space. Attacks for discrete data like text have been considered to be a lot more challenging, due to lack of direct gradient signals. My past post on Controllable Text Generation is quite relevant to this topic, as attacking LLMs is essentially to control the model to output a certain type of (unsafe) content.

Building RAG-based LLM Applications for Production

Third United Nations Conference on Landlocked Developing Countries (LLDC3)

Third UN Conference on Landlocked Developing Countries Awaza, Turkmenistan 5-8 August 2025 Driving Progress through partnerships The United Nations General Assembly (UNGA) decided to convene the Third United Nations Conference on Landlocked Developing Countries (LLDC3) from 5-8 August in Awaza, Turkmenistan at the highest possible level, including Heads of State and Government, on the theme “Driving progress through partnerships”. Stakeholder Participation in the Conference: As guided by resolution A/RES/77/246 , other relevant stakeholders, including the non-governmental organizations, civil society organizations, academic institutions and the private sector whose work is relevant to the Conference are invited to participate as observers in the Conference and its preparatory meeting. For additional information, please visit the Conference website: 3rd UN Conference on Landlocked Developing Countries (LLDC3) Relevant stakeholders who have received special accreditation to any of the conferences and summits listed below (check the consolidated list here ) may participate in the conference by following the REGISTER NOW link here . The registration deadline is 12 July 2025. The Global Conference on the Sustainable Development of Small Island Developing States The International Meeting to Review the Implementation of the Programme of Action for the Sustainable Development of Small Island Developing States The third International Conference on Small Island Developing States The f…

Neural algorithmic reasoning

In this article, we will talk about classical computation : the kind of computation typically found in an undergraduate Computer Science course on Algorithms and Data Structures [1]. Think shortest path-finding, sorting, clever ways to break problems down into simpler problems, incredible ways to organise data for efficient retrieval and updates.

Multimodality and Large Multimodal Models (LMMs)

For a long time, each ML model operated in one data mode – text (translation, language modeling), image (object detection, image classification), or audio (speech recognition). However, natural intelligence is not limited to just a single modality. Humans can read, talk, and see. We listen to music to relax and watch out for strange noises to detect danger. Being able to work with multimodal data is essential for us or any AI to operate in the real world. OpenAI noted in their GPT-4V system card that “ incorporating additional modalities (such as image inputs) into LLMs is viewed by some as a key frontier in AI research and development .” Incorporating additional modalities to LLMs (Large Language Models) creates LMMs (Large Multimodal Models). Not all multimodal systems are LMMs. For example, text-to-image models like Midjourney, Stable Diffusion, and Dall-E are multimodal but don’t have a language model component. Multimodal can mean one or more of the following: Input and output are of different modalities (e.g. text-to-image, image-to-text) Inputs are multimodal (e.g. a system that can process both text and images) Outputs are multimodal (e.g. a system that can generate both text and images) This post covers multimodal systems in general, including LMMs. It consists of 3 parts. Part 1 covers the context for multimodality, including why multimodal, different data modalities, and types of multimodal tasks. Part 2 discusses the fundamentals of a multimodal system, using the…

How to deal with a Stationary DV and a Trend-Stationary IV in using OLS?

I have a dependent variable that is stationary in levels. However, one of the IVs is only trend-stationary (stationary around a deterministic trend that I can extract from the series). In other words, I have a regression with variables that have different transformations. My question is, does this bias the significance of the results, including the significance of any other IVs (that are stationary in levels in the regression)? If not, it is probably reasonable to assume that it biases the estimated coefficient of the DP on the trend-stationary IV but not the significance of the effect since it shouldn't impact the standard errors (due to the trend being deterministic). Is that an accurate assessment? Also, what if I detrend (i.e. extract a deterministic trend) from my stationary DV? Is this equivalent to over differencing and thus losing important information from the series? Does extracting a deterministic trend from a stationary series even make sense?

The Artificiality of Alignment

This essay first appeared in Reboot . Credulous, breathless coverage of “AI existential risk” (abbreviated “x-risk”) has reached the mainstream. Who could have foreseen that the smallcaps onomatopoeia “ꜰᴏᴏᴍ” — both evocative of and directly derived from children’s cartoons —

How to force Transformer to give more weight to certain tokens

I'm developing an encoder-decoder based transformer model and I would like to ask if there are ways to incentivize or penalize certain tokens during training. I'm working on a translation task where the encoder input must be decoded into its proper product name. I have labels such as brand, name, and unit of measure, etc which are available during training but not on inference. Currently when predicting the brand portion (which usually appears early in the sequence) of the output, the heatmap shows that it does not give focus to the latter part of the encoder which produce an output that the brand and product name, and unit of measure does not belong to each other. I was thinking if there's a way to force the transformer during training to give more weight to different token types other that its own. For example: Brand tokens (decoder) should give more weight to name tokens (encoder) than other brand tokens (encoder) Name tokens (decoder) should give more to brand token (encoder) and unit of measure token (encoder)

Ray Serve: Tackling the cost and complexity of serving AI in production

Announcing Ray Serve and Anyscale Services general availability. Update June 2024: Anyscale Endpoints (Anyscale's LLM API Offering) and Private Endpoints (self-hosted LLMs) are now available as part of the Anyscale Platform. Click [here](https://console.anyscale.com/?utm_source=anyscale&utm_medium=blog&utm_campaign=blog_callout&utm_content=june2024_product_update_subheading) to get started on the Anyscale platform.

Why does the policy gradient theorem have two different forms?

I have been studying policy gradients recently but found different expositions from different sources, which greatly confused me. From the book "Reinforcement Learning: an Introduction (Sutton & Barto Chapter 13)", we get the following policy gradient: $$ \nabla J(\theta) = \mathbb E_\pi\left[G_t\nabla\log\pi(A_t | S_t, \theta)\right]. $$ As we can observe from the equation, it does not relate to trajectory distributions . However, a more intuitive and widely-used introduction to policy gradient starts from defining the distribution of trajectories: $p(\tau)$ . For example, in OpenAI Spinning Up , the policy gradient has the form similar to the following equation: $$ \nabla J(\theta) = \mathbb E_{\tau \sim \pi}\left[\sum_{t=0}^{T}G_t\nabla_\theta\log\pi_\theta(a_t | s_t)\right]. $$ The confusion comes from the fact that the first policy gradient does not have a summation over timestamps and is not sampling from trajectories, but the second samples from trajectories and has a summation. I did find some relevant questions about this confusion, but none of them seemed to have a good answer. Also, I could not identify any source that explained the difference/connection between the two forms. My question is why are there two different ways to describe the policy gradient and are the two forms mathematically equivalent? Update I found a great RL theory book (draft) written by some expert Professors in this field that shows two different formulations: https://rltheorybook.github.io…

#UNGA78 - Wrap Day 4

Groups audience: General Assembly 78 Live Blog Video embed a la iseek:

Until we meet again

In a few days, world leaders will head back to capitals and barricades will be dismantled, but the General Assembly keeps working. Global threats don’t stop at the end of the high-level week, and neither does the search for responses. Countries continue grappling with the problems affecting us all: from the outer reaches of space to the seabed floor. And while this blog comes to a close, there’s always time to open our many platforms from websites to social media to the UN News App. مع السلامة 再见 See you later Au revoir До свидания Adios Groups audience: General Assembly 78 Live Blog

last day

Groups audience: General Assembly 78 Live Blog

UN tour guides

Christian Dior designed the iconic UN tour guide uniforms in the early 1980s. Prior to that, designers included Edith Head and, for the UN’s first male tour guides, Brooks Brothers. The guides got a fashion refresh in 1985 by Harvé Benard, then changed their uniforms, courtesy of Benneton, and Mondrian. Image: Groups audience: General Assembly 78 Live Blog

China

Vice-President of China, Han Zheng, said in his speech to the GA said: "China supports all efforts that are conducive to the peaceful resolution of the Ukraine crisis, and stands ready to continue playing a constructive role for the early attainment of peace." Image: Groups audience: General Assembly 78 Live Blog

Sudan

"Since 15 April, Sudanese people have been facing a destructive war launched by rebel RSF," said Abdel-Fattah Al-Burhan Abdelrahman Al-Burhan, President of the Transitional Sovereign Council of Sudan. "We call upon the international community to designate these groups and their allies as terrorist groups to be countered by arms and fought to protect Sudan, region and the entire world." Image: Groups audience: General Assembly 78 Live Blog

Johan Santana GA78

Happening now at the SDG Media Zone: Johan Santana, young innovator who codes to address SDG 3 and 10 (speaking). He expresses his desire to assist individuals with visual impairments and other disabilities. They've developed a hands-free blind cane and smart glasses and will work on coding a wheelchair capable of moving in all directions: forward, backward, left, and right. Also in the picture: Michael Melillo, Sr. Director of Network Monitoring and Management Software Products, Broadcom Inc.(left) Bervin Harris, Renaissance Youth Center (second from left). Image: Groups audience: General Assembly 78 Live Blog

International Seabed Authority

Michael Lodge (Secretary General, International Seabed Authority) and Emilie McGloane (Director, Peace Boat US) met at the SDG Media Zone to discuss the importance of a new historic treaty - the Treaty of the High Seas - which protects biodiversity beyond national jurisdiction (often shortened to BBNJ). Michael Lodge emphasized the vastness of oceans and said that “1% of oceans contain more minerals than exist on land”. He continued: "Good regulation is important to ensure that the ocean’s resources are not exploited". So far, some 60 UN Member States have signed onto the treaty since its opening this week at UNGA78. Image: Groups audience: General Assembly 78 Live Blog

Renaissance Youth Center GA78

The Renaissance Youth Center choir visits the SDG Media Zone for a session on creative problem-solving and coding for the SDGs. The session explores the passion and enthusiasm of today’s youth to solve the challenges they face through creativity in STE(A)M and showcases successful outcomes when the private sector joins forces with NGOs to educate, inspire and empower youth as future STEM leaders in their communities and the world. Image: Groups audience: General Assembly 78 Live Blog

Intuition Behind the Gradual Increase of Noise Variance in Diffusion Models

I've been studying diffusion models and came across the noise schedule, particularly how the noise variance $\beta_t$ is adjusted over iterations. I've observed that $\beta_t$ typically starts from a very small value during the initial steps and increases to a much larger value in the final steps. What is the underlying reason for this progression? Why is it important to begin with a low noise variance and end with a high one? I'd appreciate any intuitive explanations or references that shed light on this choice.

Fine-Tuning LLMs: LoRA or Full-Parameter? An in-depth Analysis with Llama 2

Can LLMs learn from a single example?

We’ve noticed an unusual training pattern in fine-tuning LLMs. At first we thought it’s a bug, but now we think it shows LLMs can learn effectively from a single example.

What strategy does ChatGPT use to manage its context in very lengthy conversations?

I'm asking specifically about ChatGPT4, but the question could apply to either that or 3.5. When you use the ChatGPT API, it's of course up to you to manage conversation history and include that in successive API calls within available context length in whatever manner you choose. In the case of the web interface, they've obviously implemented some system to manage conversation history in context. It clearly doesn't "remember" the entire thing once the conversation gets very long, because it doesn't have infinite context length. So, what strategy does it use to send conversation history to the model once it's exceeded its context length? Does it truncate all content prior to the max context length? Does it summarize earlier parts of conversations to more efficiently fit them within the context? Does it do some dynamic strategy combining many inputs? Or is this just another case where we just don't know, and OpenAI is being tight-lipped about what it's actually doing?

Appropriate statistical test to determine if uplift between control group and multiple test groups is significant (pretest/posttest evaluation)

I'm trying to evaluate whether the difference in uplift seen in below table between the test groups and the control group is statistically significant. I'm unsure about the appropriate statistical test. The test and control groups are all of different sizes already before the test, which is why I have included the relative numbers. I first thought about the chi-square test, but that don't think it captures the pre- & post-treatment aspect correctly. For context: Four different geographic regions were selected, one of them as control. Each test region received a different mix of marketing measures with the goal to raise awareness. I am now trying to evaluate whether the uplift in the test regions is statistically different from the control region. Control Group 1 Group 2 Group 3 Number of visitors (pretest) 59800 9993 19284 17876 Number of visitors (posttest) 65993 11781 23373 20883 Relative Change +10.36% +17.89% +21.20% +16.82% Which statistical test would be needed to find an answer to my question? Thank you very much. Edit: The below table shows the weekly visitors by test group. Week 23 to 28 are pre-treatment, week 29 to 34 are post-treatment. Control Group 1 Group 2 Group 3 Week 23 8590 1492 2929 2837 Week 24 9217 1588 3138 2846 Week 25 9534 1599 2992 2812 Week 26 10213 1714 3440 3005 Week 27 10435 1704 3187 2987 Week 28 10932 1817 3180 3234 Week 29 11489 1948 3566 3159 Week 30 10936 1974 3707 3273 Week 31 11856 2061 3885 3609 Week 32 10621 1851 3926 3586 Week 33 9905…

Open challenges in LLM research

[ LinkedIn discussion , Twitter thread ] Never before in my life had I seen so many smart people working on the same goal: making LLMs better. After talking to many people working in both industry and academia, I noticed the 10 major research directions that emerged. The first two directions, hallucinations and context learning, are probably the most talked about today. I’m the most excited about numbers 3 (multimodality), 5 (new architecture), and 6 (GPU alternatives). 1. Reduce and measure hallucinations Hallucination is a heavily discussed topic already so I’ll be quick. Hallucination happens when an AI model makes stuff up. For many creative use cases, hallucination is a feature. However, for most other use cases, hallucination is a bug. I was at a panel on LLM with Dropbox, Langchain, Elastics, and Anthropic recently, and the #1 roadblock they see for companies to adopt LLMs in production is hallucination. Mitigating hallucination and developing metrics to measure hallucination is a blossoming research topic, and I’ve seen many startups focus on this problem. There are also ad-hoc tips to reduce hallucination, such as adding more context to the prompt, chain-of-thought, self-consistency, or asking your model to be concise in its response. To learn more about hallucination: Survey of Hallucination in Natural Language Generation (Ji et al., 2022) How Language Model Hallucinations Can Snowball (Zhang et al., 2023) A Multitask, Multilingual, Multimodal Evaluation of ChatGPT…

Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Models to Unique Applications

The real risk of AI is how it concentrates power

Moving AI ethics beyond explainability and fairness to justice

Machine Learning for Developers

StyleGAN runtime phenomenom

I was playing around with MobileStyleGAN pretrained model and multithreading and came along with an interesting phenomenom. After a while application is running MobileStyleGAN starts to produce video clip alike pieces. And I am wondering does anyone have an idea what takes? I made this lengthy video for it where you can see since it looks cool; https://youtu.be/8kpxHZK1NTQ Here's the source code used in the video if it's any help; https://github.com/harism/i_style_gan