AI/ML News & Innovations Hub

AI/ML news, top picks, and generated innovation digests.

★ Visit ai-karthik.com
422Sources
23587News Items
8Top Picks
147Blogs
successLast Run

Latest AI/ML News

23587 matching items

Should dummy variables be allowed to correlate (R code & data provided)
Cross Validated 2021-03-07 18:32 UTC Score 12.0 AI-113-20210307-social-media-d484c114 Full article

Should dummy variables be allowed to correlate (R code & data provided)

tl;dr: What is the difference between a regression model that allows its dummy variables to be correlated and one that doesn't do so? For example, how does that affect Standard Error of the regression coefficients (coefs)? In my reproducible SEM ( structural equation model ) modelA below, imagine I remove the Group_HI ~~ 0*Group_MT (denoting correlation bet. two dummies is 0 ) altogether leading to modelB . What is the difference between a model that uses Group_HI ~~ 0*Group_MT ( modelA ) and one that doesn't include anything to describe the relation between Group_HI and Group_MT ( modelB )? Specifically, it seems modelB produces Standard Errors for coefs that are closer to those given by a MANOVA: summary(lm(cbind(MP,SE)~Group,data=d)) Response MP : Estimate Std. Error t value Pr(>|t|) (Intercept) 4.6667 0.6150 7.588 8.19e-10 *** GroupHI 3.1833 0.8136 3.913 0.000281 *** ## This SE GroupMT 6.7451 0.8438 7.994 1.95e-10 *** ## This SE Response SE : Estimate Std. Error t value Pr(>|t|) (Intercept) 4.2000 0.3378 12.432 But modelA produces Standard Errors for coefs that are closer to those given by: stacked_DVs |t|) DVMP 4.666667 0.4961705 9.405370 2.364383e-15 DVSE 4.200000 0.4961705 8.464833 2.559422e-13 GroupHI 3.183333 0.6563718 4.849893 4.648941e-06 ## This SE GroupMT 6.745098 0.6807403 9.908475 1.912183e-16 ## This SE Reproducible data and R code: library(tidyverse); library(fastDummies) ; library(lavaan) d |z|) MP ~ Group_HI 3.183 0.659 4.830 0.000 ## This SE SE ~ Group_HI…

Read →
AI Stack Exchange 2021-03-04 12:04 UTC Score 17.0 AI-110-20210304-social-media-d2ba293b

Can the law of iterated expectation be used on the inner expectation of the DQN cost function described in the DQN paper

Is the expression for the DQN cost function, Equation (2) of the DQN paper $$\begin{align}L_1 &= E_{\mu,\pi}\left[\left(y_i - q(s,a;\theta)\right)^2\right]\\ &=E_{\mu,\pi}\left[\left(E_{\mathcal{E}}[r + \gamma \max\limits_{a'}q(s',a';\theta^-)] - q(s,a;\theta)\right)^2\right] \end{align}$$ equivalent to this? (Substituting the expression for $y_i$ defined in the paragraph directly after, $\mathcal{E}$ represents the transition distribution governed by the environment, $\pi$ represents the behaviour policy and $\mu$ represents the stationary distribution of states) $$L_2 = E_{\mu,\pi,\mathcal{E}}\left[\left(r + \gamma \max\limits_{a'}q(s',a';\theta^-) - q(s,a;\theta)\right)^2\right]$$ Can the law of iterated expectation be used to derive the second expression from the first, if not, is there another way to go about showing their equivalence IF they are equivalent. It seems as though $L_2$ is used for sampling but I'm not sure how it's possible to get here from the original cost function $L_1$ . If it is possible to use $L_2$ to sample I assume that means the two expressions must be equivalent. The second expression is used for sampling in the DQN paper here. I do realise that the gradient for each function is the same and thus so is the $n^{th}$ derivative for some $n\geq1$ and since the curvature and optimas align I guess that also means they are the same function (minus some constant difference)? $$\nabla_{\theta} L = E_{\mu,\pi,\mathcal{E}}\left[\left(r + \gamma \max\limit…

AI Research & Papers RAG
Read →
Anyscale Blog 2021-03-03 00:00 UTC Score 35.0 USR-0085-20210303-ai-specialis-34d3e5fb

How to Speed Up Pandas with Modin

The pandas library provides easy-to-use data structures like pandas DataFrames as well as tools for data analysis. One issue with pandas is that it can be slow with large amounts of data. It wasn’t designed for analyzing 100 GB or 1 TB datasets. Fortunately, there is the Modin library which mitigates these issues.

AI Research & Papers
Read →
Oxford Machine Learning Research Group 2021-03-01 17:41 UTC Score 25.0 USR-0027-20210301-research-aca-79a93137 Full article

aistats_iclr_2021

AISTATS and ICLR acceptances 2021 Alexander Camuto, Matthew Willetts, Brooks Paige, Chris Holmes and Stephen Roberts (2021). Learning Bijective Feature Maps for Linear ICA. Proceedings of AISTATS 2021 (to appear). Alexander Camuto, Matthew Willetts, Stephen Roberts, Chris Holmes, Tom Rainforth (2021).

Read →
Diff-in-Diff with a binary outcome? (i.e., comparing proportions of two groups over two time points in a paired setting)
Cross Validated 2021-03-01 17:33 UTC Score 15.0 AI-113-20210301-social-media-e21897ed Full article

Diff-in-Diff with a binary outcome? (i.e., comparing proportions of two groups over two time points in a paired setting)

Let's say I have students randomly split into two groups (treatment and control). Each student is given a test and they either pass or fail. They then either take a course (the students in "treatment") or they don't (students in "control") - then they take the exam again. I want to know if the course helped the students who took the exam in the second time. If we had only the second time, and wanted to compare treatment to control, we could use Chi-square test (or Fisher's exact test). If we had only one group (e.g., treatment), and we wanted to compare the success in the example before-after the course, we could have used McNemar's test. But what test would I use to compare the effect of the course on the improvement in the exam's pass rates? If the outcome wasn't pass/fail, but a normal outcome, we could have taken the difference between the before and after for each student, and then compare the two groups using a two sample $t$ -test (i.e., Diff-in-Diff). But how should I proceed with a binary outcome?

Read →
Cross Validated 2021-02-25 21:19 UTC Score 12.0 AI-113-20210225-social-media-6348763c

Differencing in a simple OLS regression

I am attempting to do a simple OLS regression on time series data from 1990 to 2019, and there are trends in my variables: GDP per capita (dependent variable), oil prices, etc., which I can difference. I cannot difference variables such as inflation, as they are already stationary, as demonstrated on Eviews. Is it acceptable in a model to have differenced variables and variables which are already stationary and if so, why? For context I am using log( gdp per capita) as my dependent variable, with oil price as my primary independent in combination with net exports and inflation to calculate the impact of oil price on a net exporting country. I am unable to conduct (S)VAR analysis as I have not been taught this yet. Any further help would be greatly appreciated.

Read →
Cross Validated 2021-02-24 11:01 UTC Score 9.0 AI-113-20210224-social-media-22671c95

Checking significance in differences between groups (gaps in income between genders and education levels)

I wonder if you could give me a hint on how to find out whether there is a significant difference in the income gap between genders and their education level. The picture below shows education levels (edu_level_old), means for women and men income (in Russian currency), and a gap in income measured by %. I had no problem with finding out whether, for example, women with higher education earn less than men with higher education (done this by ttest). BUT what I would like to discover is whether this gap in earnings between different education levels (for example, a gap between those with higher education and secondary education) is statistically different . Is there a way I could do that? I have a feeling that it is rather simple and obvious for some but have no idea how to accomplish that. Your help is much appreciated!

Read →
AI Stack Exchange 2021-02-16 20:43 UTC Score 16.0 AI-110-20210216-social-media-45ad770b

Late Onset Augmentation

If I train a U-Net model for image segmentation (e.g. medical images) and start training until it converges and then add augmentation - can i expect similar results as if i train with augmentation from the beginning ?

Read →
Anyscale Blog 2021-02-16 00:00 UTC Score 30.0 USR-0085-20210216-ai-specialis-40161c22

Data Processing Support in Ray

This blog post highlights two features in the latest Ray 1.2 release: native support for spilling to external storage, and support for libraries from the Python data processing ecosystem, including integrations for PySpark and Dask.

RAG
Read →
Distill Archive 2021-02-11 20:00 UTC Score 10.0 AI-038-20210211-ai-specialis-e4474d4a Full article

Self-Organising Textures

Neural Cellular Automata learn to generate textures, exhibiting surprising properties.

Read →
Data Science Stack Exchange 2021-02-11 19:29 UTC Score 29.0 AI-111-20210211-social-media-43792a8a Full article

Train MLP Neural Network on time series data?

Newbie question here but I was curious to ask if an MLP Neural type network can be trained on time series data? The dataset that I have is an electricity type data set from a building power meter and I can find I can train a decent NN model with including a lot of weather data and also a lot of one hot encoding dummy variables for time-of-week. (day, hour, month number, etc.) I am experimenting in Python with the Tensorflow Keras library and I know the default during the training process randomly shuffles the data. Is this a No-No for a time series type problem where the random shuffle will take out the seasonality from the data? (stationary/non-stationary) The results shuffling the data really aren't that bad at a glance but not-randomly shuffling the data the results for MLP NN are poor, like the model doesn't train well. I know some other times series forecast methods can include ARIMA, LSTM, etc. but I was curious to inquire if MLP can be used for these purposes too? What I ultimately need is a short term forecast method that can incorporate hourly weather forecast (from a web API) to forecast future hourly building electricity. Any tips greatly appreciated. Thanks

AI Research & Papers
Read →
Cross Validated 2021-02-09 14:50 UTC Score 12.0 AI-113-20210209-social-media-46aef6a3

Is there any statistically meaningful definition for object confidence in object detection?

Most modern object detection algorithms rely on neural networks and output a bounding box and confidence for each object (or more accurately, a confidence for each possible object class considered, the highest of which is taken to be the true object confidence and to define the predicted class for that object). Beyond giving a relative likelihood of "objectness" (a predicted bounding box with 0.9 confidence is more likely to actually contain an object than a predicted bounding box with 0.1 confidence), is there any statistical significance to these predictions? For instance, should we be able to say that a predicted confidence of 0.1 indicates that roughly 10% of predictions at this confidence are actually objects? Is there another statistically meaningful definition? I understand that at best this would be a loose statistical meaning because neural networks would be trained with loss functions that codify this statistical distribution into a loss, and the network would then only approximate this statistical distribution. Thus the distribution would also depend on the loss function used to train the classification portion of the neural network. These issues notwithstanding, is there or was there at some point inherent meaning to the confidence measure, or is it purely a relative quantity?

Computer Vision AI Chips & Hardware
Read →
Cross Validated 2021-02-09 03:18 UTC Score 9.0 AI-113-20210209-social-media-616459ea

Why does Chinese Restaurant Table Distribution look like a Gaussian Distribution?

The Chinese Restaurant Table Distribution describes the probability distribution for the number of non-empty tables in the Chinese Restaurant Process after $T$ customers have been seated. Specifically, if $K_T$ is the (random) number of non-empty tables after seating $T$ customers in a $CRP(\alpha)$ , its PMF is defined as $$P(K_T = k) = \frac{\Gamma(\alpha)}{\Gamma(\alpha + T)} |S(T, k)| \alpha^k$$ where $k \in \{1, ..., T\}$ and $|S(\cdot, \cdot)|$ are unsigned Stirling numbers of the first kind. I discovered that when I fix $\alpha$ and plot the distribution for increasing values of $T$ , the distribution looks more and more like a Gaussian. For alpha=10.01, I plot the distribution from T=1 (red) to T=50 (blue): My question is: why does the Chinese Restaurant Table distribution look like a Gaussian distribution for medium-to-large $T$ ?

Read →
Distill Archive 2021-02-04 20:00 UTC Score 10.0 AI-038-20210204-ai-specialis-a9b7e7bb Full article

Visualizing Weights

We present techniques for visualizing, contextualizing, and understanding neural network weights.

Read →
fAIr LAC 2021-02-03 14:10 UTC Score 27.0 USR-0219-20210203-ai-specialis-69725fc4 Full article

fAIr LAC: Adopción ética y responsable de la inteligencia artificial en América Latina y el Caribe

fAIr LAC: Adopción ética y responsable de la inteligencia artificial en América Latina y el Caribe @administrador Mié, 03/02/2021 - 14:10 Adopción ética y responsable de la inteligencia artificial en América Latina y el Caribe Imagen principal Categoría Publicaciones Tipo de recurso Documentos Archivos de recurso Prueba de nota técnica Descargar Enlace externo https://publications.iadb.org/es/fair-lac-adopcion-etica-y-responsable-de-la-in… Destacado Desactivado Order destacado -1

Read →
Anyscale Blog 2021-02-03 00:00 UTC Score 36.0 USR-0085-20210203-ai-specialis-11e59c6e

How to Speed up Scikit-Learn Model Training

This post gives an overview of different ways to speed up your scikit-learn models and discusses some limitations of each approach.

Read →
Distill Archive 2021-01-30 20:00 UTC Score 10.0 AI-038-20210130-ai-specialis-0dc8e03e Full article

Curve Circuits

Reverse engineering the curve detection algorithm from InceptionV1 and reimplementing it from scratch.

Read →
Distill Archive 2021-01-27 20:00 UTC Score 10.0 AI-038-20210127-ai-specialis-0ede2f7c Full article

High-Low Frequency Detectors

A family of early-vision neurons reacting to directional transitions from high to low spatial frequency.

Read →
Cross Validated 2021-01-24 07:21 UTC Score 18.0 AI-113-20210124-social-media-afa476fc

Are there constraints for the variance of predicted probability on calibrated models?

I'm sorry if the title is too vague. I'm not really sure of what I ask, this is a somewhat speculative question... The setting is that I'm using XGBoost in a binary classification problem (40% positive class) , and I want to get predicted probabilities, so after the model is trained, I train an isotonic calibrier. What I get is the following: The first model, before calibration, was good enough in terms of AUC, obtaining 0,82. The probabilities were not calibried at all. After calibration, the probabilities are well calibrated, but I plotted the histogram of predicted probability and I got this: So, after seeing this histogram, I began to think that the model was not very informative, or too conservative. I mean, let's imagine that we had all possible models with perfect calibration in front of us. Which one will be the best? And the worst? I think the worst will be the model that predicts the base rate for all instances, as this is the bare minimum that a non-informative but calibrated model can give. And the best will be the model with two peaks, on 0 and 1, a model that "knows" for certain the class of each instance. In between, the rest of models could be ordered in terms of how concentred is the histogram around the mean. I'm not sure of what I'm saying, so I ask just in case there is a theory that supports this intuitions: Is it true that, among calibrated models, the more "disperse" the better? For example, the model with the bigger variance the better? And if so, if…

Read →
Data Science Stack Exchange 2021-01-22 13:01 UTC Score 21.0 AI-111-20210122-social-media-08200684 Full article

TypeError: Expected sequence or array-like, got

I am training a CNN, it runs well but I get an error when I try to get precision_score , recall_score and the f1_score . Here is a snippet of my code; # The next step is to split training and testing data. For this we will use sklearn function train_test_split(). features_train, features_test, labels_train, labels_test = train_test_split(features, labels, test_size=.2) features_train.shape, features_test.shape, labels_train.shape, labels_test.shape ((180568, 2677356), (45143, 2677356), (180568,), (45143,)) features_train.shape[0], features_train.shape[1], labels_train.shape[0] (180568, 2677356, 180568) n_timesteps, n_features, n_outputs = features_train.shape[0], features_train.shape[1], labels_train.shape[0] X_train = np.zeros((180568, 82, 1)) y_train = np.zeros((180568, 82)) n_timesteps, n_features, n_outputs = X_train.shape[1], X_train.shape[2], y_train.shape[1] n_samples = 1000 X = np.random.uniform(0,1, (n_samples, n_timesteps, n_features)) y = pd.get_dummies(np.random.randint(0,n_outputs, n_samples)).values model = tf.keras.models.Sequential([ tf.keras.layers.Conv1D(input_shape=(n_timesteps, n_features), activation='relu', kernel_size=2, filters=32), tf.keras.layers.MaxPooling1D(strides=3), #tf.nn.local_response_normalization((1, 1, 1, 1), depth_radius=5, bias=1, alpha=1, beta=0.5, name=None), tf.keras.layers.LayerNormalization(axis=1), tf.keras.layers.Conv1D(input_shape=(n_timesteps, n_features), activation='relu', kernel_size=2, filters=64), tf.keras.layers.MaxPoolin…

AI Ethics AI Chips & Hardware
Read →
Cross Validated 2021-01-21 21:07 UTC Score 18.0 AI-113-20210121-social-media-1fe94bf1

Why are the p-values of this mixed effect model non-uniform when there is no effect (synthetic data)?

I've been looking into using mixed effect models for a project, and along the way wanted to experiment with how they'd work on synthetic data. However, I'm confused by the results — if I generate what I think is null data (e.g. no true difference between two inputs), the p-value for their difference is skewed towards higher significance than I would expect. I'm not sure if I'm making an incorrect assumption about how mixed effect models are supposed to work, or if I'm generating it incorrectly. Here's what I've been doing: I have two sets of inputs, each with N distinct groups, and M samples per group. To draw outputs, for both sets of inputs, I draw N values to represent each group's baseline effect, and then M values per group to represent the random variances. I then add those up for both sets of inputs, run a mixed effect model, and generate a p-value. I repeated this 1000 times and made a histogram of the p-values. Here's the code: import numpy as np import pandas as pd import statsmodels.api as sm import matplotlib.pyplot as plt num_groups = 10 num_samples_per_group = 20 num_iter = 1000 group_var = 1.0 random_var = 1.0 # Build basic dataframe w/ two inputs (input_0 and input_1), each with num_groups different groups, each # containing num_samples_per_group individual samples. df = pd.DataFrame({ 'input': ['input_0'] * num_samples_per_group * num_groups + ['input_1'] * num_samples_per_group * num_groups, 'group': ( ['input_0_group_%s' % (i % num_samples_per_group) for i…

AI Chips & Hardware
Read →
Jay Alammar Blog 2021-01-19 00:00 UTC Score 33.0 USR-0113-20210119-ai-specialis-e1508f35 Full article

Finding the Words to Say: Hidden State Visualizations for Language Models

By visualizing the hidden state between a model's layers, we can get some clues as to the model's "thought process". Figure: Finding the words to say After a language model generates a sentence, we can visualize a view of how the model came by each word (column). Each row is a model layer. The value and color indicate the ranking of the output token at that layer. The darker the color, the higher the ranking. Layer 0 is at the top. Layer 47 is at the bottom. Model:GPT2-XL Part 2: Continuing the pursuit of making Transformer language models more transparent, this article showcases a collection of visualizations to uncover mechanics of language generation inside a pre-trained language model. These visualizations are all created using Ecco, the open-source package we're releasing In the first part of this series, Interfaces for Explaining Transformer Language Models, we showcased interactive interfaces for input saliency and neuron activations. In this article, we will focus on the hidden state as it evolves from model layer to the next. By looking at the hidden states produced by every transformer decoder block, we aim to gleam information about how a language model arrived at a specific output token. This method is explored by Voita et al.. Nostalgebraist presents compelling visual treatments showcasing the evolution of token rankings, logit scores, and softmax probabilities for the evolving hidden state through the various layers of the model.

AI Chips & Hardware Open Source AI
Read →
Cross Validated 2021-01-17 13:01 UTC Score 12.0 AI-113-20210117-social-media-f9e63af0

Maximum likelihood observation with a constraint

I have a vector $\mathbf{x}$ of $n$ real random variables. The random variables can be assumed to follow a multivariate normal or log-normal distribution (assuming $\mathbf{x}>0$ is ok). I would like to analytically obtain the probability that the sum of the random variables decreases below a certain threshold, i.e. ${\mathbf{1}^T\mathbf{x}}\leq\sqrt{\mathbf{x}^T\mathbf{Q}\mathbf{x}}$ , where $\mathbf{1}$ is a vector $n$ ones and $\mathbf{Q}$ is a positive definite $n\times n$ real matrix. I cannot find the distribution of ${\mathbf{1}^T\mathbf{x}}-\sqrt{\mathbf{x}^T\mathbf{Q}\mathbf{x}}$ or ${\mathbf{1}^T\mathbf{x}}/\sqrt{\mathbf{x}^T\mathbf{Q}\mathbf{x}}$ . Therefore I have started to look for alternative ways to find the probability: Is there any sense to find the maximum likelihood observation s.t. the constraint and then obtain the probability from a cdf? For example, could I apply the normal log-likelihood as follows: $\arg\max$ $f(\mathbf{x} \mid \mathbf{\mu},\mathbf{\Sigma}) =\frac{1}{2} [\ln(|\mathbf{\Sigma}|) + n \ln\left (2\pi \right ) + (\mathbf{x}-\mathbf{\mu})^T\mathbf{\Sigma}^{-1}(\mathbf{x}-\mathbf{\mu})]$ subject to ${\mathbf{1}^T\mathbf{x}}-\sqrt{\mathbf{x}^T\mathbf{Q}\mathbf{x}} \leq 0$ . If this is nonsense please elaborate why.

Read →
Train autoencoder to reconstruct 2D Gaussian data
Cross Validated 2021-01-16 05:58 UTC Score 18.0 AI-113-20210116-social-media-d7b6dd31 Full article

Train autoencoder to reconstruct 2D Gaussian data

I am trying to train an autoencoder to reconstruct 2D Gaussian data. These datapoints are simply sampled from Gaussians with means and covariances chosen randomly. An illustration of what I am trying to do can be visualized below for datapoints from 3 different Gaussians The blue crosses are the training data and the orange points are the reconstruction of the autoencoder. The goal is to have the orange points match as closely as possible to the blue points. I am using MSE loss for now, although KL divergence between the input and output distributions seem to make more sense, but is trickier to implement for backprop. For each training set I sample 100 2D coordinates from a different Gaussian, say (x,y) and I stack them into a 200-long feature vector [x1, y1, x2, y2, ... , x100, y100] and pass in mini-batches of these into the autoencoder. I sort these points by the L2 norm of the tuples (x,y), which I've found to help a bit. I've tried basic fully connected autoencoders (200 input/output neurons) and also convolutional autoencoders. Any tips on how I could make it work better? Thanks!

AI Chips & Hardware
Read →
Is it ok to use volunteers for prospective cohort studies
Cross Validated 2021-01-15 03:07 UTC Score 12.0 AI-113-20210115-social-media-6681c7ee Full article

Is it ok to use volunteers for prospective cohort studies

Prospective cohorts are ones where the exposure is known but not the disease outcome. Volunteer bias is where volunteers tend to be healthier. I've been told because of this volunteer bias, we should not use volunteers for our epidemiology experiments. I can see why this is true in the case of prevalence studies. Volunteers tend to smoke less and thus we will underestimate the prevalence of smoking if we only get volunteers. But I don't see why this must be true for prospective cohort studies. Let's say there's been a nuclear accident and we rush to attract volunteers from those who've been exposed to radiation. All my volunteers, exposed and unexposed, are very fit and don't smoke. Their general better health could mean they recover better from radiation. One could say this skews results. However, given that my other unexposed volunteers are equally healthy - wouldn't this be a fair comparison? Is there still volunteer bias here?

AI Ethics
Read →
How to extract parameters from a text using AI/NLP
AI Stack Exchange 2021-01-13 13:34 UTC Score 23.0 AI-110-20210113-social-media-8c81b4b2 Full article

How to extract parameters from a text using AI/NLP

lets say I have three texts: "make a heading that says hello word" "make a heading of hello world" "create heading consist of hello world" How can I fetch those groups of words using AI which is referring to heading i.e hello world in this case. Which AI frameworks or libraries can do that? in all examples heading is pointing to hello world (which i am referring as group of words). so basically i want those words which will be a part of heading or in other word there is a relationship between them. another example i can give is "I am watching Breaking bad" so there is a relationship between watching and breaking bad and i want to extract what are you watching. What's the best approach? Do I have to train a model for that or there are some other techniques that can get it done?

NLP
Read →
VAR model variable selection
Cross Validated 2021-01-11 21:06 UTC Score 16.0 AI-113-20210111-social-media-e55e7f3e Full article

VAR model variable selection

I'm required to use two time series models in my exam project. I want to use a stock price of an energy company, and then explain it first using ARIMA, and then adding other variables and using VAR. My problem is that I can't find variables which pass the Granger test of causality. I have tried prices of commodities, competitor stock prices, etc. So now I have opted to use two of the same company's stocks, but just listed on different stock exchanges (same company, same currency - one stock trading in Germany, the other in Brussels) Obviously, the stock on both markets is impacted by the same unobserved variables. Is this a huge mistake, or can I proceed if I discuss and show that I am aware of omitted variable bias?

AI Ethics
Read →
Data Science Stack Exchange 2021-01-11 06:25 UTC Score 12.0 AI-111-20210111-social-media-b6b9df2d Full article

Time series forecasting with constraints

I want to predict the passenger flow volume of an airline route, which subjects to supply capacity constraints of the route (i.e., the passenger flow volume should not be higher than the supply capacity). Are there any algorithms that could be used to do this kind of forecasting problem with constraints?

Read →
Cross Validated 2021-01-08 19:01 UTC Score 12.0 AI-113-20210108-social-media-4cc878b0

Quasi-random sequence with discrete variables with differing number of levels

My question is probably worded incorrectly but here it is: I have (say) 3 discrete random variables: x1: has 15 levels (uniform pdf for simplicity) x2: has 3 levels (uniforms) x3: has 4 levels (uniform) I want to generate n samples in such a way that: there are no repetitions of samples as n --> 15x3x4 the entire "full factorial" sample is generated; When generating (n+d) set of samples, the first n samples are the identical for any d I tried to use Sobol quasi random sequence to generate this sample BUT I am not able to satisfy #1 above without retaining all prior points in memory and checking for duplicates. This is something I would like to avoid. Thanks for your answers.

Read →
Alignment Newsletter 2021-01-04 01:32 UTC Score 35.0 USR-0153-20210104-ai-specialis-4e07dab9 Full article

FAQ: Advice for AI alignment researchers

Consider reading How to pursue a career in technical AI alignment. It covers more topics and has more details, and I endorse most if not all of the advice. To quote Andrew Critch: I get a lot of emails from folks with strong math backgrounds (mostly, PhD students in math at top schools) who are […]

AI Safety & Alignment AI Research & Papers
Read →
Cross Validated 2020-12-24 07:31 UTC Score 12.0 AI-113-20201224-social-media-974e2d3b

Prove the linear transformation of Gaussian distribution from Bayesian perspective

I understand the linear transformation property of Gaussian distribution, i.e., if $x\sim\mathcal{N}(\mu, \sigma^2)$ , then $cx\sim\mathcal{N}(c\mu, c^2\sigma^2)$ given $c$ is a constant. If we write this as a marginalization: $$p(f(c)|c) = \int p(f(c)|c, x)p(x)dx$$ and here $f(c) = cx$ . $p(x)$ can be expressed as $\dfrac{1}{\sqrt{2\pi}\sigma}e^{-\frac{1}{2\sigma^2}(x-\mu)^2}$ . But how do we obtain $p(f(c)|c)$ from the above marginalization form (or the posterior predictive distribution, here I simplified the posterior to just $p(x)$ ), that is to get $$\quad \, p(f(c)|c) = \dfrac{1}{\sqrt{2\pi}c\sigma}e^{-\frac{1}{2c^2\sigma^2}(f(c)-\mu c)^2} \quad\text{?}$$ What troubles me is $p(f(c)|c, x)$ . I think it is $1$ , as it is deterministic when $c$ and $x$ are given, but I'm not sure how to proceed.

Read →
Cross Validated 2020-12-23 11:40 UTC Score 18.0 AI-113-20201223-social-media-1cae7345

Choice of Smoothing Kernel in ABC

In Approximate Bayesian Computation, one approximates an intractable likelihood by convolving it with some smoothing kernel $K$ as \begin{align} \ell^{\text{ABC}} ( x | \theta ) = \int \ell ( z | \theta ) \, K (x, z) \, dz. \end{align} Common choices for $K$ include $K(x, z) = \mathbf{I} \left[ d(x, z) \leqslant \varepsilon \right]$ and $K(x, z) = \exp \left( - \frac{d (x, z)^2 }{2 \varepsilon^2 }\right)$ , where $d$ is some sort of distance-like function, and $\varepsilon$ is some 'tolerance / threshold' parameter. My question is: in practice, what guides the choice of $K$ ? For example, which features of a problem should encourage a user to prefer a uniform kernel to a Gaussian kernel? Ignore the choice of summary statistics, threshold parameter, and so on. I am aware of the 'model error' perspective of R. Wilkinson (as in this paper ), which is very useful in some circumstances. Have alternative perspectives arisen in the interim? e.g. are there purely algorithmic reasons to prefer a specific kernel?

AI Research & Papers RAG
Read →
Jay Alammar Blog 2020-12-17 00:00 UTC Score 39.0 USR-0113-20201217-ai-specialis-fb351fb3 Full article

Interfaces for Explaining Transformer Language Models

Interfaces for exploring transformer language models by looking at input saliency and neuron activation. Explorable #1: Input saliency of a list of countries generated by a language model Tap or hover over the output tokens: Explorable #2: Neuron activation analysis reveals four groups of neurons, each is associated with generating a certain type of token Tap or hover over the sparklines on the left to isolate a certain factor: The Transformer architecture has been powering a number of the recent advances in NLP. A breakdown of this architecture is provided here . Pre-trained language models based on the architecture, in both its auto-regressive (models that use their own output as input to next time-steps and that process tokens from left-to-right, like GPT2) and denoising (models trained by corrupting/masking the input and that process tokens bidirectionally, like BERT) variants continue to push the envelope in various tasks in NLP and, more recently, in computer vision. Our understanding of why these models work so well, however, still lags behind these developments. This exposition series continues the pursuit to interpret and visualize the inner-workings of transformer-based language models. We illustrate how some key interpretability methods apply to transformer-based language models. This article focuses on auto-regressive models, but these methods are applicable to other architectures and tasks as well. This is the first article in the series. In it, we present explo…

Computer Vision AI Chips & Hardware Fine-tuning NLP
Read →
Cross Validated 2020-12-12 20:09 UTC Score 26.0 AI-113-20201212-social-media-cb7b55bc

Accuracy is 100% but model.predict is totally wrong! what could be the problem? (Autoencoder NN) [closed]

I have an Autoencoder model that receives 4 different vectors { (1,0,0,0) , (0,1,0,0) , (0,0,1,0) , (0,0,0,1) } The encoder transforms the vectors to vectors size 2, which get inside an "NLPN channel" which is a channel with nonlinear phase noise, and then goes to the decoder that reconstructs the original vector. Architecture: 4->50->2->NLPN->2->50->4 (Softmax) The loss function is mse between the input, and the input ( self.model.fit(self.x, self.x) ) The code is pretty basic: the first cell is just some simple functions defined, then it's the autoencoder class, then it's the training, and then it's the prediction which shows that for each vector 1,0,0,0 and so on, the decoder predicts (~0.25,~0.25,~0.25,~0.25) which shows that the NN has no idea what it's doing. CODE: https://colab.research.google.com/drive/1nFymmEloUSa1yjS7e5CPGsKwqP1HjS57?usp=sharing

AI Chips & Hardware AI Research & Papers NLP
Read →
Cross Validated 2020-12-09 16:55 UTC Score 23.0 AI-113-20201209-social-media-7cbce1bb

Statsmodels Linear Regression with ARIMA errors model not including the ARIMA error in prediction?

I have fit a SARIMAX (1, 0, 0) model to a timeseries dataset consisting of 1 endogenous timeseries ("Y") and 1 exogenous timeseries ("X"). My exogenous timeseries in the model was defined with sm.add_constant. Stationarity and invertibility were enforced in the Statsmodels SARIMAX model. trend = 'n' The model parameters were: const = -0.0218685769848199 X = 1.02191782244191 ar.L1 = 0.780715685830694 sigma2 = 4.942018879444325E-06 I generated the predictions (including out of sample forecasts) using predicted_mean. For the last (oldest) observed record in the timeseries: the model's predicted (and last fitted) value is 0.9948866129977041 the Y value is 1.00547070891269 the X value is 0.994948094292246 the residual (.resid) is 0.010584096 For the first prediction: the model's predicted value is 0.990566738090985 the X value is 0.990720870937109 (There is no Y value, and also no residual) In trying to manually recreate the forecasts in Excel, I was only able to match the Statsmodels forecast when I don't include the ARIMA-modelled residuals in my calculation. I was able to match the provided residual. Here's the model I followed: ϵ^t−1 = yt−1 − (β0 + β1xt−1) = 0.010584096 (matches the Statsmodels residual for the last (oldest) observed record) ϵt = ϕ(ϵt−1) = 0.00826316972 Now we can compute yt which should be: yt = β0 + β1xt + ϵt = -0.0218685769848199 + (1.02191782244191 * 0.990720870937109) + 0.00826316972 = 0.99882990780828 (which doesn't match the Statsmodels prediction valu…

AI Research & Papers
Read →
How to match regions in difference-in-differences analysis
Cross Validated 2020-12-04 02:39 UTC Score 9.0 AI-113-20201204-social-media-02a6b804 Full article

How to match regions in difference-in-differences analysis

I've had an idea for a project to work on over winter break. It involves a difference-in-differenced analysis of two similar-sized cities on different sides of a state border. How do I confirm that the two cities are similar enough in their characteristics? Is it an eye test or is there a statistical methodology? I would also appreciate any suggested readings.

Read →
Extracting the fitted expected number of points from mppm in spatstat (in R)
Cross Validated 2020-12-02 23:52 UTC Score 12.0 AI-113-20201202-social-media-c3b00510 Full article

Extracting the fitted expected number of points from mppm in spatstat (in R)

When fitting a Poisson process model on multiple point patters using the spatstat R package (mppm function), is there a computationally efficient way to extract the fitted expected number of points from the mppm fit? I expand with a toy example below: Let's say that my data are point patterns over T time periods and I have a single spatio-temporal predictor in the form of an image (spatstat im class). I generate such toy data here: set.seed(1234) library(spatstat) # Number of time periods. For each time period I observe a point pattern. time_points Then, we can use mppm to fit a poisson process model for the point pattern data on the predictor. We can do so with the following code: mod_dta In order to extract the estimated conditional intensity function we can use the predict.mppm function: mod_dta$fitted_cif which we can use to extract the expected number of points for each time period as the integral of the conditional intensity function: plot(with(mod_dta, integral(fitted_cif))) However, this process can be slow if the number of time periods time_points is large, and the value for ngrid in predict.mppm is set to large (to improve accuracy). I wonder if the fitted expected number of points can be extracted directly from the mppm fit ( pp_mod in the example) without having to define the conditional intensity as an image and integrate over it. Thank you for your help.

Read →
Cross Validated 2020-12-02 11:33 UTC Score 17.0 AI-113-20201202-social-media-96f1e73a

Determining coin mint year from a known sample based on elemental composition

I have data on the metallic compositions (percentages of specific elements) of 30 coins, each minted in a different year from 1902 to 1967. I also have the composition of a coin minted in an unknown year which I would like to try and classify to a year of minting through comparison with the reference dataset. How would I go about doing this? I attempted discriminant analysis in SPSS using Coin Year (numeric; nominal) as the grouping variable and the percentages of each metal element (Cu, Sn, Zn, Pb, As, Zr, Ni, Nb, Co, W, Mn, Se) as the Independents, but received the error: “Warnings: There are 31 non-empty groups and 31.000 (31 unweighted) cases that are valid. Not enough weighted or unweighted cases. Execution of this command stops.” I suspect this is because there is only one set of values associated with each year, but I’m stuck on how to overcome this. I thought about potentially using a Euclidean distance measure, but I’m not sure. Any advice?

AI Research & Papers
Read →
Cross Validated 2020-12-01 16:44 UTC Score 9.0 AI-113-20201201-social-media-5818c691

Measure of distance between two survey responses

I've found some survey data where respondents answer 63 question by giving a response for each question between 0-10 (0 for strong disagreement, 10 for strong agreement). So I can view every respondent as a integer point in $[0,10]^{63}$ . Question: is there a "standard" way to measure the distance between two respondents? Something like the taxicab distance between them seems somewhat reasonable, but likewise so does any $L^p$ -distance. Though these wouldn't account for questions that contributed the most of least variation in responses, for example. My intended application is using this to measure distances between points to experiment with persistent homology.

Read →
Using Monte Carlo standard error to determine the ideal number of trials?
Cross Validated 2020-12-01 00:27 UTC Score 12.0 AI-113-20201201-social-media-6b28a823 Full article

Using Monte Carlo standard error to determine the ideal number of trials?

I am doing a simulation study that involves estimating the parameter $\theta$ under a specific experimental design. $\theta$ is the parameter that take on the value 1 if algorithm A is better than algorithm B, and 0 otherwise. I am repeating my simulation study for $N$ times so that I can count the number of times where the algorithm A performed better than the algorithm B. If the number of times that algorithm A performed better than algorithm B is greater than $\frac{N}{2}$ , then I can conclude that $\theta$ is likely to be equal to 1. But I am not sure how to pick the "right" value for $N$ , and someone advise me that I can analyze the Monte Carlo standard error of the probability to determine the value for $N$ . However, I am not very familiar with doing this kind of thing, and I am seeking a guidance on this. The only idea that I can come up with is that if we let $\hat{\theta}$ to be an indicator variable that is equal to 1 if the event occurs, then $\sqrt{\frac{1}{N(N-1)}\sum^{N}_{i=1}(\hat{\theta}_i-\bar{\theta})^2}$ can be used as the Monte Carlo SE of estimate of bias for $\theta$ , where $\bar{\theta}$ is the sample average of $\hat{\theta}$ 's. However, I am not exactly sure how to use this information to determine the optimal value for $N$ . Should I just try varying the value of $N$ and collect $\theta_1 \cdots \theta_N$ from each value of $N$ , calculate the Monte Carlo SE for each case, and see if which value of $N$ would minimize Monte Carlo SE? How can I d…

AI Ethics AI Research & Papers RAG
Read →
Cross Validated 2020-11-27 01:58 UTC Score 26.0 AI-113-20201127-social-media-813cfebe

What is a good autocorrelation value?

I have a dataset of 400 time series. I want to evaluate if there is some relationship between consecutive data points. Thus, I have calculated the autocorrelation (AC) of the time series with different lags. For the best lag value, I obtain an autocorrelation of about 0.59 on average. Now my problem is how can I know if this is a good AC value? Is there some way to evaluate if an autocorrelation value like 0.59 is good? I have tried the following approaches to evaluate if this is a good value. I have calculated the AC of random sequences to compare it with the AC of my time series. I have found that the AC of random sequences is always close to 0. This is interesting as 0.59 is far from 0 but still does not tell me clearly if 0.59 is a good value. I have thus decided to do another experiment to compare the AC of my time series with the autocorrrelation of an ascending time series (a function that is linearly increasing) and a random time series. In that experiment, I randomly tamper each sequence according to a tampering ratio that is varied from 0 to 100%. The result is like this: It can be observed that the AC of my time series is close to that of the ascending time series and decreases quickly when the tampering ratio is increased. This seems to indicate that there is a good AC in my time series. However, it does not really tell me what is a good value for the AC. Thus, is there other ways to determine what is a good AC value? Or do you have any other suggestions about ho…

AI Funding & Startups AI Research & Papers RAG
Read →
Data Science Stack Exchange 2020-11-26 07:43 UTC Score 18.0 AI-111-20201126-social-media-935ce692 Full article

Validation loss and validation accuracy stay the same in NN model

I am trying to train a keras NN regression model for music emotion prediction from audio features. (I am a beginner in NN and I am doing this as study project.) I have 193 features for training/prediction and it should predict valence and arousal values. I have prepared a NN model with 5 layers: model = Sequential() model.add(Dense(100, activation='elu', input_dim=193)) model.add(Dense(200, activation='elu')) model.add(Dense(200, activation='elu')) model.add(Dense(100, activation='elu')) model.add(Dense( 2, activation='elu')) And this is my loss and optimizer metrics: model.compile( loss = "mean_squared_error", optimizer = 'RMSprop', metrics=['accuracy'] ) When I try to train this model, I get this graph for loss and validation: So the model is trained and reaches accuracy of >0.9 on training data, but on test data accuracy wont fall, but it stays on ~0.5. I don't know how to interpret this graph. I don't think this is overfitting, because validation accuracy wont fall, but it stays the same. How can I try fix this? Update: I tried to add dropout and regularization and it worked in a way that now I clearly see that I have a problem with over-fitting. But now I am stuck again. I can not make my model to decrease validation loss. It always stops at about 0.3 validation loss. I tried changing my model architecture, data preprocessing, optimizer function, and nothing helped.

AI Chips & Hardware AI Research & Papers
Read →
Cross Validated 2020-11-23 20:50 UTC Score 9.0 AI-113-20201123-social-media-5941c3b4

Can Kruskal-Wallis be used for discrete data?

I have a data frame with variables that are counts, nominal (two groups) and continuous. Two of the continuous variable follow a normal distribution; the counts variables do not follow a normal distribution. The samples are independent. Can I use a Kruskal-Wallis test to check the difference in the means of the two groups? I am lost regarding if non-parametric tests can be used for discrete data (specifically, counts). Moreover, there are a lot of ties in the data (it's an activity data frame, where the entrances and exits of birds in nests were counted periodically).

Read →
Cross Validated 2020-11-21 16:11 UTC Score 21.0 AI-113-20201121-social-media-8e58bf86

Feature Selection -

This is my first time working on a dataset with this sort of complexity. https://www.kaggle.com/c/house-prices-advanced-regression-techniques I am thinking of trying to maybe use a logistic or linear regression model to try and predict the prices (i.e. SalePrice). My issue I'm stuck with is how to approach the feature selection of the independent variables since there are 80 attributes. Things I've tried: A correlation matrix (using Pearson) with corrplot()/cor() in R after converting the categorical variables from characters to factor form: as.numeric(factor(Train$Street)) but the issue is the output of the graph is way too big to view trying to bunch 80 variables in. My question is... Is a correlation matrix the right method for selecting the mixture of categorical and numerical attributes? Maybe it's not for me to post in this part of the website, but is there a way to filter the scores in R to finetune and choose ones that are strongly related (i.e. >0.5 or

AI Chips & Hardware AI Research & Papers
Read →
AI Stack Exchange 2020-11-21 13:19 UTC Score 18.0 AI-110-20201121-social-media-569ca473

How should we interpret this figure that relates the perceptron criterion and the hinge loss?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.2 Relationship with Support Vector Machines says the following: The perceptron criterion is a shifted version of the hinge-loss used in support vector machines (see Chapter 2). The hinge loss looks even more similar to the zero-one loss criterion of Equation 1.7, and is defined as follows: $$L_i^{svm} = \max\{ 1 - y_i(\overline{W} \cdot \overline{X}_i), 0 \} \tag{1.9}$$ Note that the perceptron does not keep the constant term of $1$ on the right-hand side of Equation 1.7, whereas the hinge loss keeps this constant within the maximization function. This change does not affect the algebraic expression for the gradient, but it does change which points are lossless and should not cause an update. The relationship between the perceptron criterion and the hinge loss is shown in Figure 1.6. This similarity becomes particularly evident when the perceptron updates of Equation 1.6 are rewritten as follows: $$\overline{W} \Leftarrow \overline{W} + \alpha \sum_{(\overline{X}, y) \in S^+} y \overline{X} \tag{1.10}$$ Here, $S^+$ is defined as the set of all misclassified training points $\overline{X} \in S$ that satisfy the condition $y(\overline{W} \cdot \overline{X}) . This update seems to look somewhat different from the perceptron, because the perceptron uses the error $E(\overline{X})$ for the update, which is replaced with $y$ in the update above. A key point is that the (integ…

AI Research & Papers
Read →
GlobalPolicy.AI 2020-11-20 09:03 UTC Score 28.0 USR-0163-20201120-ai-specialis-f3a2f6c3 Full article

Council of Europe

Achieving impact through intergovernmental co-operation on artificial intelligence About Key focus areas Events AI events calendar Events on Globalpolicy.AI Partners Reports FAQ Contact Search English Français The Council of Europe (CoE) The Council of Europe’s mission is to guide its member States towards a better protection of human rights, in accordance with the values of […]

Read →
Generating data that satisfies certain mean, median and covariance in python
Cross Validated 2020-11-19 15:32 UTC Score 12.0 AI-113-20201119-social-media-5afde366 Full article

Generating data that satisfies certain mean, median and covariance in python

I would like to generate pseudo-random data of 16,000 instances with two positive features each. One with a mean value of 6300 and median of 4600 and the other feature with mean of 12500 and median of 12000. Up until now I managed to get close to what I desire by generating two random batches of below and above the median and then tweaking it a bit to get the required means. The problem now is that the covariance between the two features needs to be equal to 0.97*σ(feature1)*σ(feature2), and I am lost in how to generate the whole data with these requirements. Is there some package or function for generating data with specific values?

Read →