AI/ML News & Innovations Hub

How to appropriately run an adonis or anosim test?

I have microbiome data structured as follows: PatientID Treatment Response P1 Pre. Yes. P1. Post. Yes. P2. Pre. No. P2. Post. No. where I want to look at differences in the microbiome between patients who did and didn't respond to drug treatment. I have approximately $40$ patients. I have used phyloseq to generate a Bray-Curtis distance matrix: bray_dist = phyloseq::distance(physeq, method="bray") I now want to use that distance matrix to test for differences between patients who did and didn't respond. I was thinking of this: adonis(bray_dist ~ sample_data(physeq)$Response, distance = "bray", strata = sample_data(physeq)$Treatment) Is this the correct way to do it? Would the above test the pre and post samples individually? What is the correct way to do this with anosim or adonis ?

Are there Explainable GNN methods for node regression tasks?

I am wondering if there are any explainable methods for GNNs designed for regression tasks (e.g., traffic forecasting) where nodes have numerical features and the predicted output is a numerical value. Most of research papers focus on node classification tasks (GNNexplainer, etc.) but do not specify if these techniques are fit for node-regression tasks.

For "fine-tuning", does the "domain adaptation" approach make sense?

I understand "domain adaptation" to be a type of "transfer-learning" technique. Domain Adaptation: By applying knowledge obtained from a domain with sufficient teacher labels (Source Domain) to a target domain without sufficient information (Target Domain), a discriminator, etc. that works with high accuracy in the target domain is learned. (Domain is a term that refers to a collection of data.) Can I use the "domain adaptation" approach for "fine-tuning" in object detection tasks?

Significance of regression coefficients in two different linear models

Suppose that I have data $\left\{ (x_i, y_i, z_i ) : i=1, 2, \dots, N\right\} $ . I have fitted two linear models: $$ \left[\begin{matrix} z_1\\ z_2\\ \vdots\\ z_N \end{matrix}\right]=\left[\begin{matrix} 1&x_1\\ 1&x_2\\ \vdots&\vdots\\ 1&x_N \end{matrix}\right]\left[\begin{matrix} a_1\\ b_1 \end{matrix}\right] $$ and $$ \left[\begin{matrix} z_1\\ z_2\\ \vdots\\ z_N \end{matrix}\right]=\left[\begin{matrix} 1&x_1&y_1\\ 1&x_2&y_2\\ \vdots&\vdots&\vdots\\ 1&x_N&y_N \end{matrix}\right]\left[\begin{matrix} a_2\\ b_2\\ c_2 \end{matrix}\right].$$ That is, the first model uses $x_i$ values and the second one uses both $x_i$ and $y_i$ values to explain $z_i$ . Now I am considering whether the estimated coefficients $b_1$ and $b_2$ ('slopes') are statistically significant (that is, are the $x_i$ :s significant to the model). First of all, I am not sure how to get started with that that problem. Secondly, is it possible that $b_1$ would be significant and $b_2$ would not, or vice versa?

Train-Test-Split across correlated time series with small sample

I need advice on how to create a train-test-split with a small data set of correlated time series to predict historical values. My data consists of 30 different time series covering each 4000 periods, which are spatially and temporally correlated (e.g. GDP in different countries, measured daily across several years). I do not want to predict future values but compute predictions across the time series (e.g. predict GDP for a country given other countries). Based on my understanding, the best validation strategy would be leave-one-out nested cross-validation. However, I have been asked by my supervisor also to implement a train-test-split in the most classical way (e.g. to fully put a few of the time series completely away as a test set.). How do I choose these time series? Randomization does not seem to make sense, given the small sample, and I am unsure how to do stratification in this context... Would be thankful for any advice!

Comment on The state of ICT in Mozambique by How Instagram could help Mozambican artists and promote a positive international image of Mozambique – Art in Moz

[…] generally focus on exploring the locally available markets. In large part, this is due to the inherent barriers for Mozambican artists to sell their work online such as poor connectivity, banking limitations and […]

Why is VQ-VAE considered a variational encoder?

I'm reading about VQ-VAE and I'm not sure why do they say we can view it as a VAE . Can you explicitly show: what is the latent z-space - is it the discrete space where z can take the integers 1..K where K is the number of embedding vectors? what is the true posterior p(z|x)? is it indeed intractable to use for calculating the marginal p(x), if we are dealing just in a scalar that can hold integers 1..K where K is a moderate 128? why do we need to sample from z's distribution in VAE while in VQ-VAE we just take a deterministic distribution and just use the closest embedding to the encoder output? can you show the ELBO formulation, and show how it boils down to only $\log p(x|z_q(x))$ ?

Dummy variable trap in neural networks and class visualization

Let's say I have data records looking like that: (x1, x2, x3, x4, ..., x100) , where each x can be either alpha , gamma or omega . An example of record could be ('gamma', 'alpha', 'omega', 'alpha', ..., 'gamma') . So the shape of my dataset is (N, 100) (with N the number of records). I want to train a neural network to predict some binary label. As there is no underlying ordering in my input categories, I use dummy variables to feed my network. Therefore, I end up with a dataset of the following shape: (N, 100, 3) . My problem is that I don't really know how to deal with the dummy variable trap. According to this answer , I should drop one category when I use a network without weight decay. However, I thought that even without weight decay, the non-linearity of neural networks (assuming I'm using non-linear activation functions like relu), would be enough to avoid the issue without actually needing to drop one category. Am I wrong? Would a neural network without weight decay behave badly if I do not drop one column? Some context Ideally, I would like to avoid dropping one column as my next step is to create inputs that maximize a class prediction (starting from noise and using gradient ascent to "improve" the input). If I do that with a model trained with a dropped category, I can end up with values close to 0 for my (n-1) categories, probably meaning that the category that would maximize the output would be the dropped one. This interpretation looks correct to me, but it le…

machine learning for a budgeting application

I am interested in finding references and previous applications where prior year budgets are analyzed to provide guidance for a current year budget. Specifically, each year some two thousand items are evaluated for funding, with perhaps 500 funded in that year. Information is available in a spreadsheet with multiple parameters that are manually evaluated to determine if an individual item is funded in the budget. I would appreciate any guidance as to how best to make use of such data for say the previous 5 years, where I know what has been funded in those years, to assist in screening items for the current budget year, in particular what approach to ML would be best. I have attempted a literature search but have not found anything directly relevant. Edit: Found this reference in my literature search, looks to be applicable: https://www.datacamp.com/courses/case-study-school-budgeting-with-machine-learning-in-python

For which problem sizes is Deep Q-Learning suitable and why?

I am wondering for which problem sizes a Deep Q-Learning algorithm is most appropriate. For example, whether it is particularly suited for low complexity problems or not for high complexity problems. And if that is the case, why?

How to discover/approximate the causations/correlations between multiple time-series and related open source libraries?

I have the following time-series data with two value columns. (t: time, v1: time-series values 1, v2: time-series values 2) t | v1 | v2 ---+----+---- 1 | 1 | 0 2 | 2 | 2 3 | 3 | 4 4 | 3 | 6 5 | 3 | 6 6 | 4 | 6 7 | 5 | 8 (7 rows) I am trying to discover (or approximate) the correlation between the $v1$ and $v2$ , and use that approximation for the next step predictions. Please note, the most obvious correlation is $v2(t)=2.v1(t-1)$ . My question is, what are the algorithms to employ for such approximations and are there any open source implementations of those algorithms for SQL/python/javascript?

What does the adversarial loss in a GAN represent?

I'm working on Pix2Pix an image-to-image translation GAN, and I noticed that there is an adversarial loss implemented using BCE, and a L1 loss implemented using MAE. I know L1 loss represents the difference between the predicted image and actual image, but I am not sure what does the GAN adversarial loss represent? This is the official definition The adversarial loss influences whether the generator model can output images that are plausible in the target domain but the meaning is tough to understand. Is it representing the difference between the predicted probability distribution and actual probability distribution?

The Illustrated Stable Diffusion

Translations: Chinese, Vietnamese. (V2 Nov 2022: Updated images for more precise description of forward diffusion. A few more images in this version) AI image generation is the most recent AI capability blowing people’s minds (mine included). The ability to create striking visuals from text descriptions has a magical quality to it and points clearly to a shift in how humans create art. The release of Stable Diffusion is a clear milestone in this development because it made a high-performance model available to the masses (performance in terms of image quality, as well as speed and relatively low resource/memory requirements). After experimenting with AI image generation, you may start to wonder how it works. This is a gentle introduction to how Stable Diffusion works. Stable Diffusion is versatile in that it can be used in a number of different ways. Let’s focus at first on image generation from text only (text2img). The image above shows an example text input and the resulting generated image (The actual complete prompt is here). Aside from text to image, another main way of using it is by making it alter images (so inputs are text + image).

How does Supervised learning models handle time-varying data

I need to train a supervised learning model which would take some input which differs in its output relating to time. to better understand my question I would give a simple binary classification, the model would receive an object that changes according to time so on time period t0 the output of the model related to that input differs from the output of the time period t1. I might be describing something either very easy and I missed it or something that doesn't yet exist. EDIT: I am preparing a model that classifies fruit based on its appearance to either be consumable or not. The only different thing in my work is that in specific time periods what was considered not consumable should be considered consumable so a picture of a fruit in a certain state could be classified both ways in general but in a specific time period it has only one classification.

How to get the conditional probabilities from joint probability table?

I have a table of 3 binary variables whose joint probability is given. a b c p(a,b,c) 0 0 0 0.192 0 0 1 0.144 0 1 0 0.048 0 1 1 0.216 1 0 0 0.192 1 0 1 0.064 1 1 0 0.048 1 1 1 0.096 I see that there is a joint probabilities formula for 3 variable $P(a,b,c) = P(c|a,b)P(a,b) = P(c|a,b)P(b|a)P(a)$ But when I cross-validate it with the table value and the formula it's not exact. What's happening here I am not getting it. It's not the same formula I assume. But how do I get the conditional probabilities from this table? If I can get the form, I can build the DAG. I can see some patterns like 0.192 and 0.048 is twice even though $a$ is changed. Then I thought maybe $a$ is independent of $b$ and $c$ ? I also find this formula $p(θ|X,α)=\frac{p(X|θ)p(θ|α)}{p(X|α)}$ So, I tried with a=1, b=1, c=1, $ P(c|a, b) * P(b|a) * P(a) = P(c|a, b) * \frac{P(b \cap a)}{P(a)} * P(a) = \frac{P(c\cap a \cap b)}{P(a\cap b)} * \frac{P(b \cap a)}{P(a)} * P(a) = \frac{1}{8} $

What is the information plane theorem for an autoencoder neural network?

Slide 8 (about 19 minutes into the video) of the Stanford Seminar - Information Theory of Deep Learning, Naftali Tishby has the following (rather informally stated) theorem. Theorem (Information Plane) For large typical $\mathbf{X}$ , the sample complexity of a DNN is completely determined by the encoder mutual information $\mathbf{I(X;T)}$ , of the last hidden layer; the accuracy (generalization error) is determined by the decoder information, $\mathbf{I(T;Y)}$ , of the last hidden layers. I am having difficulty following what is meant by "the sample complexity is completely determined". What is the precise statement of this theorem?

Why Phasic Policy Gradient (PPG) can update value function in auxiliary phase?

My questions is that how could we train the value network (separated from shared network) by using data from previous policies, which varies a lot since we collect data from different policies with many training phases done to start a auxiliary phase, especially returns are calculated by those policies. Wouldn't it hurt the stability of fitting value function?

keras model accuracy not improving

I am trying to do multi class(16) classification, however no matter what parameters or number of layers I use my accuracy is not improving, its in 30s the max I got was 43. I have tried early stopping to red overfilling but my testing accuracy is still low. I have 750 images in training and 350 in testing. I am also getting high traning accuracy vs low validation accuracy. features_train=features_train/255 features_test=features_test/255 cnn = models.Sequential([ layers.Conv2D(filters=16, kernel_size=(3, 3), activation='relu', strides=(2, 2), padding="same", input_shape=(224, 224, 3)), layers.MaxPooling2D((2,2)), layers.Dropout(0.25), layers.Conv2D(32 ,(3, 3), activation='relu'), layers.MaxPooling2D((2,2)), layers.Dropout(0.25), layers.Conv2D(64, (3, 3), activation='relu'), layers.MaxPooling2D((2,2)), layers.Dropout(0.25), layers.Conv2D(128, (3, 3), activation='relu'), layers.MaxPooling2D((2,2)), layers.Flatten(), layers.Dense(64,activation='relu'), layers.Dense(16, activation='softmax') ]) cnn.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy']) cnn.fit(features_train,labels_train,epochs=20, batch_size = 4 ,validation_split = 0.25)

Some Math behind Neural Tangent Kernel

Neural networks are well known to be over-parameterized and can often easily fit data with near-zero training loss with decent generalization performance on test dataset. Although all these parameters are initialized at random, the optimization process can consistently lead to similarly good outcomes. And this is true even when the number of model parameters exceeds the number of training data points. Neural tangent kernel (NTK) ( Jacot et al. 2018 ) is a kernel to explain the evolution of neural networks during training via gradient descent. It leads to great insights into why neural networks with enough width can consistently converge to a global minimum when trained to minimize an empirical loss. In the post, we will do a deep dive into the motivation and definition of NTK, as well as the proof of a deterministic convergence at different initializations of neural networks with infinite width by characterizing NTK in such a setting.

Understanding train vs validation loss chart

I am training an LSTM to a univariate time series and I have some questions about how to evaluate the train vs validations loss charts and which number of epochs to use in the model. To give more context about my data. It is a monthly univariate time series and the LSTM wants to predict the next 12 data points. The data is in sliding window format with 12 inputs and 12 outputs. A summary of the model is below. In both charts I see that the error in the validation dataset is smaller than the error in the training set. It means that I cannot generalize well so I am underfitting, right? The training and validation loss seems to converge around 40 epochs for the MAE loss and for the MSE. Should I use MAE as loss? As far as I know, MAE and MSE are the error metrics generally used for time series. Which number of epochs should I use for this model? #DEFINE THE MODEL lstm_model % layer_lstm(units = 12, #24, # size of the layer batch_input_shape = c(1, 12, 1), # batch size, timesteps, features return_sequences = TRUE, stateful = TRUE, name = "LSTM") %>% time_distributed(keras::layer_dense(units = 1), name = "Output") #COMPILE lstm_model %>% compile(loss = 'mae', optimizer = optimizer_adam(lr = 0.001, decay = 1e-6), metrics = 'mse') summary(lstm_model) #FIT THE MODEL validation_split = 0.25 train_history = lstm_model %>% fit( x = x_train_arr, y = y_train_arr, batch_size = 1, epochs = 100, verbose = 1, validation_split = validation_split, shuffle = FALSE )

How to solve Nonlinear least squares problem?

Initial idea is to use euclidean distances. But I do not understand how should I solve this task.

publications

See relevant faculty pages for publications Stephen Roberts Michael Osborne Xiaowen Dong Jan-Peter Calliess Stefan Zohren

members - [Faculty]

The Machine Learning Research Group comprises several groupings of Faculty, Postdocs and Students. Each member may have multiple local affiliations to sub-groups in the MLRG. Faculty Stephen Roberts Miake Osborne Xiaowen Dong Jan-Peter Calliess Stefan Zohren ---------- Principal Researchers Dr Vu Nguyen Dr Waqas Rafique Dr Ivan Kiskin

AAAI Announces New President-Elect and New Executive Council Members for 2022

AAAI is pleased to announce that Stephen Smith has been elected to serve as the next President-Elect. He will serve in that role for two years, and will then serve as President for two years. AAAI is also pleased to announce four incoming Councilors elected to three-year terms. The post AAAI Announces New President-Elect and New Executive Council Members for 2022 appeared first on AAAI .

Three versions of the independent two sample t-test (and R)

This post concerns three versions of the independent two sample t-test: Student's t-test uses a pooled standard deviation in the denominator (all equations are shamelessly copied from Wikipedia - Student's t-test ): $$ {\displaystyle t={\frac {{\bar {X}}_{1}-{\bar {X}}_{2}}{s_{p}\cdot {\sqrt {{\frac {1}{n_{1}}}+{\frac {1}{n_{2}}}}}}}} $$ where: $$ {\displaystyle s_{p}={\sqrt {\frac {\left(n_{1}-1\right)s_{X_{1}}^{2}+\left(n_{2}-1\right)s_{X_{2}}^{2}}{n_{1}+n_{2}-2}}}} $$ and: $$d.f. = n_1 + n_2 -2 $$ It is well known that Student's t-test can be seriously biased for unequal sample sizes and variances (Welch 1938 demonstrated this analytically, before computers and the many published simulation studies on this subject). Next is "an alternative criterion that has often been employed," which is less biased for unequal sample sizes and variances than is Student's t (Welch 1938). This version uses the standard error of the difference in the denominator: $$ {\displaystyle t={\frac {{\bar {X}}_{1}-{\bar {X}}_{2}}{s_{\bar {\Delta }}}}} $$ where: $$ {\displaystyle s_{\bar {\Delta }}={\sqrt {{\frac {s_{1}^{2}}{n_{1}}}+{\frac {s_{2}^{2}}{n_{2}}}}}.} $$ Note that if $ n_1 = n_2 $ or $ s_{X_{1}}^{2} = s_{X_{2}}^{2} $ then Student's t-test is identical to this alternative (I'll let the reader do the algebra). Welch (1938) presented an approximate degrees of freedom for the above statistic that further reduces bias (see Wikipedia - Welch's t-test for the formula). Edit: I replaced the exam…

Calculating absolute risk from odds ratio, relative risk, or hazard ratio

Say I have a known cumulative probability distribution for the probability (absolute risk) of developing breast cancer vs. age in a group representing the general population. I also know the odds ratio, relative risk, and hazard ratio for the development of breast cancer in a population that carries a cancer susceptibility gene mutation (ATM, for example). How would I calculate the cumulative probability distribution for the development of breast cancer vs. age in ATM mutation carriers? I appreciate your help!

Mathematics books for reinforcement learning

This question is not about the math prerequisites of reinforcement learning, but about the textbooks of mathematics that are enough to understand the literature on reinforcement learning. What are the mathematics books that are recommended to study in order to understand the majority of the reinforcement learning literature?

Different MSE for each observation for SARIMA-model

I am doing a time series forecast with the SARIMA-model in STATA. I wanted to calculate the MSE to set up confidence intervals for my prediction. I have done that using the following command: predict MSE, mse dynamic(tm(2022m5)) STATA then generates a new variable of course with the MSE in it. Now I am wondering why the MSE is not similar for each observation. It is about roughly the same value for all observations of one year with some exceptions. It heavily differs for different years though. For me (if i have not fundamentally misunderstood the concept of MSE) it just does not make sense to have different values for the MSE for each observation. I would apreciate any help.

#LearnAI Stories – Barnabas Chua

bimodal outcome - non normally distributed residuals

I have an outcome variable that is bimodal, this is because in about half the sample is measured from 0 to 5, and half the time from 0 to 7. Because of the different scales, I have decided to normalize this variable, but the normalization is particular; I discuss it below. In my study, there are two people who are assessed by three people, so there are six assessments; I rank these six assessments (1st, 2nd, 3rd..., 6th), subtract 1 so that the rank is 0th, 1st, ...5th, and divide by 5. In this way, this transformed variable goes from 0 to 1. The sample is composed of a few hundred of such six assessments (i.e. this gives about 1k assessments in total). As mentioned above about half of the times the three assessors asses the two assessed people on a scale from 0 to 5, while other times they assess them on a scale from 0 to 7. Within the triplet of assessors there is no scale variation, and what scale is going to be used within a triplet of assessors is random. I run an OLS with this transformed outcome on multiple explanatory variables, and cluster the standard errors at group and assessor level. If I run an OLS on the original variable, without normalization, the residuals are not normally distributed. Moreover, any other transformation of the outcome (i.e. natural logarithm; standardization at group level; unitization at group level, with zero minimum), with clustered standard errors at group and assessor level, do not solve the non-normality of the residuals. Have you hea…

How do you estimate the magnitude of seasonality in time-series data?

I have some time-series data showing the monthly counts of hospital admissions. It has both a long-term trend (increasing) and seasonality (highest in summer). I am trying to measure the magnitude of the seasonality (i.e. how much greater is the rate of admission in summer than in winter?). Here is some sample data (this is R code): ts We can fit Poisson models with and without seasonality, and plot the predicted values: model1 $pred1 pred2 Clearly there is strong evidence of seasonality: anova(model1, model2, test = 'LRT') Analysis of Deviance Table Model 1: N ~ period Model 2: N ~ period + month Resid. Df Resid. Dev Df Deviance Pr(>Chi) 1 58 202.753 2 47 62.334 11 140.42 However, how do we measure the magnitude of this seasonality (rather than the statistical evidence for seasonality or goodness-of-fit of the model)? My first thought was to calculate the extent to which residuals are reduced by adding a term for seasonality into the model, in comparison to the data. # calculate residuals: ts $resid1 pred1 - ts $N ts$ resid2 $pred2 - ts$ N # calculate the reduction in residuals in comparison to the data (sum(abs(ts $resid1)) - sum(abs(ts$ resid2))) / sum(ts$N) This suggests there is 2.7% seasonality in this data. Does this make sense? Is there a standard approach to this problem?

Finding coefficients for a loss function with multiple parts and different scales

I'm working on reusing the paper Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation , which deforms a 3D mesh to make it fit some 2D images, in another scenario than the original one. The loss function is comprised of four terms : the chamfer loss, the cosine loss, a Laplacian regularization and an edge length regularization ( description here ). These 4 terms have very different scales and are somewhat unbound, meaning that I can't normalize all of them using a maximum value and that if I leave them as-is, some of them will be lost (lol) in the optimization. To add to the problem, not all parts of the loss are equally important : the chamfer loss, minimizing the distance between the expected and actual meshes, should for example have a higher weight in the optimization when compared to the regularization factors. In the github repository , some coefficients were found that may or may not be optimal for the problem at hand. I'm guessing they're not bad, considering that's what the authors decided to use. However, the method used to find these coefficients isn't explained anywhere (I even tried opening a github issue regarding this but I didn't get any answer). Now, I've read many questions here regarding optimization of the questions of multi-loss optimization and loss scaling, but I still don't know how to approach the problem of finding the optimal coefficients. I thought about using a hyperparameter optimizer like Hyperband to find the best coefficients, but what…

Can I use Sentence-Bert to embed event triples?

I extracted event triples from sentences using OpenIE. Can I concatenate the components in the event triple to make it a sentence and use Sentence-Bert to embed the event? It seems no one has done this way before so I am questioning my idea.

Generalized Visual Language Models

Processing images to generate text, such as image captioning and visual question-answering, has been studied for years. Traditionally such systems rely on an object detection network as a vision encoder to capture visual features and then produce text via a text decoder. Given a large amount of existing literature, in this post, I would like to only focus on one approach for solving vision language tasks, which is to extend pre-trained generalized language models to be capable of consuming visual signals .

LinkBERT: Improving Language Model Training with Document Link

Language Model Pretraining Language models (LMs), like BERT 1 and the GPT series 2 , achieve remarkable performance on many natural language processing (NLP) tasks. They are now the foundation of today’s NLP systems. 3 These models serve important roles in products and tools that we use every day, such as search engines like Google 4 and personal assistants like Alexa 5 . These LMs are powerful because they can be pretrained via self-supervised learning on massive amounts of text data on the web without the need for labels, after which the pretrained models can be quickly adapted to a wide range of new tasks without much task-specific finetuning. For instance, BERT is pretrained to predict randomly masked words in original text (masked language modeling), e.g. predicting the masked word “dog” from “My __ is fetching the ball”. GPTs are pretrained to predict the next word given a previous sequence of text (causal language modeling), e.g. predicting the next word “ball” from “My dog is fetching the”. In either cases, through pretraining, LMs learn to encode various knowledge from a text corpus that helps to perform downstream applications involving language understanding or generation. In particular, LMs can learn world knowledge (associations between concepts like “dog”, “fetch”, “ball”) from training text where the concepts appear together, and help for knowledge-intensive applications like question answering. 6 Challenges. A challenge with most common LM pretraining strateg…

Stanford AI Lab Papers and Talks at ACL 2022

The 60th Annual Meeting of the Association for Computational Linguistics (ACL) 2022 is taking place May 22nd - May 27th. We’re excited to share all the work from SAIL that’s being presented, and you’ll find links to papers, videos and blogs below. Feel free to reach out to the contact authors directly to learn more about the work that’s happening at Stanford! List of Accepted Papers LinkBERT: Pretraining Language Models with Document Links Authors : Michihiro Yasunaga, Jure Leskovec*, Percy Liang* Contact : myasu@cs.stanford.edu Links: Paper | Website Keywords : language model, pretraining, knowledge, hyperlink, bionlp When classifying grammatical role, BERT doesn’t care about word order… except when it matters Authors : Isabel Papadimitriou, Richard Futrell, Kyle Mahowald Contact : isabelvp@stanford.edu Links: Paper Keywords : large language models, analysis, word order, order invariance, grammatical role, syntax, semantics Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words Authors : Kaitlyn Zhou, Kawin Ethayarajh, Dallas Card, Dan Jurafsky Contact : katezhou@stanford.edu Keywords : cosine similarity, training data frequency, model analysis Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization Authors : Faisal Ladhak, Esin Durmus, He He, Claire Cardie, Kathleen McKeown Contact : esdurmus@stanford.edu Links: Paper Keywords : text summarization, text generation, evaluation, faithfulness Sp…

Comment on Beyond multistakeholder tokenism: A provisional examination of participation in the IGF by babul

It is рerfect time to make somе plans for the futurе and it is time to be hapρy. I have learn tһiѕ submіt and if I may just I desire to suggest you few fascinating things or ѕuggestions. Mɑybe you could write next articles referring to this article. I want to learn more issues approximately it!

Making of AI4K Storybook, Daisy and her AI Friends

Is there any variant of perceptron convergence algorithm that ensures uniqueness?

The perceptron convergence algorithm given below ensures the convergence of weights of the perceptron provided enough data points and iterations. Although it ensures convergence by finally getting a decision hyperplane that can separate positive samples (P) from negative samples N. It does not ensure the uniqueness of the decision hyperplane. The solution is not unique, because there are more than one hyperplanes separating two linearly separable classes. Are there any variants to this algorithm in the literature that are capable of ensuring uniqueness?

Multiple linear regression: Do all independent variables need to have good adjusted R-squared independently?

I'm very sorry if this should be obvious, I'm just feeling a little lost with this assignment.. I have four independent variables X1,X2,X3,X4 plus a constant, modelled against Y. I know X4 to be heavily correlated already, it's mostly a control variable. I've checked for multicollinearity. There are 52 observations. These are the results with only one independent variable at a time: X1 X2 X3 X4 coefficient 0.77 -0.32 0.34 0.95 p-value 0.0001 0.03 0.027 0.00005 adjusted r-squared 0.567 0.0632 0.074 0.645 And these are the results when combined with X4: (X1, X4) (X2, X4) (X3, X4) coefficient (0.43, 0.64) (-0.34, 0.96) (-0.03, 0.93) p-value (0.00001, 0.0000000084) (0.000031, 0.00000001) (0.73, 0.000006) adjusted r-squared 0.757 0.747 0.63 I'm not sure if it's relevant, but the constant terms is varying positive and negative, sometimes with a significant p-value and sometimes not. My question is: X2 only has 0.06 adjusted r-squared with Y by itself, while X4 has 0.645 by itself. But combined, r-squared increases to 0.747. Does that mean something is wrong with my model? Or that the tiny variance in Y that X2 explains (6%) is not included in X4, so that X2 is actually a significant variable in the model? Is 0.1 increase in r-squared even enough to say the combined model is better? Please help!

Which model should I use for Time series data?

I need to regress one dependent variable (dummy variable), against several other independent variables (dummy and non dummy variables). (FYI : I'm not using the past performance of dependent variable, the independent variables are different.) I have time series data for loans. My dependent variable is default status (1, if defaulted , 0 otherwise) and independent variables are loan characteristics. I know I need to check for stationarity among the dependent and independent variables. But what is the next step? Can I use classical linear regression model after I check for stationarity (and the data is stationary)? Is there a specific model to use in this case. Can I then just follow up with the residual diagnostic tests and stability diagnostic test (Ramsey's reset)? Is this sufficient? If not, can someone explain what is step by step process in layman's terms. I am quite new to econometrics and would really appreciate any help. Thank you @InstitutoEconometriaLima Thank you so much for your response. No the dataset is the loanbook of one institution for 2 years. It includes a balanced sample of all loans provided in this period. I considered it as time series data as the variables are all of one institution. Although I do not use lagged values of Y (dependent variables) as the explanatory variables, the multiple independent variables are all derived from the same dataset. My dependent variable is default status of loans and independent variables are loan characeteristics. M i…

Stanford AI Lab Papers and Talks at ICLR 2022

The International Conference on Learning Representations (ICLR) 2022 is being hosted virtually from April 25th - April 29th. We’re excited to share all the work from SAIL that’s being presented, and you’ll find links to papers, videos and blogs below. Feel free to reach out to the contact authors directly to learn more about the work that’s happening at Stanford! List of Accepted Papers Autonomous Reinforcement Learning: Formalism and Benchmarking Authors : Archit Sharma*, Kelvin Xu*, Nikhil Sardana, Abhishek Gupta, Karol Hausman, Sergey Levine, Chelsea Finn Contact : architsh@stanford.edu Links: Paper | Website Keywords : reinforcement learning, continual learning, reset-free reinforcement learning MetaShift: A Dataset of Datasets for Evaluating Contextual Distribution Shifts and Training Conflicts Authors : Weixin Liang, James Zou Contact : wxliang@stanford.edu Links: Paper | Video | Website Keywords : benchmark dataset, distribution shift, out-of-domain generalization An Explanation of In-context Learning as Implicit Bayesian Inference Authors : Sang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu Ma Contact : xie@cs.stanford.edu Links: Paper | Video Keywords : gpt-3, in-context learning, pretraining, few-shot learning GreaseLM: Graph REASoning Enhanced Language Models for Question Answering Authors : Xikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren, Percy Liang, Christopher D. Manning, Jure Leskovec Contact : xikunz2@cs.stanford.edu Award nominations: Sp…

Testing for difference in means between two groups for which each measurement has a unique associated uncertainty

I am wondering what type of statistical test would be appropriate to test for a difference in means between two groups for which each measurement has a different uncertainty. For example, I want to see if there is a significant difference in height when comparing men and women, but each person was measured using a different type of ruler (let's say one ruler only measures up to a precision of 1 cm, another to 1 mm, etc). So, I would have a set of measurements like: m = [170 cm, 150 cm, 200 cm] w = [150 cm, 120 cm, 170 cm] u_m = [1 cm, 0.5 cm, 0.1 cm] u_w = [0.1 cm, 0.25 cm, 1 cm] where order is preserved (i.e., u_m[1] is the uncertainty of m[1]). What type of test could tell me if there is a significant difference in means (outputting a significance value) with the associated uncertainties taken into account?

Learning with not Enough Data Part 3: Data Generation

Here comes the Part 3 on learning with not enough data (Previous: Part 1 and Part 2 ). Let’s consider two approaches for generating synthetic data for training. Augmented data . Given a set of existing training samples, we can apply a variety of augmentation, distortion and transformation to derive new data points without losing the key attributes. We have covered a bunch of augmentation methods on text and images in a previous post on contrastive learning. For the sake of post completeness, I duplicate the section on data augmentation here with some edits. New data . Given few or even no data points, we can rely on powerful pretrained models to generate a number of new data points. This is especially true in recent years given the fast progress in large pretrained language models (LM) . Few shot prompting is shown to be effective for LM to learn within context without extra training. Data Augmentation The goal of data augmentation is to modify the input format (e.g. text wording, visual appearance) while the semantic meaning stays unchanged.

How do I use A.I. to analyse & score news articles?

I'm working on a project that would benefit from using A.I. or machine learning to analyse news feeds from a variety of websites and grade each article between 0 and 10. We would manually grade hundreds of articles to train the A.I. on what we like and what we don't like using the scoring range. The A.I. is expected to learn how we grade by identifying similarities between articles. When the A.I. starts to grade similar to humans, then we would go more hands of and leave this task to the A.I. Not sure where to start with A.I. what tools and approaches would be the easiest to achieve this?

Discovering the systematic errors made by machine learning models

Discovering systematic errors with cross-modal embeddings In this blog post, we introduce Domino, a new approach for discovering systematic errors made by machine learning models. We also discuss a framework for quantitatively evaluating methods like Domino. Links: 📄 Paper (ICLR 2022) 🌍 Longer Walkthrough 💻 GitHub 📘 Docs 📒 Google Colab Machine learning models that achieve high overall accuracy often make systematic errors on coherent slices of validation data. What is a slice? A slice is a set of data samples that share a common characteristic. As an example, in large image datasets, photos of vintage cars comprise a slice (i.e. all images in the slice share a common subject). The term slice has a number of synonyms that you might be more familiar with (e.g. subgroup, subpopulation, stratum). These terms are largely interchangeable, but we’ll stick with “slice” throughout this post. We say that a model underperforms on a slice if performance on the data samples in the slice is significantly worse than its overall performance. The search for underperforming slices is a critical, but often overlooked, part of model evaluation. When practitioners are aware of the slices on which their models underperform, they can make more informed decisions around model deployment. This is particularly important in safety-critical settings like medicine: a diagnostic model that underperforms on younger patients should likely not be deployed at a pediatric hospital. Slice awareness can also he…

model.fit vs model.evaluate gives different results?

The following is a small snippet of the code, but I'm trying to understand the results of model.fit with train and test dataset vs the model.evaluate results. I'm not sure if they do not match up or if I'm not understanding how to read the results? batch_size = 16 img_height = 127 img_width = 127 channel = 3 #RGB train_dataset = image_dataset_from_directory(Train_data_dir, shuffle=True, batch_size=batch_size, image_size=(img_height, img_width), class_names = class_names) ##Transfer learning code from mobilenetV2/imagenet here to create model initial_epochs = 10 history = model.fit(train_dataset, validation_data=test_dataset, epochs=initial_epochs) train accuracy: 0.8030709732662548 from history.history['accuracy'] train loss: 0.26204305738210676 from history.history['loss'] test accuracy: 0.6920163252136924 from history.history['val_accuracy'] test loss: 0.6397175192832947 from history.history['val_loss'] The below are the shapes and datatypes of the datasets and how I'm passing them to the model.fit: train_dataset: Found 5216 files belonging to 2 classes. test_dataset: Found 624 files belonging to 2 classes. initial_epochs = 10 history = model.fit(train_dataset, validation_data=test_dataset, epochs=initial_epochs) Then I tried to check the evaluate method on the test_dataset and I'm not sure if what I'm doing is correct, but it doesn't seem to match what the fit method results give: score = model.evaluate(test_dataset, verbose=0) print(score) print(f'Test loss: {score[0]} /…

Grading Complex Interactive Coding Programs with Reinforcement Learning

[Summary] tl;dr: A tremendous amount of effort has been poured into training AI algorithms to competitively play games that computers have traditionally had trouble with, such as the retro games published by Atari, Go, DotA, and StarCraft II. The practical machine learning knowledge accumulated in developing these algorithms has paved the way for people to now routinely train game-playing AI agents for many games. Following this line of work, we focus on a specific category of games – those developed by students as part of a programming assignment. Can the same algorithms that master Atari games help us grade these game assignments? In our recent NeurIPS 2021 paper , we illustrate the challenges in treating interactive coding assignment grading as game playing and introduce the Play to Grade Challenge. Introduction Massive Online Coding Education has reached striking success over the past decade. Fast internet speed, improved UI design, code editors that are embedded in a browser window allow educational platforms such as Code.org to build a diverse set of courses tailored towards students of different coding experiences and interest levels (for example, Code.org offers “Star War-themed coding challenge,” and “Elsa/Frozen themed for-loop writing”). As a non-profit organization, Code.org claims to have reached over 60 million learners across the world 1 . Such organizations typically provide a variety of carefully constructed teaching materials such as videos and programming…

Deep Neural Nets: 33 years ago and 33 years from now

The Yann LeCun et al. (1989) paper Backpropagation Applied to Handwritten Zip Code Recognition is I believe of some historical significance because it is, to my knowledge, the earliest real-world application of a neural net trained end-to-end with backpropagation. Except for the tiny dataset (7291 16x16 grayscale images of digits) and the tiny neural network used (only 1,000 neurons), this paper reads remarkably modern today, 33 years later - it lays out a dataset, describes the neural net architecture, loss function, optimization, and reports the experimental classification error rates over training and test sets. It’s all very recognizable and type checks as a modern deep learning paper, except it is from 33 years ago. So I set out to reproduce the paper 1) for fun, but 2) to use the exercise as a case study on the nature of progress in deep learning. Implementation . I tried to follow the paper as close as possible and re-implemented everything in PyTorch in this karpathy/lecun1989-repro github repo. The original network was implemented in Lisp using the Bottou and LeCun 1988 backpropagation simulator SN (later named Lush). The paper is in french so I can’t super read it, but from the syntax it looks like you can specify neural nets using higher-level API similar to what you’d do in something like PyTorch today. As a quick note on software design, modern libraries have adopted a design that splits into 3 components: 1) a fast (C/CUDA) general Tensor library that implement…

Applying massive language models in the real world with Cohere

A little less than a year ago, I joined the awesome Cohere team. The company trains massive language models (both GPT-like and BERT-like) and offers them as an API (which also supports finetuning). Its founders include Google Brain alums including co-authors of the original Transformers paper. It’s a fascinating role where I get to help companies and developers put these massive models to work solving real-world problems. I love that I get to share some of the intuitions developers need to start problem-solving with these models. Even though I’ve been working very closely on pretrained Transformers for the past several years (for this blog and in developing Ecco), I’m enjoying the convenience of problem-solving with managed language models as it frees up the restrictions of model loading/deployment and memory/GPU management. These are some of the articles I wrote and collaborated on with colleagues over the last few months: Intro to Large Language Models with Cohere This is a high-level intro to large language models to people who are new to them. It establishes the difference between generative (GPT-like) and representation (BERT-like) models and examples use cases for them. This is one of the first articles I got to write. It's extracted from a much larger document that I wrote to explore some of the visual language to use in explaining the application of these models. A visual guide to prompt engineering Massive GPT models open the door for a new way of programming. If yo…