AI/ML News & Innovations Hub

Cross Validated 2022-10-18 16:54 UTC Score 9.0 AI-113-20221018-social-media-85290277 Full article

Why is VQ-VAE considered a variational encoder?

I'm reading about VQ-VAE and I'm not sure why do they say we can view it as a VAE . Can you explicitly show: what is the latent z-space - is it the discrete space where z can take the integers 1..K where K is the number of embedding vectors? what is the true posterior p(z|x)? is it indeed intractable to use for calculating the marginal p(x), if we are dealing just in a scalar that can hold integers 1..K where K is a moderate 128? why do we need to sample from z's distribution in VAE while in VQ-VAE we just take a deterministic distribution and just use the closest embedding to the encoder output? can you show the ELBO formulation, and show how it boils down to only $\log p(x|z_q(x))$ ?

AI Chips & Hardware

Read →

Cross Validated 2022-10-17 15:12 UTC Score 9.0 AI-113-20221017-social-media-d3589beb

Efficiently calculating leave-one-out conditional multivariate normal distributions

I have a multivariate normal distribution for vector $\mathbf{x}$ with mean vector $\boldsymbol{\mu}$ and covariance matrix $\boldsymbol{\Sigma}$ . In my specific use-case, $\boldsymbol{\Sigma}$ is actually a correlation matrix. For the sake of calculating some leave-one-out log-likelihood values downstream, I need to efficiently calculate conditional distributions for each dimension (i.e. get the conditional distribution for $\mathbf{x_1}$ when setting the other values to a pre-specified vector $\mathbf{a}$ ). As shown on the relevant wikipedia page , these are: $$\bar{\boldsymbol\mu} = \boldsymbol\mu_1 + \boldsymbol\Sigma_{12} \boldsymbol\Sigma_{22}^{-1} \left( \mathbf{a} - \boldsymbol\mu_2 \right) \\ \overline{\boldsymbol\Sigma} = \boldsymbol\Sigma_{11} - \boldsymbol\Sigma_{12} \boldsymbol\Sigma_{22}^{-1} \boldsymbol\Sigma_{21} $$ Where $\boldsymbol{\Sigma}_{11}$ , $\boldsymbol{\Sigma}_{12}$ , $\boldsymbol{\Sigma}_{21}$ and $\boldsymbol{\Sigma}_{22}$ represent sub-blocks of $\boldsymbol{\Sigma}$ . In my case I need every leave-one-out conditional distribution, so repeatedly computing $\boldsymbol\Sigma_{22}^{-1}$ becomes computationally burdensome. I think the complexity for one inverse (which is feasible) is $O((n-1)^3)$ , so $n$ of those is $O(n(n-1)^3)$ which is too expensive. For my purposes, $n$ gets up to the neighborhood of ~5000. This might be nothing more than a simple linear algebra problem. My instinct is to compute $\boldsymbol{\Sigma}^{-1}$ once at first, the…

Read →

AI Stack Exchange 2022-10-17 07:49 UTC Score 23.0 AI-110-20221017-social-media-bf6ece3a Full article

Dummy variable trap in neural networks and class visualization

Let's say I have data records looking like that: (x1, x2, x3, x4, ..., x100) , where each x can be either alpha , gamma or omega . An example of record could be ('gamma', 'alpha', 'omega', 'alpha', ..., 'gamma') . So the shape of my dataset is (N, 100) (with N the number of records). I want to train a neural network to predict some binary label. As there is no underlying ordering in my input categories, I use dummy variables to feed my network. Therefore, I end up with a dataset of the following shape: (N, 100, 3) . My problem is that I don't really know how to deal with the dummy variable trap. According to this answer , I should drop one category when I use a network without weight decay. However, I thought that even without weight decay, the non-linearity of neural networks (assuming I'm using non-linear activation functions like relu), would be enough to avoid the issue without actually needing to drop one category. Am I wrong? Would a neural network without weight decay behave badly if I do not drop one column? Some context Ideally, I would like to avoid dropping one column as my next step is to create inputs that maximize a class prediction (starting from noise and using gradient ascent to "improve" the input). If I do that with a model trained with a dropped category, I can end up with values close to 0 for my (n-1) categories, probably meaning that the category that would maximize the output would be the dropped one. This interpretation looks correct to me, but it le…

AI Chips & Hardware AI Research & Papers

Read →

Cross Validated 2022-10-15 00:59 UTC Score 28.0 AI-113-20221015-social-media-0477b739

Cross lingual transfer for summarisation using XLM-R

I have a question. There's a library (uses this paper) which suggests in its cross lingual part that if the XLM-R is trained in english dataset, it can be directly applied to datasets in other languages, and zero shot cross lingual transfer can be conducted. So, my question is, if I trained XLM-R for english summarisation task, will it be able to transfer that knowledge and generate summaries in other languages using zero shot cross lingual transfer? I already have code written and tested for small dataset, but it requires a good amount of computational power for the whole dataset, so that's the reason for asking this question. Edit: A little update for anyone who was interested in this question. I tried to train the XLM-R on a very little portion of food review dataset (5500 examples), it still seems to output some summaries (1-2 words on test dataset). This was for english language. I switched the language to german (I think it's lexically similar to english. I might be wrong as neither of the languages are my first language, anyways), but didn't train the model on the german dataset, and the output was horrible. It's giving the same words as the summaries. I even switched the language to japanese for the test dataset alone, still no results were found (as expected).

AI Chips & Hardware AI Research & Papers

Read →

Cross Validated 2022-10-12 18:10 UTC Score 26.0 AI-113-20221012-social-media-6553d71b

What kind of architecture to use for non-binary output multi-label image clasification

I want to make a network for making multi-label attribute classifications on images of clothing. This is a simplified case of what I want to do, I have 9 different attribute categories that I wish to detect on the image of a T-shirt: Color, Arm Type, Collar type, Pattern, Fabric Type, Size, Usage Area, and Style. However, you may notice that these attributes are non-binary, for example, color attribute of the clothing can take 18 different values or the collar type can take 11 different values like a round collar, bicycle collar, turtleneck, etc. Most tutorials I could find on the internet are mostly for binary labels like is the picture of a celebrity is bald or not, blonde or not. It is all 1 or 0. Another important point is that not every attribute has a value for every entry in my dataset. I am planning to train this on a dataset that I custom-made myself and there are different lines of clothing than just t-shirts. For example, in the data labeling files of my dataset, there is not a "collar type" specified for a picture of pants because pants obviously do not even have a collar. So I want to train a multi-label classification model with 9 different outputs with images in a database that has 3-4 attributes labeled each. I am an engineering student who is just getting in on machine learning and AI so it would be awesome if you can include similar tutorials, articles etc in your answers so I can self-teach myself how to do it. Thank you so much Note: I am adding an excel…

AI Chips & Hardware AI Research & Papers

Read →

AI Stack Exchange 2022-10-12 15:30 UTC Score 23.0 AI-110-20221012-social-media-16896338 Full article

machine learning for a budgeting application

I am interested in finding references and previous applications where prior year budgets are analyzed to provide guidance for a current year budget. Specifically, each year some two thousand items are evaluated for funding, with perhaps 500 funded in that year. Information is available in a spreadsheet with multiple parameters that are manually evaluated to determine if an individual item is funded in the budget. I would appreciate any guidance as to how best to make use of such data for say the previous 5 years, where I know what has been funded in those years, to assist in screening items for the current budget year, in particular what approach to ML would be best. I have attempted a literature search but have not found anything directly relevant. Edit: Found this reference in my literature search, looks to be applicable: https://www.datacamp.com/courses/case-study-school-budgeting-with-machine-learning-in-python

AI Funding & Startups AI Research & Papers

Read →

AI Stack Exchange 2022-10-12 07:49 UTC Score 9.0 AI-110-20221012-social-media-00c7f595 Full article

For which problem sizes is Deep Q-Learning suitable and why?

I am wondering for which problem sizes a Deep Q-Learning algorithm is most appropriate. For example, whether it is particularly suited for low complexity problems or not for high complexity problems. And if that is the case, why?

Read →

AI Stack Exchange 2022-10-11 12:14 UTC Score 14.0 AI-110-20221011-social-media-35d60c29 Full article

How to discover/approximate the causations/correlations between multiple time-series and related open source libraries?

I have the following time-series data with two value columns. (t: time, v1: time-series values 1, v2: time-series values 2) t | v1 | v2 ---+----+---- 1 | 1 | 0 2 | 2 | 2 3 | 3 | 4 4 | 3 | 6 5 | 3 | 6 6 | 4 | 6 7 | 5 | 8 (7 rows) I am trying to discover (or approximate) the correlation between the $v1$ and $v2$ , and use that approximation for the next step predictions. Please note, the most obvious correlation is $v2(t)=2.v1(t-1)$ . My question is, what are the algorithms to employ for such approximations and are there any open source implementations of those algorithms for SQL/python/javascript?

Open Source AI AI Funding & Startups

Read →

Cross Validated 2022-10-11 08:38 UTC Score 20.0 AI-113-20221011-social-media-f975aed7

Combine two data sets from two different regions

This is a actually very basic question, but I can't get my head around it. I have two datasets for Europe and U.S. that contain the same two variables. These two variables are in a linear relationsship, which I proved by statistical t-testing. Now I want to check if the two regional differentiated data sets can be combined and are in fact based upon one mechanism. Should I simply combine the data, implement a new linear relationsship and t-test the entirety again on statistical signifance? Or is there a other way? Best!

AI Research & Papers

Read →

Cross Validated 2022-10-10 14:24 UTC Score 21.0 AI-113-20221010-social-media-56533333

Repeated measures with nested data mixed effects model (singularity)

I hope you can help me. I performed a pre-post study with two Trainings (ViStra & LeStra), and I measured safety outcomes (e.g. Knowledge, attitudes, behavior, etc...) with questionnaires before (T1) and after (T2) the training ( Time ). Participants ( ID ) took just one of the Trainings were nested into 5 companies , and we want to know if there are differential changes in the trainings over time. Participants per company per training V-Training L-Training B 10 14 C 9 10 G 18 14 H 6 8 K 12 14 U 31 32 Therefore, I used GLMM to evaluate safety training outcomes, like this: Model_A When I run the model, the first warning appears: boundary (singular) fit: see help('isSingular') And when I want to calculate the ICC the following message appears: performance::icc(Model_A, by_group=TRUE) Can't compute random effect variances. Some variance components equal zero. Your model may suffer from singularity (see `?lme4::isSingular` and `?performance::check_singularity`). Solution: Respecify random structure! You may also decrease the `tolerance` level to enforce the calculation of random effectn variances. However, when I run the following model (take a look into the random effects): Model_B No warning appears and it does calculate the ICC. What am I doing wrong? Participants (IDs) are assumed to be nested in companies. Is it possible that it is because within each company there are few participants? Thank you. Here you have the Data in Long format (ID is twice as it was tested before an…

AI Research & Papers

Read →

AI Stack Exchange 2022-10-09 16:34 UTC Score 23.0 AI-110-20221009-social-media-ca0b3007

Which model should I apply on sequential data?

I need to predict a binary vector given a sequential dataset meaning the current datapoint depends on its predecessors as well as (known) successors. So, it looks something like this: Given the sequence: X = [x_1, x_2, x_3, ..., x_N] I want to predict: Y = [y_1, y_2, y_3, ..., y_N] with y_i \in {0, 1} a binary label Now, the sequence X is fully known, that is: future observations in the sequence are completely known at any time. Therefore, in contrast to normal time series data, I can also harness any x_i+1, x_i+2,... from the full X sequence for predicting y_i at any time and not only x_i-2, x_i-1, x_i etc. Also the data X is a sequence of R^dxN vectors, i.e. N d-dimensional datapoints containing real numbers. In my case the dimensionality of the data is d=140 . Now, what I want to predict is the following: What is y_i given X , e.g. what is y_3 given the observations x_1, x_2, x_3, x_4, x_5 ? So, eventually I need something like this: for i in range(N): predict y_i given X = [x_1, x_2, x_3, ..., x_N] This actually is a many-to-one prediction task. Now you could use a RNN, but the problem imho is that "future" observations are not taken into account when applying an RNN. Maybe I am wrong about this assumption. But therefore I am asking: Which model would you suggest to use for this problem?

AI Research & Papers

Read →

Cross Validated 2022-10-08 15:00 UTC Score 15.0 AI-113-20221008-social-media-c646161d

Random Forest Models for Time Series and Cross Validation

If one were to build a model using a random forest model that uses lagged variables, for simplicity we'll describe this just using a single feature describing lag 1: $x_{t-1}$ . Which attempts to predict $x_{t}$ Will this model still be subject to standard time series CV rules? I believe the feature vector from one instance to another will be independant and therefore a forecast of rolling origin isn't required and standard K-Fold can apply? Is there any issues that can arise from not doing a rolling origin CV under this context?

AI Funding & Startups

Read →

AI Stack Exchange 2022-10-08 04:22 UTC Score 15.0 AI-110-20221008-social-media-2450280e Full article

What does the adversarial loss in a GAN represent?

I'm working on Pix2Pix an image-to-image translation GAN, and I noticed that there is an adversarial loss implemented using BCE, and a L1 loss implemented using MAE. I know L1 loss represents the difference between the predicted image and actual image, but I am not sure what does the GAN adversarial loss represent? This is the official definition The adversarial loss influences whether the generator model can output images that are plausible in the target domain but the meaning is tough to understand. Is it representing the difference between the predicted probability distribution and actual probability distribution?

AI Chips & Hardware

Read →

AI Stack Exchange 2022-10-07 17:02 UTC Score 9.0 AI-110-20221007-social-media-a1a68ee0

References on Theoretical Bandit Problem

I am going to start learning the bandit problem and algorithm, especially how to bound the regret. I found the book `` Bandit Algorithms'' but it is not easy to follow. It is based on advanced stochastic processes and measure theory in some cases. I am wondering if there are any lecture notes, or courses to start.

Read →

Jay Alammar Blog 2022-10-04 00:00 UTC Score 33.0 USR-0113-20221004-ai-specialis-dafdda9c Full article

The Illustrated Stable Diffusion

Translations: Chinese, Vietnamese. (V2 Nov 2022: Updated images for more precise description of forward diffusion. A few more images in this version) AI image generation is the most recent AI capability blowing people’s minds (mine included). The ability to create striking visuals from text descriptions has a magical quality to it and points clearly to a shift in how humans create art. The release of Stable Diffusion is a clear milestone in this development because it made a high-performance model available to the masses (performance in terms of image quality, as well as speed and relatively low resource/memory requirements). After experimenting with AI image generation, you may start to wonder how it works. This is a gentle introduction to how Stable Diffusion works. Stable Diffusion is versatile in that it can be used in a number of different ways. Let’s focus at first on image generation from text only (text2img). The image above shows an example text input and the resulting generated image (The actual complete prompt is here). Aside from text to image, another main way of using it is by making it alter images (so inputs are text + image).

Stability AI AI Chips & Hardware

Read →

AI Stack Exchange 2022-10-03 18:30 UTC Score 18.0 AI-110-20221003-social-media-f2b47331 Full article

How does Supervised learning models handle time-varying data

I need to train a supervised learning model which would take some input which differs in its output relating to time. to better understand my question I would give a simple binary classification, the model would receive an object that changes according to time so on time period t0 the output of the model related to that input differs from the output of the time period t1. I might be describing something either very easy and I missed it or something that doesn't yet exist. EDIT: I am preparing a model that classifies fruit based on its appearance to either be consumable or not. The only different thing in my work is that in specific time periods what was considered not consumable should be considered consumable so a picture of a fruit in a certain state could be classified both ways in general but in a specific time period it has only one classification.

AI Chips & Hardware

Read →

Cross Validated 2022-10-01 13:27 UTC Score 9.0 AI-113-20221001-social-media-b90c8ad2

Lyapunov CLT for dependent random variables

Suppose $\{X_{1},\ldots ,X_{d}\}$ is a sequence of independent random variables, each with finite expected value $ E[X_{i}]$ and variance $ \text{Var}[X_{i}]$ . We define $$s_{d}^2 = \sum_{i=1}^{d} \text{Var}[X_{i}].$$ If for some $\delta >0$ , the following Lyapunov's condition holds true \begin{align*} &\lim_{d \to \infty} \frac{1}{s_{d}^{2+\delta}} \sum_{i=1}^{d} E\left[ |X_{i} - E[X_i]|^{2+\delta}\right] = 0, \qquad \text{then},\\ &\frac{1}{s_{d}} \sum_{i=1}^{d} (X_{i} - E[X_{i}]) \overset{\mathcal{D}}{\to} \mathcal{N}(0,1), \end{align*} as $d$ tends to infinity. Here $\overset{\mathcal{D}}{\to}$ signifies the convergence in distribution. Is there any similar kind of Lyapunov's result that exists for dependent random variables $\{X_1,…, X_d\}$ ? Any help or lead would be highly appreciated.

Read →

Cross Validated 2022-09-29 18:12 UTC Score 9.0 AI-113-20220929-social-media-65b9fd18 Full article

How to get the conditional probabilities from joint probability table?

I have a table of 3 binary variables whose joint probability is given. a b c p(a,b,c) 0 0 0 0.192 0 0 1 0.144 0 1 0 0.048 0 1 1 0.216 1 0 0 0.192 1 0 1 0.064 1 1 0 0.048 1 1 1 0.096 I see that there is a joint probabilities formula for 3 variable $P(a,b,c) = P(c|a,b)P(a,b) = P(c|a,b)P(b|a)P(a)$ But when I cross-validate it with the table value and the formula it's not exact. What's happening here I am not getting it. It's not the same formula I assume. But how do I get the conditional probabilities from this table? If I can get the form, I can build the DAG. I can see some patterns like 0.192 and 0.048 is twice even though $a$ is changed. Then I thought maybe $a$ is independent of $b$ and $c$ ? I also find this formula $p(θ|X,α)=\frac{p(X|θ)p(θ|α)}{p(X|α)}$ So, I tried with a=1, b=1, c=1, $ P(c|a, b) * P(b|a) * P(a) = P(c|a, b) * \frac{P(b \cap a)}{P(a)} * P(a) = \frac{P(c\cap a \cap b)}{P(a\cap b)} * \frac{P(b \cap a)}{P(a)} * P(a) = \frac{1}{8} $

Read →

Cross Validated 2022-09-28 19:47 UTC Score 17.0 AI-113-20220928-social-media-0f0afd7e

Is it incorrect to calculate residuals directly from a phylogenetically-controlled linear regression?

I would like to calculate the residuals from a regression of log body mass and log brain mass, controlling for phylogeny. I originally used phylolm in R to run this regression, under a Brownian Motion assumption. I then saw that there is a function (phyl.resid) in phytools that is designed to calculate this, so I used that (again, set under the Brownian Motion assumption). I compared the two methods and the results are very different (I've double-checked that they are both using the same data and tree). Is the first method wrong? I also ran a non-phylogenetic regression and the pattern of residuals from that is almost identical to that given by phyl.resid. Can anyone explain why phylolm might be giving such a different result? I'm not sure which method I should be using.

Read →

Cross Validated 2022-09-28 16:20 UTC Score 18.0 AI-113-20220928-social-media-7674f430

Fixed effect model: different estimation approaches with R - how to demean variables - unbalanced panel

I want to use R to estimate a fixed effects model using different estimation approaches. Note that I am using an unbalanced panel . The easiest way to do this is using the function lm . Example: # load packages and create data library(dplyr) set.seed(123) x % group_by(id) %>% mutate(firm = 1:n()) %>% pull(firm) id.eff % group_by(id) %>% summarise(firm = max(firm)) %>% filter(firm == 1) %>% pull(id) db = db[-which(db$id %in% rm), ] # Run regression test A more efficient approach is demeaning the variables included into the model specification. In this way, one can exclude the fixed effects from the model. Of course, point estimates will be correct, while standard errors will be not (because we are not accounting for the degrees of freedom used in the demeaning). # demean data dbm % group_by(id) %>% mutate(y = y - mean(y), x = x - mean(x)) %>% ungroup() # run regression test2 $coefficients[2,1] > 0.9753364 summary(test2)$ coefficients[2,1] > 0.9753364 Another way to do this is to demean the variables and add their grand average # create data n = length(unique(db $id)) dbh % mutate(yh = y + (sum(db$ y)/n), xh = x + (sum(db $x)/n)) # run regression test3 coefficients[2,1] > 0.9753364 summary(test2) $coefficients[2,1] > 0.9753364 summary(test3)$ coefficients[2,1] > 0.9753364 As one can see, the three approaches report the same point estimates (again, standard errors will be different instead). When I include an additional set of fixed effects in the model specification, the three…

RAG

Read →

Cross Validated 2022-09-24 18:15 UTC Score 12.0 AI-113-20220924-social-media-761b3966 Full article

What is the information plane theorem for an autoencoder neural network?

Slide 8 (about 19 minutes into the video) of the Stanford Seminar - Information Theory of Deep Learning, Naftali Tishby has the following (rather informally stated) theorem. Theorem (Information Plane) For large typical $\mathbf{X}$ , the sample complexity of a DNN is completely determined by the encoder mutual information $\mathbf{I(X;T)}$ , of the last hidden layer; the accuracy (generalization error) is determined by the decoder information, $\mathbf{I(T;Y)}$ , of the last hidden layers. I am having difficulty following what is meant by "the sample complexity is completely determined". What is the precise statement of this theorem?

Read →

Cross Validated 2022-09-24 17:55 UTC Score 18.0 AI-113-20220924-social-media-210b72c4

Creating synthetic data for time series, Hidden Markov Model

Suppose that I have a task of classifying a time series. I decide to use Hidden Markov Model $\lambda(A, B, \pi)$ , where $A$ is a transition matrix, $B$ is an emission probability, $\pi$ is an initial distribution. An observed stochastic process looks as below and there are two unobserved states: State 1 and State 2, also shown on the plot. Let's say, that I would like to fit HMM and test how it performs in terms of states recognition. The best thing I can probably do is to split my data into the train and test set but in a specific manner: the last $n$ observations make up my test set (the part to the right of the dotted green line), as I cannot use crossvalidation due to the temporal structure of the data. But there is a problem: to reliably assess the performance (measured as i.e. accuracy), I need at least a few transitions between the states that this data doesn't provide. Question: is there any method that allows creating synthetic data for this kind of data? One thing I was considering was to reverse the whole training set, add some gaussian noise and tack on the end of the original training set.

Read →

Cross Validated 2022-09-23 12:29 UTC Score 9.0 AI-113-20220923-social-media-5397e0ae

Law of Iterated Expectation for a Probability

In my understanding of a derivation, the following statement seems to be used: $$Pr[V\leq X]=E[Pr[V\leq X|X]]$$ where both $X$ and $V$ are random variables. Is this a kind of the law of iterated expectation?

Read →

AI Stack Exchange 2022-09-15 08:23 UTC Score 15.0 AI-110-20220915-social-media-47e156dd Full article

Why Phasic Policy Gradient (PPG) can update value function in auxiliary phase?

My questions is that how could we train the value network (separated from shared network) by using data from previous policies, which varies a lot since we collect data from different policies with many training phases done to start a auxiliary phase, especially returns are calculated by those policies. Wouldn't it hurt the stability of fitting value function?

Read →

AI Stack Exchange 2022-09-12 13:21 UTC Score 20.0 AI-110-20220912-social-media-355bd9b7

How is the noise in the forward process in Denoising Diffusion Probabilistic Models computed?

The inputs are decayed towards the origin using this formula within Denoising Diffusion Probabilistic Models (DDPMs): $$q\left(\mathbf{x}_{1: T} \mid \mathbf{x}_0\right):=\prod_{t=1}^T q\left(\mathbf{x}_t \mid \mathbf{x}_{t-1}\right), \quad q\left(\mathbf{x}_t \mid \mathbf{x}_{t-1}\right):=\mathcal{N}\left(\mathbf{x}_t ; \sqrt{1-\beta_t} \mathbf{x}_{t-1}, \beta_t \mathbf{I}\right)$$ I however do not understand how the origin is determined. How is the noise in the forward process or diffusion process computed? In the original DDPM paper it is only stated that: the forward process variances $β_t$ can be learned by reparameterization [33] or held constant as hyperparameters.

AI Chips & Hardware AI Research & Papers

Read →

Cross Validated 2022-09-11 22:30 UTC Score 15.0 AI-113-20220911-social-media-0e2e79df

SARIMAX prediction: including 'deterministic polynomial trend'

I'm wanting to use Statsmodel's SARIMAX model to predict sharemarket values into the future. This is part of a larger effort towards helping strictly non-profits and social-enterprise organisations. Question : I'm having an issue (maybe conceptual) where when I introduce a 'deterministic polynomial trend', using Statsmodels' implementation, I can't see any effects on the subsequent prediction (I would have thought the prediction would shift upwards if I introduced a larger constant, for example). Am I missing something fundemental? Typical code is below, but without source data (I can provide data if needed): # SARIMA forecasting model trendCoeffsList = [15000, 2.069764822521753, 0.0012385250036528476] def forecast_sarima(data, order, sOrder): try: model = SARIMAX(data, trend=trendCoeffsList, order=order, seasonal_order=sOrder, enforce_stationarity=False, enforce_invertibility=False) print(model.trend) # coefficients exist within model model_fit = model.fit(disp=False) start = 1 end = len(dataWhole) - dataRange predictData = model_fit.predict(start, end) except: predictData = False return predictData

Read →

Cross Validated 2022-09-11 06:06 UTC Score 9.0 AI-113-20220911-social-media-ae96e691

What does Cayley's hyperdeterminant of a 2x2x2 mixed-product moment tensor tell us about how two variables are related?

Suppose we have a collection of random variables $S = \{ X_0, X_1 \}$ encoded into the $2 \times 2 \times 2$ tensor $$\mathcal{C}[i, j, k] = \mathbb{E}[X_i X_j X_k]$$ where $X_i, X_j, X_k \in S$ and $i,j,k \in \{ 0,1 \}$ . Cayley's hyperdeterminant for $\mathcal{C}$ can be (verbosely) expanded to: \begin{align} \det (\mathcal{C}) &= (\mathbb{E}[X_0^3]^2\mathbb{E}[X_1^3]^2+ 3\mathbb{E}[X_0^2X_1]^2 \mathbb{E}[X_0X_1^2]^2) \\ &-2(3\mathbb{E}[X_0^3] \mathbb{E}[X_0^2X_1]\mathbb{E}[X_0X_1^2]\mathbb{E}[X_1^3] + 3\mathbb{E}[X_0^2X_1]^2\mathbb{E}[X_0X_1^2]^2) \\ &+4(\mathbb{E}[X_0^3]\mathbb{E}[X_0X_1^2]^3 + \mathbb{E}[X_0^2X_1]^3 \mathbb{E}[X_1^3]) \end{align} What does Cayley's hyperdeterminant of a 2x2x2 mixed-product moment tensor tell us about how two $X_0$ and $X_1$ are related?

Read →

AI Stack Exchange 2022-09-10 11:32 UTC Score 19.0 AI-110-20220910-social-media-2f29872e Full article

keras model accuracy not improving

I am trying to do multi class(16) classification, however no matter what parameters or number of layers I use my accuracy is not improving, its in 30s the max I got was 43. I have tried early stopping to red overfilling but my testing accuracy is still low. I have 750 images in training and 350 in testing. I am also getting high traning accuracy vs low validation accuracy. features_train=features_train/255 features_test=features_test/255 cnn = models.Sequential([ layers.Conv2D(filters=16, kernel_size=(3, 3), activation='relu', strides=(2, 2), padding="same", input_shape=(224, 224, 3)), layers.MaxPooling2D((2,2)), layers.Dropout(0.25), layers.Conv2D(32 ,(3, 3), activation='relu'), layers.MaxPooling2D((2,2)), layers.Dropout(0.25), layers.Conv2D(64, (3, 3), activation='relu'), layers.MaxPooling2D((2,2)), layers.Dropout(0.25), layers.Conv2D(128, (3, 3), activation='relu'), layers.MaxPooling2D((2,2)), layers.Flatten(), layers.Dense(64,activation='relu'), layers.Dense(16, activation='softmax') ]) cnn.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy']) cnn.fit(features_train,labels_train,epochs=20, batch_size = 4 ,validation_split = 0.25)

AI Chips & Hardware

Read →

Lilian Weng Blog 2022-09-08 17:00 UTC Score 41.0 USR-0112-20220908-ai-specialis-9f1dcded Full article

Some Math behind Neural Tangent Kernel

Neural networks are well known to be over-parameterized and can often easily fit data with near-zero training loss with decent generalization performance on test dataset. Although all these parameters are initialized at random, the optimization process can consistently lead to similarly good outcomes. And this is true even when the number of model parameters exceeds the number of training data points. Neural tangent kernel (NTK) ( Jacot et al. 2018 ) is a kernel to explain the evolution of neural networks during training via gradient descent. It leads to great insights into why neural networks with enough width can consistently converge to a global minimum when trained to minimize an empirical loss. In the post, we will do a deep dive into the motivation and definition of NTK, as well as the proof of a deterministic convergence at different initializations of neural networks with infinite width by characterizing NTK in such a setting.

AI Research & Papers

Read →

Cross Validated 2022-09-07 10:05 UTC Score 12.0 AI-113-20220907-social-media-3870d781

How error derivative becomes zero in gradient descent

Previous questions this & this does not answer my question import matplotlib.pyplot as plt inputs = [(0.0000, 0.0000), (0.1600, 0.1556), (0.2400, 0.3543), (0.2800, 0.3709)] targets = [230, 555, 815, 860] weights = [0.1, 0.2] b = 0.3 learning_rate = 0.1 epochs = 4 # prediction def predict(inputs): return sum([(w * i) for w, i in zip(weights, inputs)]) + b # train the network for epoch in range(epochs): # Feed forward--------- pred = [predict(inp) for inp in inputs] print("Pred:", pred) # Back propagation------ # error derivative errors_d = [(p - t) for p, t in zip(pred, targets)] # error partial derivative weight_d = [[(e * i) for i in (inp)] for e, inp in zip(errors_d, inputs)] bias_d = [(e * 1) for e in errors_d] weight_d_T = list(zip(*weight_d)) # Update weights and bias for j in range(len(weights)): weights[j] -= learning_rate * (sum(weight_d_T[j]) / len(weight_d)) b = b - (learning_rate * (sum(bias_d) / len(bias_d))) From theory, In order to minimize error, we need to take the derivative with respect to the weights and bias. In the above code, I did partial derivation of the error function. And use it with learning rate and update weights and bias. After doing some tests I figured out that if I write the weight updating equation as weights[j] -= learning_rate * (sum(weight_d_T[j]) / len(weight_d)) it will move towards down of the slope. If I write the weight updating equation as weights[j] += learning_rate * (sum(weight_d_T[j]) / len(weight_d)) I mean add partial derivat…

AI Ethics AI Chips & Hardware

Read →

AI Stack Exchange 2022-09-05 10:18 UTC Score 26.0 AI-110-20220905-social-media-81334c29

Multi-Variate Time-Series forecasting with XGBoost

I have trained an XGBoost model on a time-series dataset for predicting a value. The time series has 5 features and one label (the target value). The trained model works fine on both training and testing data, so far so good. As I said, this dataset has some features that I have used for training the XGBoost model (i.e. a multi-variate dataset). The problem is that currently, I have values of these 5 features in my current dataset, so I can train the model with, and do the testing as well. But, I do not know these features values in future. My question is, how can I predict the target value for future (Ex. next year) When I don't know the values of features in future to feed them into the trained model to do the prediction.

AI Research & Papers

Read →

Cross Validated 2022-09-03 12:54 UTC Score 9.0 AI-113-20220903-social-media-04e8e5ff

Are discrete mixtures Gauss quadrature-like integral approximations?

I noticed that the formula for Gauss (or Newton-Cotes) quadrature looks very similar to the formula for the PDF of a general mixture distribution. Let $p_{comp}(x)$ be the PDF of a compound distribution given by the integral: $$ p_{comp}(x)=\int_{\Theta} p_{param}(\theta)p(x, \theta)d\theta $$ Here we're essentially integrating $p(x, \theta)$ w.r.t. the parameter $\theta$ using the weighting function $p_{param}(\theta)$ which happens to be the PDF of the random parameter $\theta$ . Apply numerical integration like Gauss quadrature: $$ p_{comp}(x) \approx \sum_{k=1}^K w_k p(x,\theta_k) $$ This already looks exactly like the PDF of a mixture, but we need to restrict the weights $w_k$ first: Since numerical integration formulas must integrate polynomials up to some degree exactly, the first condition is that the "mixture" formula must integrate $\theta^0=1$ with weight $p_{param}(\theta)$ exactly: $$\int_{\Theta} p_{param}(\theta)\times 1 d\theta = 1 = \sum_{k=1}^K w_k \times 1$$ The integral equals to one because the weight function $p_{param}(\theta)$ is a PDF. Thus, the weights must sum to one. It's known that all weights $w_k$ of a Gaussian quadrature formula must be positive: $w_k > 0 \quad\forall k$ . Even if we used Newton-Cotes, negative weights seem to be frowned upon since they introduce numerical instabilities. Hence, the weights $w_k$ must be a discrete probability distribution (a probability mass function), mirroring the fact that $p_{param}(\theta)$ is also a prob…

Read →

Cross Validated 2022-09-01 17:11 UTC Score 26.0 AI-113-20220901-social-media-4b8e13f0 Full article

Understanding train vs validation loss chart

I am training an LSTM to a univariate time series and I have some questions about how to evaluate the train vs validations loss charts and which number of epochs to use in the model. To give more context about my data. It is a monthly univariate time series and the LSTM wants to predict the next 12 data points. The data is in sliding window format with 12 inputs and 12 outputs. A summary of the model is below. In both charts I see that the error in the validation dataset is smaller than the error in the training set. It means that I cannot generalize well so I am underfitting, right? The training and validation loss seems to converge around 40 epochs for the MAE loss and for the MSE. Should I use MAE as loss? As far as I know, MAE and MSE are the error metrics generally used for time series. Which number of epochs should I use for this model? #DEFINE THE MODEL lstm_model % layer_lstm(units = 12, #24, # size of the layer batch_input_shape = c(1, 12, 1), # batch size, timesteps, features return_sequences = TRUE, stateful = TRUE, name = "LSTM") %>% time_distributed(keras::layer_dense(units = 1), name = "Output") #COMPILE lstm_model %>% compile(loss = 'mae', optimizer = optimizer_adam(lr = 0.001, decay = 1e-6), metrics = 'mse') summary(lstm_model) #FIT THE MODEL validation_split = 0.25 train_history = lstm_model %>% fit( x = x_train_arr, y = y_train_arr, batch_size = 1, epochs = 100, verbose = 1, validation_split = validation_split, shuffle = FALSE )

AI Chips & Hardware AI Funding & Startups AI Research & Papers

Read →

AI Stack Exchange 2022-09-01 12:47 UTC Score 18.0 AI-110-20220901-social-media-edcf577b

How to Train a Decoder for Pre-trained BERT Transformer-Encoder?

Context: I am currently working on an encoder-decoder sequence to sequence model that uses a sequence of word embeddings as input and output, and then reduces the dimensionality of the word embeddings. The word embeddings are created using pre-trained models. I want to be able to decode the word embeddings returned by the decoder of the Sequence to Sequence model back to natural language. Question: How can I train a Decoder that works with the sequence of word embeddings and the original sentence for this task? See below for the code that generates the word embeddings: from typing import List import numpy as np import torch from transformers.tokenization_utils_base import BatchEncoding from transformers import BertTokenizerFast, BertModel TOKENIZER = BertTokenizerFast.from_pretrained('bert-base-uncased') MODEL = BertModel.from_pretrained('bert-base-uncased') def get_word_indices(sentence: str, separator=" ") -> List: sent = sentence.split(sep=separator) return list(range(len(sent))) def encode_sentence(sentence: str) -> BatchEncoding: encoded_sentence = TOKENIZER(sentence) return encoded_sentence def get_hidden_states(encoded: BatchEncoding, layers: list = [-1, -2, -3, -4]) -> torch.Tensor: with torch.no_grad(): output = MODEL(**encoded) hidden_states = output.hidden_states output = torch.stack([hidden_states[i] for i in layers]).sum(0).squeeze() return output def get_token_ids(word_index: int, encoded: BatchEncoding): token_ids = np.where(np.array(encoded.word_ids()) == wor…

AI Chips & Hardware

Read →

Cross Validated 2022-08-26 12:27 UTC Score 23.0 AI-113-20220826-social-media-c5726999

Train/Test splitting for seasonal adjustment

At work, I just started dealing with seasonal adjustment of monthly time series on credit data, so being new to the topic it is quite possible that my question is pretty trivial. From what I've read so far, procedures like TRAMO-SEATS and X13-ARIMA-SEATS never mention the need to split the data into train/validation/test sets, which is instead a common practice in the context of machine learnign algorithms. Indeed it seems to me that the seasonal adjustment algorithms use the entire time series and, every time a new observation comes in, it gets added to the input dataset as well, often leading to significant change in the seasonal adjustment factor and in the parameters of the underlying ARIMA model (automatically estimated by the seasonal adjustament procedures). I am a bit perplexed from a methodological point of view. Is it correct that every time new data is added, the seasonally adjusted series changes in its entirety? How are these variations conceptually justified? Wouldn't it be more correct to use the train/test split to define a single model so that adding a new observation does not lead to changes in the past of the seasonally adjusted series?

AI Chips & Hardware AI Funding & Startups AI Research & Papers

Read →

Cross Validated 2022-08-13 14:48 UTC Score 12.0 AI-113-20220813-social-media-46b1f80b

Is it possible to do a contrast in a 2x2 (or more) contingency table or am I completely missing the point?

I have the following 2x2 contingency table of observed frequencies: Contingency Table ------------------------------------------ Categories -------------- Cat 1 Cat 2 Total ------------------------------------------ Group 1 135 16 151 Participants Group 2 241 40 281 Total 376 56 432 ----------------------------------------- The Chi-squared test result is the following: χ2 Tests -------------------------------- Value df p -------------------------------- χ2 1.153 1 0.283 N 432 -------------------------------- As we can see, the p-value is > .05, therefore there is no statistically significant difference between Group and Type. Therefore, if I'm not mistaken, Group and Type seem not to be related. However, there might be a statistically significant difference between participants in Group 1 and Group 2 separately in Cat 1 and in Cat 2 (e.g., 135 vs. 241 on the one hand, and between 16 and 40 on the other hand). I am also interested to know if the difference is statistically significant in Group 1 between Cat 1 and Cat 2, and in Group 2 between Cat 1 and Cat 2 (e.g., 135 vs. 16 and 241 vs. 40). My questions: How do I test the differences within each variable? Is there an easy (relatively) way to do it? Would those be similar to ANOVA's contrasts or am I completely missing the point? Technical details: R 4.2.1 GUI 1.79 High Sierra build RStudio 2022.07.1 Build 554 for Intel macOS macOS Monterey Version 12.5 R/RStudio code (inspired by an example from ETHZ in Switzerland): tab

Read →

Data Science Stack Exchange 2022-08-11 22:26 UTC Score 9.0 AI-111-20220811-social-media-e9d3efaf Full article

How to solve Nonlinear least squares problem?

Initial idea is to use euclidean distances. But I do not understand how should I solve this task.

Read →

Cross Validated 2022-08-11 17:43 UTC Score 18.0 AI-113-20220811-social-media-3a5b3767

How to analyse results of classification for time series + sliding windows

Here is my context: I have a time series composed of only 1 features. I want to be able to classify between two classes. To get more information out of these data, I am using a sliding time windows. For example, if my time series contains 1000 observations with, I use a sliding window of 20 observations, and I iterate through all 1000 observations. I associate iteration t of my time series with the first value of the window (it t=1: window have t in [1,21]; if t=150, window have t in [150,170]) For each window of 20 samples, I extract additional features (let's say mean and variance for simplicity). Doing so, I have a new data set of size 1000*2. Then, I train a classifier on this data set. I do not use time as a feature. In order to evaluate the model, I have a test set (a time series of 200 observations). I do the same process with the time window to get a data set of 200*2. From there, I compare the true class from predicted class and create my confusion matrix. To summarise quickly the whole process: Transform a time series x of N observations and f features into a training set X of N observations and F features with sliding time window Train classifier with X Transform another time series y into a test set Y (same process as 1) ) Predict class of Y with trained model Create confusion matrix Now, here is my problem: When I get my confusion matrix, I get a lot of false positive. The reason is that if the class are extremely different in the time series, the statistical va…

Read →

AI Stack Exchange 2022-08-11 16:10 UTC Score 17.0 AI-110-20220811-social-media-2d005b39

Master theorem about polynomial classifiers?

Does anyone know if there is a theorem or counterexample establishing whether or not for any given binary classification task in some finite (possibly large) dimensional vector space of attributes, that there exists a polynomial classifier that can form a hyperplane sorting all the positive from negatively labelled data points? To clarify, I know that if a dataset is linearly separable, then we can find such a linear classifier. But my question is more general and asks if without knowing beforehand whether a dataset is separable at all, can we know ahead of time if there exists a polynomial classifier for any n-dimensional vector space of data points?

AI Research & Papers

Read →

Data Science Stack Exchange 2022-08-11 00:20 UTC Score 29.0 AI-111-20220811-social-media-485f9224 Full article

Difference Between Attention and Fully Connected Layers in Deep Learning

There have been several papers in the last few years on the so-called "Attention" mechanism in deep learning (e.g. 1 2 ). The concept seems to be that we want the neural network to focus on or pay more attention to certain features, and has demonstrated some empirical success in NLP and related sequential models. When I look at some code examples such as this one , adding an Attention layer intuitively makes sense and seems to improve performance of the LSTM model. However it looks very much like a regular fully-connected layer. In that link (and with some slight change of notation), the Attention layer outputs $$ c(x) = \tanh(\mathbf{W}x + \mathbf{b} ) $$ $$ \beta(x) = \frac{e^{c(x_j)}}{\sum_{j} e^{c(x_j)}} $$ $$ f_{Attention}(x) = x\beta $$ where $W,b$ are weights/biases, $x$ is layer input, and $f(.)$ is the layer output. In contrast, a regular fully connected layer: $$ f_{Dense} = \sigma(\mathbf{W}x + \mathbf{b}) $$ for some activation function $\sigma(.)$ . My interpretation of the Attention implementation above is that it is pretty much the same thing as a standard fully connected layer, but with a $\tanh$ activation (why?), followed by a $\text{softmax}$ (okay, so that the "attention weights" $\beta$ sum to 1), followed by a linear dot product. How does this architecture allow the model to have "attention"? I do not see how it is fundamentally different or more expressive from just adding a standard fully-connected layer. Am I misunderstanding something here? Edit/My…

AI Ethics AI Chips & Hardware AI Research & Papers NLP

Read →

Cross Validated 2022-08-09 16:42 UTC Score 20.0 AI-113-20220809-social-media-3387c8c0

How to properly impute values on the test set using imputer (missForest)

I'm trying to impute some missing values on my dataset $X$ . So first I shuffle and split data to obatin the train set X_train and the test set X_test . X_train, X_test, y_train, y_test = train_test_split(X, y, stratify = y, shuffle = True, test_size = 0.25) Then I am imputing values using missForest algorithm, but only in the train set imputer = MissForest(max_iter = 10, verbose = 0) X_train_imp = imputer.fit_transform(X_train) Now, what's the proper way of imputing the test set? Use transform() function as follows X_test_imp = imputer.transform(X_test) OR Combine imputed train set and test set and only then impute values in the test set X_combined = pd.concat([X_train_imp, Xtest]) imputer_conc = MissForest(max_iter = 10, verbose = 0) X_conc_imp = imputer_conc.fit_transform(X_combined) X_conc_imp = pd.DataFrame(Xconc_imp) Xtrain_imp = Xconc_imp.loc[:X_train_imp.shape[0] - 1] Xtest_imp = Xconc_imp.loc[X_train_imp.shape[0]:] The first approach seems to allow no data leakage, however the second one takes advantage of the available test set (I simply have it) - but I'm curious to what degree it could cause data leakage.

AI Research & Papers

Read →

AI Stack Exchange 2022-08-08 15:04 UTC Score 18.0 AI-110-20220808-social-media-979f447f

Choice of LSTM for price prediction

I have a dataset with features (f) for different stocks (S) and want to infer for price using an LSTM model. Here is my df: year S1_price S1_f1 S1_f2 S2_price S2_f1 S2_f2 Sn_price Sn_f1 Sn_f2 2010 100 0.1 0.12 200 0.2 0.22 300 0.3 0.32 2011 105 0.4 0.42 205 0.5 0.52 305 0.6 0.62 2012 110 0.7 0.72 210 0.8 0.82 310 0.9 0.92 n and so on... (example values). I would like to predict the prices of every stock by using the features as inputs looking 1yr back into the past. Example for Stock 1 (predict 2012): [[0.1 0.12] [0.4 0.42]] 110 However, I want to to that for all stocks, so I am not sure which type of LSTM to use. Your help would be very much appreciated!

AI Chips & Hardware AI Research & Papers

Read →

Oxford Machine Learning Research Group 2022-08-08 12:42 UTC Score 22.0 USR-0027-20220808-research-aca-deeb56e4 Full article

publications

See relevant faculty pages for publications Stephen Roberts Michael Osborne Xiaowen Dong Jan-Peter Calliess Stefan Zohren

Read →

Oxford Machine Learning Research Group 2022-08-08 12:41 UTC Score 33.0 USR-0027-20220808-research-aca-ddbafb4f Full article

members - [Faculty]

The Machine Learning Research Group comprises several groupings of Faculty, Postdocs and Students. Each member may have multiple local affiliations to sub-groups in the MLRG. Faculty Stephen Roberts Miake Osborne Xiaowen Dong Jan-Peter Calliess Stefan Zohren ---------- Principal Researchers Dr Vu Nguyen Dr Waqas Rafique Dr Ivan Kiskin

AI Research & Papers

Read →

Cross Validated 2022-08-08 12:37 UTC Score 9.0 AI-113-20220808-social-media-17e9b64d

How to derive the three matrices of SVD from eigenvalue decomposition in Kernel PCA?

Kernel PCA is usually done via eigenvalue decomposition of the Kernel Matrix $\mathbf{K}$ and standard PCA via SVD of the input $\mathbf{X}$ . In standard PCA as far as I know we can derive $\mathbf{S}$ and $\mathbf{U}$ via two eigenvalue decompositions, of the Gram and Covariance/Correlation matrices: $$ \begin{array}{c} X=U\Sigma V^T\\ C=\dfrac{X^TX}{N-1}\\ G=\dfrac{XX^T}{N-1}\\ C=VE_CV^T\\ G=UE_GU^T\\ S=\sqrt{E_C(N-1)}\\ K=U_KE_KU^T\\ ?=VE_?V^T \end{array} $$ But how does one get $\mathbf{V}$ in the case of a kernel? All posts I've ever read only discuss $\mathbf{U}$ Note: I've read that $\mathbf{XV}=\mathbf{U\Sigma}$ , however this relationship doesn't seem to hold for numpy.linalg.svd or scipy.linalg.svd

AI Chips & Hardware

Read →

Cross Validated 2022-08-06 10:38 UTC Score 12.0 AI-113-20220806-social-media-bccee83a

Using Confidence Intervals for finding confidence of forecasting model

My question is quite a general one. Can I use the width of the confidence interval (let's say a 95% confidence interval) to find out how confident my model is while doing time series forecasting? I am thinking that as we go on forecasting in the future, the width of the confidence interval increases, and maybe after a threshold we can say that model is not confident enough to do forecasting. Is this a correct line of thinking? Otherwise is there any other way I can use to determine how confident my model is and hence how much further in the future I can forecast?

Read →

Cross Validated 2022-08-06 09:18 UTC Score 12.0 AI-113-20220806-social-media-208f1c97

Is the mean of the left-truncated binomial distribution convex in p?

The expectation of the binomial distribution of successes in $G$ trials, left-truncated at $R$ , with success probability $p$ , is $$ E[X|p] = \frac{\sum_{l=R}^Gl\phi(l)}{\sum_{l=R}^G\phi(l)} $$ where $$ \phi(l) = \binom{G}{l}p^l(1-p)^{G-l}. $$ Is this convex in $p$ ? It looks as if it is. Update In work so far, I've taken the first derivative as $$ \frac{d}{dp}E[X|p] = \frac{\sum_{l=R}^{G}\sum_{m=l+1}^{G}(l-m)^{2}\phi(l)\phi(m)}{p(1-p)(\sum_{l=R}^G\phi(l))^2}. $$ This used the fact (I hope!) that $$ \frac{d}{dp} \phi(l) = \frac{l - Gp}{p(1-p)} \phi(l) $$ and simplified a double sum $\sum_{l=R}^G \sum_{m=R}^G...$ by merging pairs of terms to produce $\sum_{l=R}^G \sum_{m=l+1}^G...$ .

Read →

Cross Validated 2022-08-05 07:46 UTC Score 23.0 AI-113-20220805-social-media-f1153c6e

How can I calculate the odds of exposure for >1 outcome group, combined, using the rms::lrm function. Predict? Contrast?

I have been using a logistic model from rms::lrm to estimate the odds ratio of a binary exposure on an outcome, using splines [time] as Frank Harrell recommended. I am using Predict and Contrast to estimate time-varying coefficients which can be further manipulated. An example using the inbuilt ToothGrowth dataset: #Install and load packages library("rms") library("dplyr") #develop a binary predictor variable with values A and B data(ToothGrowth) ToothGrowth % mutate(dose_binary = case_when( dose >1 ~ "A", TRUE ~ "B")). #run logistic regression with 5 restricted cubic splines dd This works very well, but I would like to estimate the odds (not odds ratio) of exposure in both outcome groups , combined (e.g., the average of p1 and p2); along the time-varying spline function (and I have additional covariates in the final model too). The underlying problem is to estimate the association between therapy A vs therapy B on the probability of a severe outcome: OR (severe) = odds(severe)/odds(control) But I lack data on controls. I wish to estimate the odds in controls, borrowing probabilities I DO have from different but related sources (bold indicates data I have access to): A) OR(s/m) = odds(severe)/odds(mild) and B) OR(severe+mild) = odds(severe + mild)/odds(control) My logic was to calculate the joint probabilities in both the “severe” and “mild” groups (from A) and estimate the odds in the control group that I lack by: Odds(control) = odds(severe + mild, from A)/OR(severe + mild…

AI Research & Papers RAG

Read →

Cross Validated 2022-08-03 11:58 UTC Score 26.0 AI-113-20220803-social-media-d88ea767

Model suggestion for zero-bounded dependent variable

I have a repeated measures dataset (8 measurements) with three experimental groups (Randomly generated). The dependent variable (Y) is on a continuous scale bounded inferiorly at 0. I am trying to model the effect of the different groups and time on Y. What would be the appropriate model here? - repeated ANOVA is compromised by the non-normal distribution of Y (see histogram - I tried adding a minuscule amount and logging, but still really to no help) Mixed linear models are not bounded inferiorly at 0. Perhaps one of the more advanced non-linear mixed models? Area under the curves for all timepoints stratified by group? Bayesian approaches? Raw Y Logged Y Histogram of logged Y

AI Research & Papers

Read →

AI Stack Exchange 2022-08-01 09:13 UTC Score 31.0 AI-110-20220801-social-media-7aff1174

Datasets input at model.fit produce unexpected results of training loss vs validation loss

Im trying to train a neural network (VAE) using tensorflow and Im getting different results based on the type of input in the model.fit. When I input arrays I get normal difference between the validation loss and the total loss. When I input a dataset based on the same input I get a normal total loss and a really small validation loss. I havent changed the model. The only things that changes is the input format. The code for when I input an array. train slices is (2627,138,138,1) and define the batch size in the model.fit train_slices = preprocess_data(CropTumor, file_array[train_dataset]) val_slices = preprocess_data(CropTumor, file_array[val_dataset]) # reset model weights before training VAE.set_weights(initial_weights) # fit model fit_results = VAE.fit(train_slices,train_slices, epochs=1000, validation_data=(val_slices,val_slices), callbacks=[early_stopping_kfold, tensorboard_callback], batch_size=batch_sz, verbose=2 ) The output Epoch 1/1000 2022-08-01 11:56:35.683852: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8401 2022-08-01 11:56:36.371780: I tensorflow/core/platform/default/subprocess.cc:304] Start cannot spawn child process: No such file or directory 2022-08-01 11:56:36.461054: I tensorflow/stream_executor/cuda/cuda_blas.cc:1786] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once. 672/672 - 7s - loss: 537.2896 - val_loss: 213.7070 - 7s/epoch - 11ms/step Epoch 2/1000 672/672 - 5s - loss: 248.5211 - v…

NVIDIA AI Chips & Hardware AI Research & Papers

Read →

Latest AI/ML News