Visualizing Neural Networks with the Grand Tour
By focusing on linear dimensionality reduction, we show how to visualize many dynamic phenomena in neural networks.
Latest AI/ML News
19204 matching items
Does negative sampling sacrifice performance?
I am implementing a Seq2Seq model. Each step of the decoder has |N| outputs (the number of unique words). Since |N| is huge, I am trying to speed up the training by negative sampling (n_sample=100). I can see the training time for one epoch is reduced by 25%. However, the performance (crossentropy loss) is not as good as full prediction. Is it normal?
Covariance of conditional poisson random variable sequence
Suppose $X_0,X_1,\cdots$ are iid $Poisson(\theta)$ r.v. Define $Y_k = X_k I_{\{ X_{k-1} = 0 \}}$ for $k=1,2,3,\cdots$ Find the limit of $Var(\sqrt{n}\overline{Y_n})$ and asymptotic distribution of $\sqrt{n}\left(\bar{Y}_{n}-\mathrm{E} \bar{Y}_{n}\right)$ . It's not difficult to find $E(Y_k) = E(X_k)P(X_{k-1}=0) = \theta \exp(-\theta)$ . But when it comes to variance, I know that $Y_k$ is independent to $Y_{k+i}$ for $i>1$ . However, I dont know how to calculate the covariance of $Y_k$ and $Y_{k+1}$ . Besides, since $Y_k$ are not i.i.d. random variables, how can I use CLT to get the asymptotic distribution?
Extract confidence intervals confint() for random estimates of lmer models
I want to test the significance of the random slope in my model, i.e. if there is significant individual difference in change. I am using lmer() and confint() in R The model is: model time: 4 time points, values 1,2,3,4. n: continuous dependent variable for neuroticism summary(model) Linear mixed model fit by REML. t-tests use Satterthwaite's method [ lmerModLmerTest] Formula: n ~ time + (1 + time | id) Data: long REML criterion at convergence: -421 Scaled residuals: Min 1Q Median 3Q Max -3.6702 -0.4900 -0.0058 0.4802 3.4323 Random effects: Groups Name Variance Std.Dev. Corr id (Intercept) 0.14163958 0.376350 time 0.00008384 0.009157 0.39 Residual 0.01127142 0.106167 Number of obs: 842, groups: id, 250 Fixed effects: Estimate Std. Error df t value Pr(>|t|) (Intercept) 2.185644 0.025323 248.552766 86.312 When I extract the confidence intervals, this is the output: confint(linear.mod.n) 2.5 % 97.5 % .sig01 0.340460916 0.415590685 .sig02 -1.000000000 1.000000000 .sig03 0.000000000 0.026388745 .sigma 0.098924884 0.112977148 (Intercept) 2.135917316 2.235365845 time -0.009836903 0.003374645 I am trying to figure out which confidence intervals are presented here. .sig01 appears to match the random intercept standard deviations, .sig03 for random slope time , .sigma for random residuals, and (Intercept) and time for the fixed effects. Is this correct? If so, what is .sig02 providing the confidence interval for? Thank you all in advance!
Does p-value ever depend on the alternative?
Our tag definition of the $p$ -value says In frequentist hypothesis testing, the $p$ -value is the probability of a result as extreme (or more) than the observed result, under the assumption that the null hypothesis is true. I guess this is how Fisher thought about it, and I am comfortable with it. However, I think I have seen $p$ -value being calculated differently in one-sided hypothesis testing. Outcomes that are not in the direction of the alternative do not get considered extreme. E.g. assume $X\sim N(\mu,\sigma^2)$ and test $$ H_0\colon\mu=0 $$ against $$ H_1\colon\mu\neq 0. $$ Using the empirical mean $\bar x$ as an estimator of $\mu$ , the $p$ -value is calculated exactly as defined above. If $\bar x$ is far from zero (to either side) in terms of the estimated standard deviation $\hat\sigma$ , the $p$ -value is low. Now consider $$ H_1'\colon\mu>0, $$ I have seen $p$ -value calculated as $$ \text{p-value}=1-\text{CDF}(t) $$ where $t:=\frac{\bar x}{\hat\sigma/\sqrt{n}}$ is the $t$ -statistic and $\text{CDF}$ is the cumulative density function of $t$ under $H_0$ . Then $p$ -value is high when $\bar x$ is far to the left of zero, contrary to the case above. $\bar x$ being far to the left of zero is extreme in the perspective of $H_0$ , but in an uninteresting direction from the perspective of $H_1'$ . Questions: Does the p-value actually depend on the alternative hypothesis? Or is $\text{p-value}=1-\text{CDF}(\bar x)$ nonsense? Or are there alternative definitions depen…
Sample size calculation for correlated count data
I am wondering how I can simulate Poisson data that is correlated. Let's say, I collect data at two time points. At both time points, the data is Poisson distributed, at time point 1 with $\lambda=4$ , at time point 2 with $\lambda=3$ and there should be a strong correlation between measurement at time point 1 and 2, i.e. $\rho = 0.8$ . Obviously, for time point 1, I can sample data of size $n$ in R with rpois(n=n,lambda=4) . But how can I sample the data for time point 2 that is correlated to time point 1? Some more details on the specific lab problem: The variable of interest is the number of successful 'catches' of a mouse (it's some kind of coordinative task). We count successes per mouse at time point 1. Then some treatment is applied to the mouse, that is supposed to affect its coordinative ability. We expect a drop in the average number of successes by around 25% in each mouse. Mice that were relatively good at the task before the treatment will probably still be relatively good after the treatment. A statistical comparison between time point 1 and 2 will be performed using the Wilcoxon signed-rank test. We want to know the sample size required to achieve power of 80% (while type I error rate should be 5 %). Update : It turns out, the number of successes is limited to discrete values between 0 and 7. Further, based on previous experiments we have the following probabilities for each number of successes before treatment: $p=\{0.02,0.03,0.09,0.17,0.19,0.30,0.15,0.05\}$…
Expected value $\mathbb{E}[X-Y \mid X>Y, Y< c]$ for independent variables $X,Y$?
I am looking for the expected value $\mathbb{E}[X-Y \mid X>Y, Y for independent variables $X \sim U(x_1, x_2),Y \sim U(y_1,y_2)$ , $c\in [y_1, y_2]$ ? Is the following formula correct? \begin{align} & \mathbb{E}[X-Y \mid X>Y, Y Thanks!
Mixed model with random slope and Intercept syntax?
I posted a similar question but I am still struggling to make sure my syntax is accurate. I used the Cosinor package in R to determine the amplitude value of each participant, based on heart rate per day across a 24 hour period. Data was collected for every day for 6 weeks per subject. However, some participants have less days than other due to data loss - range is 1-42 days with average of 16 my data sample: structure(list(User.ID = c(21256L, 21256L, 21256L, 21256L, 21256L, 21256L, 21256L, 21258L, 21258L, 21258L, 21258L, 21258L, 21258L, 21259L), HR = c(93.12272727, 95.60333333, 98.29333333, 118.9666667, 84.46666667, 78.36666667, 95.03333333, 97.80333333, 80.03333333, 94.03333333, 97.88, 82.86, 85.86333333, 81.59666667), Weekday = structure(c(7L, 5L, 1L, 3L, 4L, 2L, 1L, 5L, 1L, 3L, 4L, 2L, 5L, 7L), .Label = c("Fri", "Mon", "Sat", "Sun", "Thu", "Tue", "Wed"), class = "factor"), YearDay = c(45L, 46L, 47L, 48L, 49L, 50L, 54L, 46L, 47L, 48L, 49L, 50L, 53L, 45L), Whoop_ID = c(6003L, 6003L, 6003L, 6003L, 6003L, 6003L, 6003L, 6004L, 6004L, 6004L, 6004L, 6004L, 6004L, 6001L), MESOR = c(127.8198669, 82.53790266, 80.196328, 87.94691954, 78.24139419, 85.76274123, 85.24568034, 78.76045577, 78.13040009, 82.96056761, 80.7242717, 81.09227575, 79.80820793, 76.53899579), amp = c(39.62989516, 15.90807964, 17.40106027, 20.93586176, 11.64670832, 17.36869616, 12.79682425, 15.60118873, 18.1036377, 17.95736176, 14.5964711, 12.81719691, 19.65582833, 19.85491777), acr = c(-2.096524329, -0.531214487,…
Partial Least Squares: adding unrelated variables improves fit
I am performing some simulations of partial least squares. In particular, I have 30 observations split into 20 which are for training and 10 which are for testing. I also have 23 independent variables (all unrelated to each other), the first 18 of which are related to the dependent variable while the last 5 are unrelated to the dependent variable. Then I perform two regressions on the training sample: A PLS regression with all 23 independent variables using 18 principal components. An OLS regression with only the 18 variables that are related to the dependent variable. Surprisingly, when I test the two models using the testing sample, the model estimated using 1. consistently produces substantially lower RMSE than the model estimated using 2. It appears that adding variables that are unrelated to the dependent variable improves the model's predictions. The same occurs when I use principal components regression instead of PLS. What is the cause of this? Is this expected or is it likely due to some mistake in my simulations? (I also tried estimating 2. using PLS (with 18 components) in order to confirm that the PLS function in R indeed returns the same result as OLS when the number of independent variables is lower than the number of observations.)
Comment on You’ll want Nexar’s newly released Live Map for your city by Labeling, transforming, and structuring training data sets for machine learning - Blockchain, Artificial Intelligence and Data Education Academy
[…] need for tools that can ease the transition from raw data to derived data (e.g., entities), insights, and even […]
Expectation of $AA^T$
Let $A=(bI-Y)\mathcal{I}(Y\geq b)+(aI-Y)\mathcal{I}(Y\leq a)$ where $a$ and $b$ are scalars, $Y$ is a standard normal random variable, and $\mathcal{I}(\cdot)$ is an indicator function. I would like to find $E(AA^T)$ .
Why the prior distribution can be neglected compared with likelihood function in the posterior distribution when sample is large
Here is the theorem, Bayesian posterior estimation is same as MLE under the large samples and its prove in the book All of Statistics A Concise Course in Statistical Inference page 190 . However I cannot understand why: $f(\theta|X^n)\approx \mathcal{L}_n(\theta)$ when $n\rightarrow\infty.$ $\dfrac{1}{n}\sum-\mathcal{l}''_i(\hat{\theta}_n)\approx\mathbb{E}_{\theta}[-\mathcal{l}''_i(\hat{\theta}_n)].$ I already known this is from the CLT. Can any one give me more detail of prove? I am so confused for the last two steps First, do we regard $\mathcal{l}''_i(\hat{\theta}_n)$ as a constant independent of $\theta,$ then we have $\mathcal{l}''_i(\hat{\theta}_n) = \mathbb{E}_{\theta}[\mathcal{l}''_i(\hat{\theta}_n)]?$ Second, even though, $I(\hat{\theta}_n) = \mathbb{E}_{\theta}[\mathcal{l}''_i(\theta_n)]\Big|_{\theta_n = \hat{\theta}_n}\neq\mathbb{E}_{\theta}[\mathcal{l}''_i(\hat{\theta}_n)].$
Within-subject centering of a repeatedly measured dichotomous variable in a multilevel model?
I'm currently working on a nested data set consisting of 100 subjects which answered several questions at home on five consecutive days (ecological momentary assessment). Among them, they were asked if they adhered to the study protocoll on each given day (no/ yes), leading to 0 (no) or 1 (yes) in the data for each of the five days. The continous outcome variable was also assessed on each day. The reserach question is whether beeing adherent on a given day has an effect on the outcome. Now, to explore the effect of the adherence on the outcome variable, I built a nested multilevel model (days within subjects) predicting the outcome by some covariates, a random intercept (high ICC) and the variable coding for adherence (0 or 1). Normally, I would proceed and disentangle within-subject variations (person mean centered) from between-subject variations (grand mean centering of the person means) for the adherence variable. However, it seems rather odd to me to center a dichotomous variable. On the other hand, I know that it is necessary in order to get a clear picture of the within-subject effect. When not centering, I could enter the 0/1 adherence variable as a factor. However, in this case, it would confound within- and between-subject variations (because subjects do not only differ as compared to themselves but also in their total amount of adherence in comparision to the group, the grand mean). Do you have any advice on whether I should center the 0/1 variable? If yes, how wo…
Prewhitening with seasonal response and non-seasonal independent variable
I'm working to develop a forecasting model for a quarterly seasonal variable (quarterly estimated individual income tax payments) using several candidates for non-seasonal independent variables (quarterly average value of S&P 500, dividend income, interest income, etc.). I understand that I need to prewhiten the dependent and independent variables in order to generate a cross-correlation function to identify the appropriate rational transfer function for the independent variables, but the seasonal dependent variable and candidate non-seasonal independent variables require different differencing operations to be stationary. Using year-over-year differencing for the non-seasonal variables leads to over-differencing, often with AR and/or MA terms near 1.0. Is the only solution to create a synthetic seasonally adjusted series for the dependent variable before prewhitening? Or is there another way to prewhiten variables that require different differencing opterations?
Synthetic Control Weight Selection
I have implemented synthetic controls in two of my dissertation chapters. In a recent seminar, someone asked me a question I had never encountered. A cursory scan of the literature also seemingly does not address this issue. I understand mathematically that the weights are chosen to minimize the weighted norm of the difference between the pre-treatment predictor variables. I also get the fact the weighting matrix is the positive-definite diagonal matrix that minimizes pre-intervention root mean square predicted errors. That said, how are initial weights chosen by synthetic controls? Secondly, how does the command converge to sets of weights that satisfy the regularity conditions? In other words, how do Stata or R arrive at the set in terms of mechanical implementation?
Why are probability problems involving combinations often solved indirectly?
Let's say you have a box with 25 cell phones in it, of which 2 are defective. If a person selects 10 cell phones at random, without replacement, what is the probability that both defective cell phones will be selected? I've only seen problems of this sort solved "indirectly". For example, one would solve for the probability of selecting 8 non-defective cell phones and not for selecting the 2 defective cell phones. Why is this? How do you solve for selecting the 2 defective cell phones directly? If it helps, here is how I've seen these types of problems solved. If 8 out of the 10 cell phones are non-defective , then the remaining 2 cell phones selected must be the defective cell phones. So, you first figure out the total number of ways selecting 10 cell phones at random from 25 cell phones (without replacement) which is 25 choose 10 or $\binom{25}{10} = 3,268,760$ using the binomial coefficient formula. Then, calculate the number of ways to select 8 non-defective cell phones from 23 total non-defective cell phones (25 total cell phones minus 2 defective cell phones) so $\binom{23}{8}=490,314$ . Therefore, the probability of selecting the 8 non-defective cell phones is $490,314 / 3,268,760 = 0.15$ which also equals the probability of selecting 2 defective cell phones. But, I've never seen this type of problem solved for in a "direct" manner.
What is the probability that exactly $k$ tosses are required to get exactly $2$ Heads
I have the following homework problem: What is the probability of that exactly k tosses are required to get exactly 2 Heads I wanted to validate that my approach to solving this is correct and if my answer is on the right track: $$A = P(\text{only 1 H in first $k-1$ tosses}) \cdot P(\text{H in last toss}) $$ $$A=P(\text{only 1 H in first $k-1$ tosses}) \cdot .5 $$ Is this the correct approach? Also below is my answer; is this correct or am I way off? $$ \frac{k-1}{2 ^{k-1}} \cdot .5 $$
How to exclude events with low data (eg. threshold, outliers)
I have this data set and I want to filter only "Event" with a good conversion rate. We can say that good are those that have a higher than average conversion (but maybe you have better ideas). Since the average conversion is 0.8% I will also select those events with too little data (eg Impressions = 1 or impressions = 10). What formula can I use to quickly exclude events with too little data and set a minimum threshold for impressions? I can't use an hard-coded threshold because it has to be different for each account. Impressions/clicks are not normally distributed. Maybe long-tail distribution makes more sense.
Global sensitivity of mean and variance in differential privacy?
Please explain me why global sensitivity of a mean or variance queries will be (b-a)/n and (b-a)^2/n where b is the upper bound of the data and a is the lower bound?
Drawing six cards from a deck of cards
Six cards are drawn from a well-shuffled deck. You are told that at least an ace (A) is among the cards. What is the probability that there is at least a king (K) or at least a queen (Q)? (A deck consists of 52 cards, with 13 values (A; 2; 3; 4; 5; 6; 7; 8; 9; 10; J; Q;K) each having 4 suits } I took cases in the denominator (1 ace, 2 aces, 3 aces...) and tried to take the complement (there are no kings, or there are no queens).
Teaching AI to create visuals with more common sense
Teaching AI to create visuals with more common sense aconner Mon, 07/01/2019 - 10:07 Article July 01 '19 Adam Conner-Simons, MIT CSAIL An MIT/IBM system could help artists and designers make quick tweaks to visuals while also helping researchers identify “fake” images. Today’s smartphones often use artificial intelligence (AI) to help make the photos we take crisper and clearer. But what if these AI tools could be used to create entire scenes from scratch? A team from MIT and IBM has now done exactly that with “GANpaint Studio,” a system that can automatically generate realistic photographic images and edit objects inside them. In addition to helping artists and designers make quick adjustments to visuals, the researchers say the work may help computer scientists identify “fake” images. David Bau, a PhD student at MIT’s Computer Science and Artificial Intelligence Lab (CSAIL), describes the project as one of the first times computer scientists have been able to actually “paint with the neurons” of a neural network — specifically, a popular type of network called a generative adversarial network (GAN). Available online as an interactive demo , GANpaint Studio allows a user to upload an image of their choosing and modify multiple aspects of its appearance, from changing the size of objects to adding completely new items like trees and buildings. Boon for designers Spearheaded by MIT professor Antonio Torralba as part of the MIT-IBM Watson AI Lab he directs, the project has vas…
Is the capacity of a multitask U-Net with two-decoders the same of a standard U-Net with doubled capacity in the decoder?
I implemented a U-Net with an additional decoder (one encoder, then it splits into two decoders). The first decoder predicts the normal segmentation label and the second decoder predicts the distance map of this label. I want to evaluate this model against a baseline U-Net. My question is, is it fair to compare this multitask U-Net to a standard U-Net, or should I double the capacity of the decoder of the standard U-Net, as the multitask U-Net has two decoders and therefore doubled capacity in the decoder part?
Estimating population variance considering both population sample variance AND sampling method variance
I want to estimate the variance of a normally distributed population. I can take N samples and calculate the sample mean and sample variance, which would normally suffice; however, the sampling method itself has inherent variability that is known. Is there a way to "subtract out" the known test method variance the get a better estimate of the population variance? To provide some context, I'm measuring density of an asphalt road after construction. Random locations are selected for measuring density for quality assurance. There are two methods, A and B, for measuring density and they have different known variance. The method variance was found by repeated measurements on known references. If I use Method A, with high variance, to and get N samples of the road density, I'll end up with a sample variance that is higher than if I used the more precise Method B. And in both cases, my variance will still be higher than the true variance of my sample. If I know the actual test method variance, isn't there some way to leverage that information when making an estimate of the sample and population variance?
I counted the distance (in arbitrary units) that participants took to reach a target. Is this best described by a Poisson distribution?
I can add that the variance was more than twice the mean. I looked into over-dispersed Poisson models, but traditionally my field leans heavily on normal models. Is there a way to decide on the type of model? Edit to add more information based on comments: I had two groups of participants find a target. I measured distance travelled (the program I used has an arbitrary measure of distance). I measured each participant over the course of three trials. I predicted a group by trial interaction (I expected one group to improve more than the other groups). I anticipated using a multilevel model using LMER in R: model Here is the distribution of all scores: and the distribution by group: I guess my major concern was that I wasn't sure if it was appropriate to assume a normal distribution. The residual plot showed did not seem 'random' to me, which is what caused my hesitation. I guess while I am this far in, my secondary question is, Can I include the effects of trial as a random slope without including the interaction as a random effect, such as: model Thank you!
Practical Examples: Expectation of a function with respect to a probability
I have encountered the following phrasing while reading Bishop's " Pattern Recognition and Machine Learning ": Although for some applications the posterior distribution over unobserved variables will be of direct interest in itself, for most situations the posterior distribution is required primarily for the purpose of evaluating expectations , for example in order to make predictions. As I know from Neural Networks, you usually want the posterior for predictions. Therefore I don't understand what the use-case for evaluating expectations would be. If you could please help me with a couple of practical examples for where one would decide to compute expectations of functions with respect to a probability or redirect me to an explanation, I would be very thankful.
Linear regression: comparing effects between multiple (50+) groups in R
I have a dataset of 30.000+ observations. For my thesis I am investigating the effect of the weather on rating scores. For a subquestion I need to compare the effect of precipitation on the review score for people from different countries. specifically: I’m trying to find out if the effect of precipitation on the rating score is higher for people from countries where the yearly average precipitation is lower. I ran the regression models for all different countries in the dataset and I am wondering how to best proceed from here. How can you test whether the effect of one country is significantly higher than that from another country to test if the hypothesis is true? This are the models: # regression per country by_country part of the data: output data_grouped https://docs.google.com/spreadsheets/d/1eOi10fiqIFSHVXZLYNnAHoxlP1YtUPEDyO6pjBKA6iM/edit?usp=sharing output data_grouped2 https://docs.google.com/spreadsheets/d/1-EL0-TsU9PlbKxi8xU0vIaq7JVT381msG0ZmF5xVBv4/edit?usp=sharing
MIT simulator lets users design wide range of functional soft robots
MIT simulator lets users design wide range of functional soft robots aconner Mon, 06/10/2019 - 12:37 Article June 10 '19 Adam Conner-Simons, MIT CSAIL MIT simulator lets users design wide range of functional soft robots To get robots to do things, computer scientists often use systems called physics simulators that reflect how a robot’s actions will impact the real world. These simulators don’t work particularly well, however, when it comes to soft robots made of flexible, deformable materials. This is because the underlying physical laws of deformable objects are much more complicated, requiring a lot more computational power to simulate. But in a new paper, a team from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) has developed a new simulator made specifically for soft robots, and have shown that it can realistically simulate an eclectic mix of robotic designs, from a crawling robot to a four-legged running robot. The simulator doesn’t just efficiently evaluate robot designs, but also provides feedback on how designs can be improved. (The system’s feedback is computed based on something called “the chain rule,” and so the team has dubbed the simulator “ChainQueen”.) The team developed a high-performance GPU implementation of the simulator that they hope to eventually make open-source. “We believe this system has the potential to dramatically accelerate the development of soft robots,” says PhD student Andrew Spielberg, one of the co-authors of the…
Centering input data for Robust PCA (RPCA)?
I know that before running Principal component analys , the input data needs to be centered around its mean (subtract the mean from each keypoint) before running the algorithm. Do I need to center my data before running robust PCA ? For instance, say I have a video from a static security camera (like in the original work). Do I need to subtract from each pixel (in each frame), its mean value over the duration of the video? p.s. any other pre-processing required for RPCA?
Wireless health monitoring system shows promise in clinical trials
Wireless health monitoring system shows promise in clinical trials aconner Tue, 05/28/2019 - 13:35 Article May 30 '19 Novartis Wireless health monitoring system shows promise in clinical trials Over the past year MIT CSAIL has worked with Novartis to test a novel technology for passive, contactless monitoring of physiological signals that may be used to monitor clinical trial patients in their homes. Developed by Professor Dina Katabi and her students, the technology consists of a Wi-Fi-like device that transmits low-powered radio signals and uses machine learning algorithms to analyze their reflections and produce physiological metrics. The device can gather data on patient mobility, gait, breathing, heart rate, sleep stages, sleep apnea, and other metrics without requiring the patient to wear sensors or change their behavior in any way. Novartis and the MIT team explored the potential use of this technology in clinical trials to collect digital biomarkers, both existing and new, and potentially allow continuous, real-time monitoring of patients in their own homes. As part of the collaboration, Novartis deployed the technology in a Novartis facility, as well as in a life sciences facility with a living lab, sleep monitoring, motion and behavior monitoring. Individuals were studied for multiple days in the lab, and their motion, breathing, sleep, and behavior were measured using the technology and compared against existing standards for such measurements. Comparison to the g…
This robot helps you lift objects — by looking at your biceps
This robot helps you lift objects — by looking at your biceps rachelg Wed, 05/22/2019 - 10:30 Video May 22 '19 Rachel Gordon CSAIL system can mirror a user's motions and follow nonverbal commands by monitoring arm muscles. We humans are very good at collaboration. For instance, when two people work together to carry a heavy object like a table or a sofa, they tend to instinctively coordinate their motions, constantly recalibrating to make sure their hands are at the same height as the other person’s. Our natural ability to make these types of adjustments allows us to collaborate on tasks big and small. But a computer or a robot still can’t follow a human’s lead with ease. We usually either explicitly program them using machine-speak, or train them to understand our words, à la virtual assistants like Siri or Alexa. In contrast, researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) recently showed that a smoother robot-human collaboration is possible through a new system they developed, where machines help people lift objects by monitoring their muscle movements. Dubbed RoboRaise, the system involves putting electromyography (EMG) sensors on a user’s biceps and triceps to monitor muscle activity. Its algorithms then continuously detect changes to the person’s arm level, as well as discrete up-and-down hand gestures the user might make for finer motor control. The team used the system for a series of tasks involving picking up and assembling mock…
Nonparametric approach for regression with a quadratic fit
I'm trying to figure out which nonparametric test I should run on my data. My data has residuals that are not normal, so I cannot run a linear regression unless I log transform it. However, log transforming my data would make it difficult to interpret a quadratic model, so I need to run a nonparametric test similar to regression. I'm trying to compare 2 models and determine which has a better fit - a linear model (y~x) or a quadratic model (y~x+x^2). Which nonparametric approach similar to regression should I use to construct each model?
a measure for MAE (of a regression)
I'm running a grid search, in order to fine-tune a NN hyper parameters. the question is: the MAE values I get from the trainings are too close. since I have the statistical attributes of the target values, is there a way to somehow come up with a starting value for MAE, where the worst regression can achieve. I think the question is not clear. the value I'm looking for, is analogous to a probability for classification problems. (example: an neural network which is supposed to classify the inputs into 8 possible classes, will have accuracy of 0.125 just by random classification of inputs. so a 12.5% accuracy is the measure for such classifier) now, I have the Mean, and StdDev (and other stats if need be) of the target values of my samples. how can I calculate a measure to judge my MAE (and not by comparing different MAE values from different trainings)?
CSAIL's Daskalakis wins ACM Grace Murray Hopper Award
CSAIL's Daskalakis wins ACM Grace Murray Hopper Award rachelg Wed, 05/08/2019 - 10:20 Article May 08 '19 Rachel Gordon Constantinos (“Costis”) Daskalakis, an MIT professor and CSAIL principal investigator, has won the 2018 ACM Grace Murray Hopper Award. Constantinos (“Costis”) Daskalakis, an MIT professor and CSAIL principal investigator, has won the 2018 ACM Grace Murray Hopper Award. Announced today, the prize is awarded yearly to a computer scientist on the basis of a single recent major technical or service contribution, made at or before age 35 at the time of the contribution. Daskalakis was honored for “ proving that the computational complexity of finding Nash equilibria is the same as that of finding Brouwer fixed points, a proof since extended to several other equilibrium notions.” “By challenging equilibrium theory, his work has triggered an ongoing reshaping of our understanding of strategic behavior, showing that computation must play an essential role in the foundations of game theory and economics.” His research, a fusion of computer science, economics and game theory, focuses in part on how strategic behavior complicates large-scale technological systems. To study these systems, researchers typically use equilibrium concepts, and very prominently the concept of Nash equilibrium, which occurs when every player does the best they can given other players’ choices, so no player can benefit from unilaterally changing their choice. However, Nash’s equilibrium existe…
Giving robots a better feel for object manipulation
Giving robots a better feel for object manipulation rachelg Mon, 04/29/2019 - 13:27 Video April 29 '19 Rob Matheson Model improves a robot’s ability to mold materials into shapes and interact with liquids and solid objects. A new learning system developed by MIT researchers improves robots’ abilities to mold materials into target shapes and make predictions about interacting with solid objects and liquids. The system, known as a learning-based particle simulator, could give industrial robots a more refined touch — and it may have fun applications in personal robotics, such as modelling clay shapes or rolling sticky rice for sushi. In robotic planning, physical simulators are models that capture how different materials respond to force. Robots are “trained” using the models, to predict the outcomes of their interactions with objects, such as pushing a solid box or poking deformable clay. But traditional learning-based simulators mainly focus on rigid objects and are unable to handle fluids or softer objects. Some more accurate physics-based simulators can handle diverse materials, but rely heavily on approximation techniques that introduce errors when robots interact with objects in the real world. In a paper being presented at the International Conference on Learning Representations in May, the researchers describe a new model that learns to capture how small portions of different materials — “particles” — interact when they’re poked and prodded. The model directly learns fr…
A Recipe for Training Neural Networks
Some few weeks ago I posted a tweet on “the most common neural net mistakes”, listing a few common gotchas related to training neural nets. The tweet got quite a bit more engagement than I anticipated (including a webinar :)). Clearly, a lot of people have personally encountered the large gap between “here is how a convolutional layer works” and “our convnet achieves state of the art results”. So I thought it could be fun to brush off my dusty blog to expand my tweet to the long form that this topic deserves. However, instead of going into an enumeration of more common errors or fleshing them out, I wanted to dig a bit deeper and talk about how one can avoid making these errors altogether (or fix them very fast). The trick to doing so is to follow a certain process, which as far as I can tell is not very often documented. Let’s start with two important observations that motivate it. 1) Neural net training is a leaky abstraction It is allegedly easy to get started with training neural nets. Numerous libraries and frameworks take pride in displaying 30-line miracle snippets that solve your data problems, giving the (false) impression that this stuff is plug and play. It’s common see things like: >>> your_data = # plug your awesome dataset here >>> model = SuperCrossValidator ( SuperDuper . fit , your_data , ResNet50 , SGDOptimizer ) # conquer world here These libraries and examples activate the part of our brain that is familiar with standard software - a place where clean API…
What is the corresponding model when I include drift in an SARIMA model with seasonal differences?
I fitted the following model, but I am not sure if the drift term is an intercept term or the mean of $y_t-y_{t-4}$ (as it seems in a blog post by Rob Hyndman). (f
Handling missing data in Sequence Analysis (TraMineR) within the observation window
I'm using sequence analysis. I have a question about how to deal with missing data within the observation window. The starting point of the analysis is when respondents leave secondary school (t0). I want to examine respondents' life course over a time-span of 36 months after leaving school. The dataset contains longitudinal information of respondents toward their educational histories. I arranged the data in the 'states-sequence' (STS) format. So in each month the dataset provides information on respondents' status (for example "employed" or "training"). For 58% of the sample, the data provides information over the whole observation window. So for this group, I can tell in every single month what they are doing. The sequences of the rest of the sample are shorter. Thus, the length of the sequences is not the same for all respondents. How do I handle sequences of respondents that end before month 36? What would be the way the missing values should be handled in TraMineR?
Stationary and non-stationary variables in time series - how to difference?
I want to predict a multivariate daily time series, the target output is the volume of packages that is send and the covariates are day specific information as weather, the distance to holidays but as well lagged values of the target variable. The target output time series is not stationary, when I difference it, it is. So my intention was to just difference every variable. However, some of my covariates are already stationary, so differencing makes them non-stationary. I am not really sure if I should difference everything or nothing or just some variables, where the latter sounds not really reasonable to me. Could you please help?
Confidence interval for Population Attributable Fraction with several strata
I have used aggregated data to create a table of person-years (pys) and deaths by social class, age and sex. If we consider social class to be a modifiable factor, we can calculate the number of 'expected' deaths in a situation where the low class group has the same mortality rate as the high class group. The difference is the number of attributable deaths. In the example below this is 38, and the Population Attributable Fraction for social class is 38 / 182 = 21%. +-------+-------+--------+------+--------+--------+----------+--------------+ | Class | Age | Sex | Pys | Deaths | Rate | Expected | Attributable | +-------+-------+--------+------+--------+--------+----------+--------------+ | High | Young | Male | 100 | 10 | 0.1 | 10 | 0 | | High | Young | Female | 120 | 12 | 0.1 | 12 | 0 | | High | Old | Male | 40 | 8 | 0.2 | 8 | 0 | | High | Old | Female | 80 | 12 | 0.15 | 12 | 0 | +-------+-------+--------+------+--------+--------+----------+--------------+ | Low | Young | Male | 200 | 30 | 0.15 | 20 | 10 | | Low | Young | Female | 200 | 30 | 0.15 | 20 | 10 | | Low | Old | Male | 160 | 40 | 0.25 | 32 | 8 | | Low | Old | Female | 200 | 40 | 0.2 | 30 | 10 | +-------+-------+--------+------+--------+--------+----------+--------------+ | ALL | ALL | BOTH | 1100 | 182 | 0.1655 | 144 | 38 | +-------+-------+--------+------+--------+--------+----------+--------------+ Do you know how I would calculate a confidence interval for this fraction? It seems straightforward to calculate a P…
k-fold cross validation with multiple classes
I'm working on an image retrieval system (not classification). I have 5,000 images as the data set. 500 images of this dataset are the query images used for retrieval evaluation. these 500 images represent 10 different landmarks. the retrieval evaluation requires to evaluate each landmark using the average precision. and then mean average precision is measured to evaluate the 10 landmarks. However, I have a different number of query images for each landmark. some landmarks have 200 (out of 500) images as a query image and some have only 10. I'm required to divide the 500 query images into 5 folds. My question is how to perform the k-fold cross validation when the query images for each landmark varies from 10 to 200. in other words, how to deal with k-fold cross-validation in multiple classes and the sizes of the classes are different. my work is similar to the evalaution of this paper . EDIT as an example: I have 5000 images represents 10 landmarks. I have 500 query images (out of the 5000 images). The query images are as follows: landmark 1: 50 images (out of the 500). landmark 2: 10 images (out of the 500). landmark 3: 70 images : landmark 10: 200 images. I need to measure the retrieval performance for each landmark. The required number of folds is 5. Which means the 500 are supposed to be divided into 5 folds with 100 each. My question is: how to deal with the query landmarks of different sizes when distributing them across the folds?
The expected occurences of successive draws [duplicate]
We throw a coin 1 000 000 times. How many times on average will make 13 successive? Now the problem with the naive: 1 000 000/(2^13) is that once it made 13 heads the 14 head will happen with 1/2 probability , but it will count as 2 13 successive heads. 15 will happen 4xtimes less, but it will count as 3 13 successive heads. (which is far more probable than 3x1 000 000/(2^13)) if they were happening discretely. So 14 and 15 and anything above will be counted as a 13-success with value of 2,3 and so on. Monte carlo simulation also suggests something is fishy: https://stackoverflow.com/questions...nerator?noredirect=1#comment96559899_548950720134 † So what exactly is the expected number of 13 successive heads?
Proof that the addition of a baseline to the REINFORCE algorithm reduces the variance
A widely used variation of REINFORCE is to subtract a baseline value $b$ from the return $G_t$ to reduce the variance of gradient estimation, such that \begin{align} \nabla_\theta J(\theta) & \propto \sum_s d(s|\pi_\theta) \sum_a (q_\pi(s,a)-b(s)) \nabla_\theta \pi(a|s,\theta) \\ \end{align} I haven't found any proof that the baseline reduces the variance of the gradient estimation, is there one?
Importance sampling and exponential moving average
Lets say I have got a random variable $X$ with samples $x_t\sim X$ and density $p_X(x)$ and want to compute its mean via a moving average $ \mu_{t+1}=(1-c)\mu_t + c x_t$ Assume, I can not observe $X$ directly, but instead a random variable $Y$ with density $p_Y(y)$ and samples $y_t\sim Y$ . I would like to use importance sampling to compute the mean. The naive approach is with $w_t=\frac{p_X(y_t)}{p_Y(y_t)}$ $ \mu_{t+1}=\left(1- cw_t\right)\mu_t + c w_t y_t$ This works, as long as the importance weights are small. However, if $w_t> \frac 1 c$ the above formula is obviously not correct any more. Is there a way to correct for this? It should be clear that as $w_t \rightarrow \infty$ , $\mu_{t+1}\rightarrow y_t$ //edit the approach I tried to is using a sample-size estimate, but i am not sure this is correct. The initial estimate has an estimated $1/c$ samples stored. Assuming we can interpret $w_t$ as sample-size correction, we can just try to average according to how many samples we got: $ \mu_{t+1}=\left(1- \frac{w_t}{w_t + \frac 1 c -1}\right)\mu_t + \frac{w_t}{w_t+\frac 1 c-1} y_t$ for $w_t=1$ this gives the original update $\frac 1 c-1$ is the totally stored number of samples in the path after "forgetting the oldest" but i have no idea how to show that this is correct, this is just an educated guess
Maximum likelihood joint probability distribution (discrete & continuous)
I am trying to find the values $v_1$ and $v_2$ that maximizes the likelihood of some observations. I have information about $v_1$ and $v_2$ from a set of 'experiments'. In each experiment, $v_1$ and $v_2$ are corrupted with zero mean Gaussian noise, and then compared to each other, and the maximum of the two is reported. So if I have six experiments, I end up with six binary values. The standard deviation of the noise is $\sigma_c$ , equal for all experiments. I also have additional (independent) information about $v_1$ and $v_2$ . I observe two samples, $v^o_1$ and $v^{o}_2$ , sampled from a Gaussian distribution with mean equal to $v_1$ and $v_2$ (respectively) and standard deviation $\sigma_v$ (equal for the two observations). The problem is the following. I need to maximize $p(v\mid v^o, \text{experiments}, \sigma_v,\sigma_c)$ . Yet this is maximal when $v=v^o$ and $\sigma_v$ goes to zero. This occurs because the probability density goes to infinite at this point. Not sure how to deal with this. Any pointer in the right direction is much appreciated.
Reproducing paintings that make an impression
Reproducing paintings that make an impression rachelg Thu, 11/29/2018 - 13:52 Video November 29 '18 Rachel Gordon CSAIL's new RePaint system aims to faithfully recreate your favorite paintings using deep learning and 3-D printing. The empty frames hanging inside the Isabella Stewart Gardner Museum serve as a tangible reminder of the world’s biggest unsolved art heist. While the original masterpieces may never be recovered, a team from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) might be able to help, with a new system aimed at designing reproductions of paintings. RePaint uses a combination of 3-D printing and deep learning to authentically recreate favorite paintings — regardless of different lighting conditions or placement. RePaint could be used to remake artwork for a home, protect originals from wear and tear in museums, or even help companies create prints and postcards of historical pieces. “If you just reproduce the color of a painting as it looks in the gallery, it might look different in your home,” says Changil Kim, one of the authors on a new paper about the system, which will be presented at ACM SIGGRAPH Asia in December. “Our system works under any lighting condition, which shows a far greater color reproduction capability than almost any other previous work.” To test RePaint, the team reproduced a number of oil paintings created by an artist collaborator. The team found that RePaint was more than four times more accurate than state-of…
Survival model for an epidemic -- can the observations be treated as independent?
I've been thinking about ways to tackle an epidemic modelling problem I've been working on, and I've come up against a conceptual difficulty over the way survival analysis works. Here's a really simplified version with all the extraneous details stripped out. There is a collection of $n$ individuals. At the beginning (time $t = 0$ ), 1 individual is infectious with a disease, and the other $n-1$ individuals are susceptible. As time progresses, susceptible people can get the disease through contact with infectious individuals, and pass the disease on to other people. To keep focus on the core of my question we'll use these (unrealistic) simplifying assumptions: There are no births, deaths, immigration, or emigration Individuals mix uniformly (e.g. no preferential mixing by age or sex or location, etc) No delay between contracting the disease and ability to infect others (i.e. no incubation period) Once infected, an individual remains infectious forever (no recovery) Here is how transmission works: we use a continuous-time additive hazard model. This means that we let each uninfected individual $i$ have their own hazard function $h_i(t)$ . This function tells us "given that the individual $i$ has survived until the time $t$ , what is the infinitesimal rate of failure (i.e. infection) at time $t$ ?" We define it as follows: $$h_i(t ;\lambda) = \lambda \sum_{j \neq i}^n I_j(t)$$ Where $I_j(t)$ is simply an indicator function that is 1 when individual $j$ is infectious at time $t…
Expected value of quotient of Poisson distributions
Let $X$ and $Y$ be independent random variables such that $X \sim \text{Poisson}(\lambda \cdot c)$ and $Y \sim \text{Poisson}(\lambda \cdot (1-c))$ , where $c$ is a real number in $[0, 1]$ . Is there an easy way of proving that $E\left[\frac{X}{X + Y} | X + Y > 0\right] = c$ ? Numerical computations I made indicate that this is true.
Can I report a confidence interval for a kappa in which two raters have perfect agreement?
In a test of inter-rater agreement, I planned to report a kappa and lower+upper confidence bounds. However, I found 100% perfect agreement between the two raters. Is it possible to produce a confidence interval from this? In addition, is it appropriate to report a Kappa of 1?
What is the difference between a Random Vector (Joint r.v.) and a Random Process?
What is the difference between a Random Vector (Joint r.v.) and a Random Process? Kindly, explain with a simple example (like toss of a coin, roll of a die, picking a card, etc.). Note. As far as I understand, random process is a collection of equally spaced and indexed (generally, by time) random variables.
Comment on Meet the Investor: Daniel Guasco by Dejene Mulugeta
In reply to Dejene Mulugeta . Lets discuss in box dejenemulugeya50@gmail.com Many tanks!