AI/ML news, top picks, and generated innovation digests.
40179 matching items
Christian Dior designed the iconic UN tour guide uniforms in the early 1980s. Prior to that, designers included Edith Head and, for the UN’s first male tour guides, Brooks Brothers. The guides got a fashion refresh in 1985 by Harvé Benard, then changed their uniforms, courtesy of Benneton, and Mondrian. Image: Groups audience: General Assembly 78 Live Blog
Vice-President of China, Han Zheng, said in his speech to the GA said: "China supports all efforts that are conducive to the peaceful resolution of the Ukraine crisis, and stands ready to continue playing a constructive role for the early attainment of peace." Image: Groups audience: General Assembly 78 Live Blog
"Since 15 April, Sudanese people have been facing a destructive war launched by rebel RSF," said Abdel-Fattah Al-Burhan Abdelrahman Al-Burhan, President of the Transitional Sovereign Council of Sudan. "We call upon the international community to designate these groups and their allies as terrorist groups to be countered by arms and fought to protect Sudan, region and the entire world." Image: Groups audience: General Assembly 78 Live Blog
Happening now at the SDG Media Zone: Johan Santana, young innovator who codes to address SDG 3 and 10 (speaking). He expresses his desire to assist individuals with visual impairments and other disabilities. They've developed a hands-free blind cane and smart glasses and will work on coding a wheelchair capable of moving in all directions: forward, backward, left, and right. Also in the picture: Michael Melillo, Sr. Director of Network Monitoring and Management Software Products, Broadcom Inc.(left) Bervin Harris, Renaissance Youth Center (second from left). Image: Groups audience: General Assembly 78 Live Blog
Michael Lodge (Secretary General, International Seabed Authority) and Emilie McGloane (Director, Peace Boat US) met at the SDG Media Zone to discuss the importance of a new historic treaty - the Treaty of the High Seas - which protects biodiversity beyond national jurisdiction (often shortened to BBNJ). Michael Lodge emphasized the vastness of oceans and said that “1% of oceans contain more minerals than exist on land”. He continued: "Good regulation is important to ensure that the ocean’s resources are not exploited". So far, some 60 UN Member States have signed onto the treaty since its opening this week at UNGA78. Image: Groups audience: General Assembly 78 Live Blog
The Renaissance Youth Center choir visits the SDG Media Zone for a session on creative problem-solving and coding for the SDGs. The session explores the passion and enthusiasm of today’s youth to solve the challenges they face through creativity in STE(A)M and showcases successful outcomes when the private sector joins forces with NGOs to educate, inspire and empower youth as future STEM leaders in their communities and the world. Image: Groups audience: General Assembly 78 Live Blog
I'm computing mutual information for several features where one of my datasets has one instance. One instance is because of a specific filtering criterion I used. I'm using sklearn.feature_selection.mutual_info_classif¶ to calculate MI. I am getting an ValueError due to this one instance. Anyway, I wanted to quantify this case MI. Therefore I was looking for the definition of MI and searching for ways to quantify this specific case. Is it 0 ? - meaning just one feature value and label, no information content? How do we quantify this specific situation? Any suggestions are appreciated. Example: X = array([[7., 7., 0., 0., 1., 0., 2., 1., 0.]]) #set of feature values Y = array([2.]) #label mi_scores = mutual_info_classif(X, Y,random_state=0) Something like the above. Then I'm getting ValueError: Found array with 0 sample(s) (shape=(0, 1)) while a minimum of 1 is required.
I've been studying diffusion models and came across the noise schedule, particularly how the noise variance $\beta_t$ is adjusted over iterations. I've observed that $\beta_t$ typically starts from a very small value during the initial steps and increases to a much larger value in the final steps. What is the underlying reason for this progression? Why is it important to begin with a low noise variance and end with a high one? I'd appreciate any intuitive explanations or references that shed light on this choice.
Update June 2024: Anyscale Endpoints (Anyscale's LLM API Offering) and Private Endpoints (self-hosted LLMs) are now available as part of the Anyscale Platform. Click [here](https://console.anyscale.com/?utm_source=anyscale&utm_medium=blog&utm_campaign=blog_callout&utm_content=june2024_product_update_subheading) to get started on the Anyscale platform.
We’ve noticed an unusual training pattern in fine-tuning LLMs. At first we thought it’s a bug, but now we think it shows LLMs can learn effectively from a single example.
I'm asking specifically about ChatGPT4, but the question could apply to either that or 3.5. When you use the ChatGPT API, it's of course up to you to manage conversation history and include that in successive API calls within available context length in whatever manner you choose. In the case of the web interface, they've obviously implemented some system to manage conversation history in context. It clearly doesn't "remember" the entire thing once the conversation gets very long, because it doesn't have infinite context length. So, what strategy does it use to send conversation history to the model once it's exceeded its context length? Does it truncate all content prior to the max context length? Does it summarize earlier parts of conversations to more efficiently fit them within the context? Does it do some dynamic strategy combining many inputs? Or is this just another case where we just don't know, and OpenAI is being tight-lipped about what it's actually doing?
I'm trying to evaluate whether the difference in uplift seen in below table between the test groups and the control group is statistically significant. I'm unsure about the appropriate statistical test. The test and control groups are all of different sizes already before the test, which is why I have included the relative numbers. I first thought about the chi-square test, but that don't think it captures the pre- & post-treatment aspect correctly. For context: Four different geographic regions were selected, one of them as control. Each test region received a different mix of marketing measures with the goal to raise awareness. I am now trying to evaluate whether the uplift in the test regions is statistically different from the control region. Control Group 1 Group 2 Group 3 Number of visitors (pretest) 59800 9993 19284 17876 Number of visitors (posttest) 65993 11781 23373 20883 Relative Change +10.36% +17.89% +21.20% +16.82% Which statistical test would be needed to find an answer to my question? Thank you very much. Edit: The below table shows the weekly visitors by test group. Week 23 to 28 are pre-treatment, week 29 to 34 are post-treatment. Control Group 1 Group 2 Group 3 Week 23 8590 1492 2929 2837 Week 24 9217 1588 3138 2846 Week 25 9534 1599 2992 2812 Week 26 10213 1714 3440 3005 Week 27 10435 1704 3187 2987 Week 28 10932 1817 3180 3234 Week 29 11489 1948 3566 3159 Week 30 10936 1974 3707 3273 Week 31 11856 2061 3885 3609 Week 32 10621 1851 3926 3586 Week 33 9905…
I have two time series and I computed the percentage change like: (Value at time2)/(Value at time1) - 1 My doubt is on the correlation of the original time series and the percentage change time series. The correlation for the original time series is -0.2 while for the percentage change is 0.22. My question is just on a theoretical level: is it possibile to have opposite relationships between the two pairs of time series? How should I interpret this opposite behavior of the two couple of time series?
Can we interpret the coefficients of an ordinal regression in the coefficients table without involving the intercepts? For example library(ordinal) > head(wine) response rating temp contact bottle judge 1 36 2 cold no 1 1 2 48 3 cold no 2 1 3 47 3 cold yes 3 1 4 67 4 cold yes 4 1 5 77 4 warm no 5 1 6 60 4 warm no 6 1 fm |z|) tempwarm 2.3212 0.7009 3.311 0.000928 *** contactyes 1.3475 0.6604 2.041 0.041300 * tempwarm:contactyes 0.3595 0.9238 0.389 0.697129 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Threshold coefficients: Estimate Std. Error z value 1|2 -1.4113 0.5454 -2.588 2|3 1.1436 0.5097 2.244 3|4 3.3771 0.6382 5.292 4|5 4.9420 0.7509 6.581 What does the contactyes coefficient mean on its own? EDIT: Here are the plots for the probabilities per rating level
I'm working with data from an experiment that I plan to analyze using a mixed-effects logistic model. In this study, 200 participants (identified by the variable Participant) were randomly assigned to one of four experimental conditions (variable: Condition). Within each condition, participants were tasked with determining whether a given word (variable: Word) could appropriately conclude a sentence (variable: Sentence). We used a set of 30 unique words, and each participant had to evaluate all of them. For each word, participants were presented with two sentences: one where the word could correctly be used at the end (variable: Appropriateness = 'Yes') and another where it could not (variable: Appropriateness = 'No'). Participants earned a point (variable: score) for each correct judgment they made. I would appreciate any advice on how to best approach the statistical analysis of this data using mixed-effects models. Here's what I've been considering: Model 1: score ~ Condition * Appropriateness + (1|Participant) + (1|Word) Model 2: score ~ Condition * Appropriateness + (1|Participant) + (1|Sentence/Word) Model 3: score ~ Condition * Appropriateness + (Appropriateness|Participant) + (Appropriateness|Sentence/Word) As you can see, my particular concern is related to specifying the random effects. Please share your thoughts or suggestions on which model might be the most appropriate for this type of data.
Suppose $(X_1, . . . ,X_n)$ is an i.i.d. sample from the distribution $f_{\theta,k}(x)$ , where $\theta \in (0, 1)$ and $k = 1, 2$ . Assume that $$f_{\theta, k}(x)=\begin{cases} \text{Poisson($\theta)$}, &\text{if $k=1$}.\\ \\ \text{Bernoulli($\theta$)}, & \text{if $k=2$}. \end{cases}$$ . Check if $T=\sum_{i=1}^nX_i$ is a sufficient statistic for this family. If not, then find a sufficient statistic for this family. $$$$ My Attempt to the solutions is as follows : I found that $$\mathbb{P}(X_1=x_1, ...,X_n=x_n|T=t) =\begin{cases} \frac{n!}{x_1!x_2!....x_n!}(\frac{1}{n})^t & \text{if} &\sum_{i=1}^nx_i=t, &X_1, ...., X_n \sim \text{Poisson}(\theta) \\ \\\frac{1}{n \choose t} &\text{if} &\sum_{i=1}^nx_i=t, &X_1, ...., X_n \sim \text{Bernoulli}(\theta) \end{cases}$$ So $T$ is not sufficient for this family. $$$$ Now we can write the joint density as $$f_{\theta, k}(x_1, ...., x_n)=\frac{e^{-n\theta}(\theta)^{\sum_{i=1}^nx_i}}{\prod_{i=1}^n(x_i)!}\textbf{1}(k=1)+(\theta)^{\sum_{i=1}^nx_i}(1-\theta)^{n-\sum_{i=1}^nx_i}\textbf{1}(0 \leq x_{(1)} \leq x_{(n)} \leq 1)\textbf{1}(k=2)$$ The indicator $\textbf{1}(0 \leq x_{(1)} \leq x_{(n)} \leq 1)$ is because the support in our case is $\chi=\mathbb{N} \cup 0$ . So by the Factorization Theorem we get that $T(X_1, ...., X_n)=(\sum_{i=1}^nX_i, \prod_{i=1}^n(X_i)!, X_{(1)}, X_{(n)})$ is a sufficient statistic for this family as we can take $g_{\theta, k}(T(x_1, ...., x_n))$ equal to the density and $h(x_1, ...., x_n)=1$ . $$$$ Now to find…
Update June 2024: Anyscale Endpoints (Anyscale's LLM API Offering) and Private Endpoints (self-hosted LLMs) are now available as part of the Anyscale Platform. Click [here](https://console.anyscale.com/?utm_source=anyscale&utm_medium=blog&utm_campaign=blog_callout&utm_content=june2024_product_update_subheading) to get started on the Anyscale platform.
I was looking for literature which connects prediction intervals with the bias-variance trade-off. Obviously both concepts deal with describing a mean squared deviation: the bias variance tradeoff deals with $E_{Y,x,D}[(Y-\hat{Y}(x)_D)^2]=E_{x}[E_{Y,D}[(Y-\hat{Y}(x)_D)^2]]$ , which takes the average over (potentially deterministic) x, different datasets D from the same distribution, and resulting variable y. Now $E_{Y,D}[(Y-\hat{Y}(x)_D)^2]$ looks quite related to the following quantity $Var_{Y,D}[Y-\hat{Y}(x)_D]=E_{Y,D}[((Y-\hat{Y}(x)_D)-E[Y-\hat{Y}(x)_D])^2]=E_{Y,D}[(Y-\hat{Y}(x)_D)^2]$ (assuming an unbiased estimator in the last equality) which is repeatedly used in the construction of prediction intervals. Both the prediction interval and the variance-bias tradeoff indicate a minimal mean squared error dictated by the measurement noise $Var(Y)=\sigma^2$ . So I really wonder why I cannot find papers/books discussing this close relationship. Could somebody please tell me a) literature or b) point out the weaknesses in my argument above? PS: I realize that the bias variance tradeoff typically has this additional average over x. Surely, this averaging will have an effect when talking about heteroscedastic noise, i.e. $\sigma(x)$ (this is not the scope of this question) I know that predicition intervals do not have to be symmetric, so higher moments than just the variance might be needed for constructing a prediction interval with correct coverage (but this is not the scope o…
[ LinkedIn discussion , Twitter thread ] Never before in my life had I seen so many smart people working on the same goal: making LLMs better. After talking to many people working in both industry and academia, I noticed the 10 major research directions that emerged. The first two directions, hallucinations and context learning, are probably the most talked about today. I’m the most excited about numbers 3 (multimodality), 5 (new architecture), and 6 (GPU alternatives). 1. Reduce and measure hallucinations Hallucination is a heavily discussed topic already so I’ll be quick. Hallucination happens when an AI model makes stuff up. For many creative use cases, hallucination is a feature. However, for most other use cases, hallucination is a bug. I was at a panel on LLM with Dropbox, Langchain, Elastics, and Anthropic recently, and the #1 roadblock they see for companies to adopt LLMs in production is hallucination. Mitigating hallucination and developing metrics to measure hallucination is a blossoming research topic, and I’ve seen many startups focus on this problem. There are also ad-hoc tips to reduce hallucination, such as adding more context to the prompt, chain-of-thought, self-consistency, or asking your model to be concise in its response. To learn more about hallucination: Survey of Hallucination in Natural Language Generation (Ji et al., 2022) How Language Model Hallucinations Can Snowball (Zhang et al., 2023) A Multitask, Multilingual, Multimodal Evaluation of ChatGPT…
Background | Classifier-Free Guidance Derivation To summarize the derivation of Classifier-Free Guidance, looking at this paper (Page 21.), we can write classifier guidance as: $$\nabla_{x}\log p\left(x_{t}\mid y\right) =\nabla_{x}\log\left(\frac{p\left(x_{t}\right)\cdot p\left(y\mid x_{t}\right)}{p\left(y\right)}\right) =\nabla_{x}\log p\left(x_{t}\right)+\nabla_{x}\log p\left(y\mid x_{t}\right)-\nabla_{x}\log p\left(y\right) =\nabla_{x}\log p\left(x_{t}\right)+\nabla_{x}\log p\left(y\mid x_{t}\right)$$ and we can amplify the guidance by adding a factor $\gamma$ : $$\nabla_{x}\log p\left(x_{t}\mid y\right)=\nabla_{x}\log p\left(x_{t}\right)+\gamma\cdot\nabla_{x}\log p\left(y\mid x_{t}\right)$$ Then to derive classifier-free guidance all we do is rewrite the first equation we saw: $$\nabla_{x}\log p\left(y\mid x_{t}\right)=\nabla_{x}\log p\left(x_{t}\mid y\right)-\nabla_{x}\log p\left(x_{t}\right)$$ and substitue it into the 2nd equation we saw to get $$\nabla_{x}\log p\left(x_{t}\mid y\right)=\nabla_{x}\log p\left(x_{t}\right)+\gamma\cdot\left(\nabla_{x}\log p\left(x_{t}\mid y\right)-\nabla_{x}\log p\left(x_{t}\right)\right)$$ and this is classifier-free guidance. We can also rewrite it to be $$\nabla_{x}\log p\left(x_{t}\mid y\right)=\gamma\cdot\nabla_{x}\log p\left(x_{t}\mid y\right)-\left(\gamma-1\right)\cdot\nabla_{x}\log p\left(x_{t}\right)$$ The Question It seems reasonable to me that as we can amplify and play with the guidance factor, we could also add a factor $\be…
Intro: I have dropped math since high school, therefore I am trying to describe my question in layman's terms. Please forgive me for not typing the convention notations. I even do not know what is the correct term to put in the Google search bar for doing research. Therefore, any suggestion is greatly appreciated. Scenario : Imagine I am doing a field study to record the number of animals in a field. It is impossible to count everything, and I am going to sample a 1 km2 area. It is assumed that the number of my observation is depending on the following 3 variables (for simplicity): Underlying number of that animal. Whether the animal is active (i.e. how often they "appear" or "come" to the sample area). The length of time I perform the observation. The second point is relevant because I am not able to distinguish individual animals, therefore, if the same animal comes to the area twice, I will just count twice and assume they are not the same. However, I do assume that it is less likely to count the same animal if it is less active (see distributions 5 & 6 below). The third point is relevant because I am going to film the area for 10 minutes (as an example). Assuming I will look at the video many times and am able to count everything, the observed count will be higher if I lengthen the observation period (so that both less active and abundant animals might also go into the field for counting). Assuming I counted 100 different fields (exp1), I am now having a dataset as a cou…
I'm using SPSS. I have a multi-level categorical IV and a few continuous DV's. My main analysis goals are to test the effect of the IV on the DV, and then to follow-up with pairwise comparisons of level 1 of the IV to each of the other levels. However, Levene's test for homogeneity of variance is significant for all of them, and the sample sizes are very unequal across levels of the IV. To deal with the heterogeneity of variance and unequal sample sizes, I conducted Welch tests in place of standard one-way ANOVA's, and followed up with independent sample t-tests, consulting the adjustment for unequal variances where appropriate (equivalent to a Welch t-test). However, I have now realised that my main IV is confounded with some of the measured demographic factors (age, sex, region) that also have an effect on the DV, so I'd like to control for that. To do this, it would make sense to run an ANCOVA, with pairwise comparisons using estimated marginal means, rather than raw means. However, ANCOVA (of course, as a variant of ANOVA) assumes equal sample sizes and homogeneity of variance. Is there a way I can simultaneously (i) correct for these assumption violations, while (ii) controlling for the possible effects of covariates? If not, which is the better route to take? I read that heterogeneity of variance is especially a problem if the variance ratio (largest variance/smallest variance) is greater than 2 and can perhaps be ignored if it's lower than that (though I don't know wh…
Preliminary Consider $n$ individuals each with observed data $ Z_i, i = 1, \ldots, n$ . For each individual $i$ , the longitudinal predictor $Z_i = \{Z_i(t_{i1}), \ldots, Z_i(t_{i,R_i})\}$ is measured at some finite number of observation times $t_i = (t_{i1}, \ldots, t_{i,R_i})$ . We can define a finite grid as $\bigcup_{i=1}^{n} t_i$ , covering all unique observation times for the sample, with $\tau = \max (\bigcup_{i=1}^{n} t_i)$ . The article says: Start with an arbitrary set of smooth basis functions $\psi_1(t), \ldots, \psi_K(t)$ to characterize the functional predictor, where $K$ denotes the total number of basis functions. The set of basis functions can be, for example, the empirical basis functions estimated by the conventional FPCA method , but can be flexibly extended to other basis functions. The functional data can then be rewritten in matrix notation as $(Z_1(t), \ldots, Z_n(t))^T = \bf{\lambda}\bf{\psi}(t)$ , where $\bf{\lambda} = (\lambda_1, \ldots, \lambda_n)^T$ , with $\lambda_i = (\lambda_{i,1}, \ldots, \lambda_{i,K})^T$ , and $\bf{\psi}(t) = (\psi_1(t), \ldots, \psi_K(t))^T$ , $t \in [0, \tau]$ . $\bf{M}$ is of dimension $K \times K$ , with the $(k, k')$ entry being $\langle \psi_k(t), \psi_{k'} (t) \rangle$ for $k, k' \in K$ , $t \in [0, \tau]$ . My question: As you can see $Z_{i}$ 's are irregular and sparse. So, if we were to write it in matrix notation, then we would have NAs in it. For example, #### set.seed(123) # number of observations n So, my unde…
I am interested in the comparison of Pearson correlation and Euclidean distance as measures of similarity between data points. Suppose I have 4 data points, w, x, y, z , in a multidimensional space, where w is a very extreme outlier and x, y, z are highly similar to each other . For the correlation measure, I assume that x, y, z have high correlation coefficients (positive) with each other, but not equal to 1. For the euclidean measure, I assume that x, y, z have small euclidean distances with each other, but not equal to 0. Now, if I take the arithmetic mean of w, x, y, z and call it m , how does the similarity between m and x, y, z change depending on the measure I use? Which measure is more robust to the presence of the outlier w and can still capture the similarity or closeness of x, y, z?
I'm running a regression model to see if my dichotomous independent variable (family type) predicts my outcome variable (well-being). I'm including four mediator variables, as a parallel mediation. I'm using Hayes Process Macro Model 4 for this. But...the problem is that I know that two of my four mediators have a direct curvilinear effect on the outcome variable. The other two have a linear effect on the outcome variable. How would I go about this? I've read that if I treat it all as linear, I could get a false positive because of the curvilinear. Ideally, I'd like to not split the four mediators into four models because I'm running it across four different outcomes and two comparison countries...so I'd end up with something ridiculous like 32 models (4x4x2) instead of 8, so parallel mediation would be more concise. Thank you so much!
Moving AI ethics beyond explainability and fairness to justice
I have observations from three groups of participants (A, B, C), that each represent ratings on three different measures (measure 1, 2, and 3). I would like to know if it is the case that, regardless of group, the distance between measure 3 and measure 2 is greater than the distance between measure 3 and measure 1. In STATISTICA, I'd have defined a mixed-model ANOVA with group as between subjects factor and "measure" as a repeated measures factor, defined an appropriate contrast, and then run a significance test on that contrast. I would now like to answer that question not in a GUI-based software like STATISTICA, but instead based on a posthoc pairwise test done on the output of a linear mixed model, conducted in MATLAB. However, after reading the documentation for fitlme and multcomp , I am left confused as to how to go about answering my question. I also don't really remember how that contrast vector should have been defined in STATISTICA, assuming something like this will be needed in the LMM also. An answer that replaces MATLAB with Python or R will be fine, as it should be easy for me to translate between them.
I am really confused regarding 2-way Anovas. I think I did not understand something properly. Let's say I have a 2x2 experiment design with 2 factorial variables (For the sake of an example let's say Letter and Number so A1, A2, B1, B2 as possible combinations) I would like to do a 2-way Anova since I hypothesise an Interaction between the variables Number and Letter. However, the sample sizes per group are quite small, unbalanced (n = 4-6) and not paired. As part of my experiment design I have 4 planned comparisons: A1 - A2, B1 - B2, A1 - B1, A2 - B2 Now two questions: As far as I understand I have two orthogonal contrasts? These are the simple main effects or are they interactions? After the Anova I can NOT do a post-hoc test (because I have planned comparisons and not testing opertunistic?). So I would only test my planned comparison with a t-test with a correction for multiple comparisons (since I am doing more than 3 comparisons). 3. Which t-test should I do (pairwise, two-sample, or not a t-test at all)? 4. Am I doing the planned comparisons regardless of the significance of the Anova? 4.5 . But If so why am I doing the Anova in the first place? 5. Is it okay to just do a t-test with correction, without the Anova? I looked at the following cross validation questions, but I am sill confused understand: Calculating 'k' in the bonferroni procedure when there are both post-hoc and planned constrasts Is it appropriate/acceptable to do planned comparisons for data that will…
I am using R. I have a data set with a binary (0,1) response and both continuous and categorical predictors. I would like to test the overall importance of these predictors one by one, and I am looking for suggestions on how to do this. I am not trying to select an overall model yet. I am just trying to see which variables are individually significantly associated to the response. Here is an example concerning categorical variables ... Let's call the response "y". Let's say my categorical predictor is called "xCat" and has levels A, B, C, and D. I would like to test if xCat has a statistically significant association with y. I want to test for overall significance, not just significant differences from a single reference group. Here is what I have tried so far ... Option A using LRT: fitCat Option B using drop1: fitCat I would then look at p-values for either of these outputs. If the p-value is >0.05, then I would say there is not significant association. Is this actually testing what I think it is testing? I am concerned about violating assumptions of normality and equal variance for the ANOVA. Any comments or suggestions? Here is an example of the continuous predictors... Let's call the continuous predictor "xCon". Here is what I've tried ... fitCon I would then look at the p-value from the output. If the p-value is >0.05, then I would say there is not significant association. Is there anything I'm missing here? assumptions I need to check or common pitfalls? Let me know i…
I was playing around with MobileStyleGAN pretrained model and multithreading and came along with an interesting phenomenom. After a while application is running MobileStyleGAN starts to produce video clip alike pieces. And I am wondering does anyone have an idea what takes? I made this lengthy video for it where you can see since it looks cool; https://youtu.be/8kpxHZK1NTQ Here's the source code used in the video if it's any help; https://github.com/harism/i_style_gan
I am trying to fine-tune Llama model on two task at the same time, using hugging face library: Main task: Causal language model like the model was initially trained for A classification task based on the whole input sequence (recommend an article). For this task I am getting as a reference the LlamaForCausalLM class, overwriting init and forward functions . However, I want to combine the two tasks above into one process. The main problem is that language modelling is an iterative process were the loss is calculated for every new context token in the input sequence, while for the classification task the loss should only be calculated once. How can I freeze the loss update on the classification task up and only calculated once the language modelling part has been completed. Is there any example you can recommend in order to combine a main LM task with an auxiliary classification task? First question for me here, thanks everyone for your understanding.
The example is trying to predict wether coffe is well roasted or badly. 1 is good roasted and 0 is bad. The architecture is: Now I try to visualize the model. Unit 1 has higher values when the duration of roasting is too little. Unit 2 has higher values for bad combinations of temperature and time. The blue-hatched regions show the layer outputted higher activations for the respective unit. I used a threshold of 0.5 So I can conclude higher activations from layer 1 give an output class of 0. But as I used the sigmoid activation function I thought that large values give an output class of 1. And very small values give a 0, due to the S-shape.
Model licensing & surveillance will likely be counterproductive by concentrating power in unsustainable ways
I know in order to shrink or expand the number of channels a 1x1 convolution is performed. I need to clarify the following: is the 1x1 convolution(s) just a matrix multiplication between the image with shape (h w, 3) (RGB) and a matrix that holds the learnable weights with shape (3, 1)? Which will result in a new matrix of shape (h w, 1) (in this case the number of channels shrunk from 3 to 1). If the above is correct, what happens under the hood of a NN framework, such as PyTorch, when the number of input channels is equal to the number of output channels? Does a matrix multiplication take place between the input (h*w, 3) and a matrix with learnable weights (num_channels, num_channels)? Doesn't this introduce unnecessary (and unwanted) operations?
Let Y be a vector of observations. A Gaussian Mixure Model (GMM) is fit to the dataset. The distribution can appear unsymmetrical, with different thickness of tails in both sides. What is the best way to find an optimal estimation for a confidence interval (e.g. $1-\alpha$ )? The goal is to identify the less probable observations. An straightforward solution would be to determine a threshold or a cutoff value for likelihood/probability below which a data point will be considered less probable. However there is no domain knowledge to judge this. I look for a Confidence Interval. I tried to find the bounds by optimizing the problem based on area under the curve. The solution is not the best due to unsymmetry and I assume The optimal CI should be the shortest interval too?! What is the best way to do that? Example of the fitted gaussians and the estimated PDF
I would like to find out if there is any impact of social media on the creation of entrepreneurial opportunities for entrepreneurs. I have two groups (Online Business and traditional Business) and 20 statements in total that I have measured using a 5 point Likert scale (strongly agree, agree, neutral, disagree, strongly disagree; and also not applicable, as suggested by my supervisor). Those who run traditional business chose "not applicable" for a lot of statements and I am now perplexed how to deal with this not applicable option! They are not missing values so I cannot ignore them as they bear important information about the traditional business group. My supervisor told me that I cannot even score this as 0 or 6. I am using R and R ignores not Applicable option while calculating correlation and even for other calculations as well. Moreover, I tried factor analysis but this "not applicable" option is creating problems in every step. I am new to deal with Likert scales and confused how to proceed with my data to figure out the answer of my research question.
Excellent documents for Research and Development
We are in a regression setting. Let's start by defining some notation and terminology. $y_i$ is observation $i$ of some (response) variable $Y$ . $\hat{y}_i$ is the value of $y_i$ predicted by a regression. $\bar{y}$ is the average of all observations of $Y$ . $$ y_i-\bar{y} = (y_i - \hat{y_i} + \hat{y_i} - \bar{y}) = (y_i - \hat{y_i}) + (\hat{y_i} - \bar{y}) $$ $$( y_i-\bar{y})^2 = \Big[ (y_i - \hat{y_i}) + (\hat{y_i} - \bar{y}) \Big]^2 = (y_i - \hat{y_i})^2 + (\hat{y_i} - \bar{y})^2 + 2(y_i - \hat{y_i})(\hat{y_i} - \bar{y}) $$ $$SSTotal := \sum_i ( y_i-\bar{y})^2 = \sum_i(y_i - \hat{y_i})^2 + \sum_i(\hat{y_i} - \bar{y})^2 + 2\sum_i\Big[ (y_i - \hat{y_i})(\hat{y_i} - \bar{y}) \Big]$$ $$ SSRes := \sum_i(y_i - \hat{y_i})^2 $$ $$ SSReg := \sum_i(\hat{y_i} - \bar{y})^2 $$ $$ Other = 2\sum_i\Big[ (y_i - \hat{y_i})(\hat{y_i} - \bar{y}) \Big] $$ The interpretation of the $SSRes$ seems straightforward enough, just the sum of the squared differences between the predicted and the true values. Why we would square these instead of taking the absolute value is not immediately obvious, but it at least makes sense why we would care about the difference between the true and predicted values. What intuition is there for $SSReg?$ Why should we care about the distance between the predicted values and the average value? Further, what does this have to do with an "explained" sum of squares? What is being explained? I can wrap my head around this when $Other = 0$ , such as in OLS linear regressi…
I read this interesting e-print paper on conformal predictors: A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification Conformal predictors are a way to choose a set that's guaranteed to include the true labels with some pre-chosen certainty. I was wondering if there's a way to get conformal predictors to output calibrated probabilities? For example, let's say I have a binary classification (dog or cat images). Conformal predictors can be used to predict whether an image is a dog or a cat in difficult examples. But what I'm looking for is something like calibrated p-values for the prediction. The sigmoid output values (from my neural net, for example) are well known not to reflect actual p-values. Can conformal predictors do this (assuming, of course, I have a calibration dataset available)? If so, can anyone point me to the procedure for this? I can't find it.
I have an example, where I want to automate matching up records in two datasets. I'm wondering what kind of machine learning model would potentially be able to deal with this kind of issue. I'm thinking Maybe some kind of transformer neural network without positional encoding (there's not really an obvious ordering, so LSTM or transformer with positional encoding seem less obvious). The number of records per genre in the real data is low enough that sequence length ought to be okay. Additionally, using pre-trained encoder for language may seem really obvious for capturing embeddings for the text fields, to capture information models have seen during training of role/film/actor name. While the text information will in fact often manage to directly produce the match, working with text only may often not be enough, because we also need to use the date information that may often matter (esp. when things are ambigious/incomplete). Possibly multiple binary losses? Many other models (e.g. GBDTs etc.) cannot deal with the multiple-inputs-multiple-outputs format (while they could of course take in text embeddings). Below are examples of what the data might look like (I cannot share the real data, but the below shares the core features): Genre show_movie year_start year_end Science Fiction Star Trek (original series) 1966 1969 Science Fiction Star Trek: The Motion Picture 1979 1979 Science Fiction Star Trek (2009) 2009 2009 Science Fiction Star Trek: Strange New Worlds 2022 Superhero…
I am trying to extract only the important neurons from the latent space of an Autoencoder to be converted later to a pattern for a model pattern recognizer. PCA Loadings helps in finding the highest correlation coefficient on the neurons of the latent space. Thus, the output of the PCA is not used, only the eigenvalues and eigenvectors to extract which neurons correlate more to the highest eigenvalues and pick only those neurons. For Example, Cifar-10 dataset. Extracting the latent Space of the Autoencoder. Then do PCA and extract the loadings. Pick only the neurons with high correlation to the principal components, with explained variance above 90%. My Questions: Before doing PCA on the latent space, do/do not normalize the data? Is this approach of employing PCA wrong to start with? There are multiple PCA variations like SparsePCA, KernelPCA, and RobustPCA. Is one of variation might be more beneficial for this task?
When we are applying a mask onto the padded values in an input sequence, it is typically done through setting the padded values as negative infinity. For example, a tensor of values [1,2,3,0,0] should result in a padding mask of pad_mask = [True, True, True, False, False] (or the opposite depending on your flavour). However, if we apply the mask i.e attention_scores = attention_scores.masked_fill_(pad_mask.T == False, float('-inf')) before applying softmax, won't we get the 4th and 5th row of the attention_scores as 'nan' when we softmax attempts to calculate the probability distribution along each row? Does that mean the step of where to apply the mask is incorrect, and we should apply a zero-ing out of the pad token rows in the attention_score matrix after applying the softmax function? or is there another key concept/step I am missing here
Cross posted from Mathoverflow Let $X$ and $Y$ be two random variables such that $X\sim Exp(\lambda)$ and $Y$ have positive support and (strictly) increasing hazard rate $h_Y$ . $X$ and $Y$ are independent. Let $Z=X+Y$ . We observe $Z$ and want to infer the hazard rate of $X$ conditional on $Z$ ; $h_{X|Z}$ . The interpretation is that some Poisson event occurs, then we observe ``it occurred" after a stochastic delay $Y$ , and we want to estimate when it occurred based on this observation. My conjecture is that the hazard rate $h_{X|Z}$ is decreasing. I have some elements, but I was not able to complete the proof. Here I go: By definition we have $h_{X|Z}(X=t_0|Z=t)=\frac{P(X=t_0|Z=t)}{P(X\geq t_0|Z=t)}.$ Let's compute the numerator and denominator: $$P(X=t_0|Z=t)=\frac{P(Z=t|X=t_0)P(X=t_0)}{P(Z=t)}=\frac{P(Y=t-t_0)P(X=t_0)}{P(Z=t)}$$ and, $$P(X\geq t_0|Z=t)=\frac{P(Z=t|X\geq t_0)P(X\geq t_0)}{P(Z=t)}.$$ Now observe that conditional $X\geq t_0$ , if $Z=X+Y=t$ , then it must be that $Y\geq t-t_0$ . [THIS WRONG, SEE BELLOW] So $P(Z=t|X\geq t_0)=P(Z=t \cap Y\geq t-t_0|X\geq t_0)$ . And therefore, we have: $$P(X\geq t_0|Z=t)=\frac{P(Z=t|X\geq t_0 \cap Y\geq t-t_0)P(X\geq t_0)P(Y\geq t-t_0)}{P(Z=t)}.$$ So putting numerator and denominator together we have: $$h_{X|Z}(X=t_0|Z=t)=\frac{h_Y(t-t_0) h_X(t_0)}{P(Z=t|X\geq t_0 \cap Y\geq t-t_0)}=\frac{h_Y(t-t_0) \lambda}{P(Z=t|X\geq t_0 \cap Y\geq t-t_0)}$$ . The nominator is decreasing in $t_0$ (because $h_Y$ is increasing). To finish th…