“One-tailed” Levene Test


F-tests can be two-tailed (to test that $s_1^2 ne s_2^2$) or one-tailed (to test that $s_1^2 > s_2^2$).

How can I modify Levene/Brown-Forsythe to be “one-tailed”, that is, to test $s_1^2 > s_2^2$ instead of $s_1^2 ne s_2^2$?

Here is a demo:

demo

The image shows normally distributed training data (n=1000) and a model. An F-test is used to compare the variance of one point’s residuals (n=2) to the variance of all of the residuals (n=2000), so the point is an outlier if its residual variance is “too large.” The points are colored by p-value, where light points fit the model and dark points are outliers, and you can see that the two-tailed Brown-Forsythe rejects points that are too close to the model as well as too far.

Note: A different non-parametric, one-sided variance test would be fine as well.


Glen_b gave the information I needed, but I thought I would leave some implementation details (using scipy).

#basic F-test
F = var(a) / var(b)
Fp = stats.f.sf(F, df1, df2)

#Brown Forsythe
BF, BFp = stats.levene(a, b, center='median')

#two tailed t-test on transformed data
za = abs(a-median(a))
zb = abs(b-median(b))
t, tp_two_tailed = stats.ttest_ind(za, zb)

#the two tailed t test recapitulates the BF test
assert(t**2 == BF)
assert(p_BF == p_two_tailed)

#one tailed t test p value
tp = stats.t.sf(t, df)

scatter plots

Above shows scatter plots of the p values from the one-tailed $F$-test and two-tailed BF-test (left), and the one-tailed $t$ tests (right). Red points are “too close” ($s_1^2 < s_2^2$).

Questions about Factorial MANOVA


I have a few questions about the factorial MANOVA below which I hope can be answered:

1)What type of follow-up tests should be done after finding significant interaction effects in a factorial manova?

2)According to what I have read, Box’s test can essentially be ignored if I have equal sample sizes for different levels of my independent variable. Or if a significant result is found (violating the assumption), the Pillai’s Trace test can be used. Is this correct?

3)Is it necessary to use Bonferroni correction?

4)Even if the Levene’s test yields significant results for the dependent variable (this violates the assumption), I have read that it is okay to go ahead with follow-up tests as long as the sample sizes are equal and the standard deviations are within 20% of each other or are not 4 times greater. Is this correct?

5)Lastly, if the factorial MANOVA is run and it is not appropriate to proceed because of the violation of assumptions, how would you report this in a paper?

Thanks and hope that at least some of these questions can be addressed!

Test for equal variability in mixed model setting


I have a setting where I normally would model the variability in measurements by a linear mixed model which would look in R as follows.

require(lme4)
lmer(measure ~ equipment + (1|operator) + (equipment|batchid), data=mydataset) 

So basically

  • a fixed effect of equipment
  • random intercept per operator
  • random intercept per batch, differing for each type of equipment

Now in this model, I would like to define a test, which checks whether the variability explained by the equipment (fixed effect + random effect) changes by equipment level. There are 2 equipment levels and a unit based on which data is collected is called a batch.

Where can I find the specification of such a test?

Assumptions of two-way ANOVA and k-fold cross validation


I want to compare 3 classifiers (kNN, SVM and CT) by using their classification accuracies on 10 folds, to highlight eventual differences between them.

I think it could be done by a two-way ANOVA analysis, where classifiers=factors and folds=blocks, if some assumptions on data are verified.

Following wikipedia, the assumptions are:

  1. The populations from which the samples are obtained must be normally distributed.

  2. Sampling is done correctly. Observations for within and between groups must be independent.

  3. The variances among populations must be equal (homoscedastic).

  4. Data are interval or nominal.

I need an help on how to verify them in my case.

  1. Do I have to verify that for every classifier, its 10 accuracies are normally distributed and/or that for each fold, the 3 accuracies are normally distributed?

  2. Observations for within groups are independent because I use a different test set for every fold. Am I wrong? Observations for between groups are independent because I suppose classifiers to act in an independent way. Aren’t they?

  3. Do I have to verify that each group described in the first point has the same variance?

  4. No problems in my case.

Is there a quick way to verify all of the assumptions in Matlab?

Does Levene's test assume separate samples?


I want to run Levene’s test to test the equality of variances between a full sample a number of sub-samples. I can’t find anything about Levene’s test that states whether this would violate the assumptions of the test. In other words, given the null hypothesis that $mathrm{Var}(X_{1}) = mathrm{Var}(X_{2})$, does Levene’s test require that $X_{1} cap X_{2} = varnothing$?

ANOVA and Levene test problem (2)


I want to reword my previous question.

I’m running ANOVA for 4 clusters to understand is there any difference between these clusters in terms of their reasons for travel.

After I checked Levene test, one of my variables had a p-value lower than 0.05, but that variable has F(3,324) = 6.35, p=0.000 for the ANOVA. Now, I have 2 questions :

  1. When the Levene test is rejected, but on the other hand, the ANOVA test is also rejected, i.e. there is a statistical difference, can we accept this ANOVA analysis?

  2. When we used ANOVA for all of the variables and the Levene test is not rejected for all of them, with regard to this fact that the Levene test of one variable is rejected, can we use Kruskal-Wallis test just for that problematic variable or not? in other words, is it true that I just use a Kruskal-Wallis test for one variable?

Comparison of p values for Levene mean test and Levene median test?


I am doing Levene’s mean test and Levene’s median test (Brown-Forsythe).
I want to compare the p-values of these two tests to see which is better.
I get large p-values for both tests which are 0.562 (Levene mean) and 0.611 (Levene median) for normal distribution.

  • Which test shows the better type I error rate?
  • does Levene’s mean test perform best when the data follows a normal distribution?
Question and Answer is proudly powered by WordPress.
Theme "The Fundamentals of Graphic Design" by Arjuna
Icons by FamFamFam