T test and ANOVA

A review of t test and basic concepts in ANOVA

Psychology Statistics

T test and ANOVA

Created 3 years ago

Duration 0:57:28
119
A review of t test and basic concepts in ANOVA
Slide Content
1. Slide 1 - Comparing Groups data

• Dwarves vs. Minions Which are better?
2. Slide 2 - Objectives

• When to use stat
• Independent vs. paired samples
• Within/Between design
• Calculate a df
• Why is multiple t test bad?
• One tails vs. two tails
• Orthogonality
• Assumptions of all stats
• How to report stats
• When to use different stats for assumptions
• What do the numbers mean when reported?
• Follow-up options in ANOVA, when & what?
3. Slide 3 - A study

• What’s the difference between Prozac and Ketamine in treatment of depression symptoms?
• Prozac= 6.58
• Ketamine= 9.77
• What is the IV?
4. Slide 4 - What should Dr. Williams do?

• The researcher has two independent samples- the behavior in one sample is not related to the behavior in the other sample.
• She could run an independent samples T test to compare the two (and only 2) means and see if they are significantly different.
• What is her research question?
5. Slide 5

• Research Question: Does type of drug affect (Prozac vs. Ketamine) depression symptoms?
6. Slide 6 - Categorical IV Continuous DV

• What is Dr. Williams doing?
• Comparing the distribution of one group with the distribution of the other group to find that point, or critical value, where group differences are improbable (p<.05) as a function of chance

8. Slide 8 - Remember it matters both mean and variance/distribution of your two groups

• Ketamine
• Prozac
• Ketamine
• Prozac
• Ketamine
• Prozac
• Looking really different
• Probably different at p<.05
• Are they different with
• P<.05?
9. Slide 9 - Variance (spread) & T-test

• When we are looking at the differences between scores for two groups, we have to judge the difference between their means relative to the spread or variability of their scores.
• The t-test does this.
• Compare two groups’ means and variances. Each group has its’ own sample size too.
10. Slide 10 - Formula

• N= total number of participants, n= number of participants within a group
• Numerator is just difference between means
• Denominator “ingredients” is variance (square of standard deviation) plus n= number of participants
• What you need to know: The t value is not just difference between means but also accounts for variance and sample size
11. Slide 11 - Explain to your neighbor

• For a t test to be significant what do you need in terms of mean and variance?
• Come up with another study that could be a t test- what is the IV and DV?
12. Slide 12 - Degrees of Freedom

• Depends on degrees of freedom (df): number of values in a set of scores that are free to vary after certain restrictions are placed on the data.
• For a mean score , if you were to sum up all the differences between each score in a study and the mean they would equal 0.
• Suppose we have just 5 scores in a distribution, 4 people give me a deviation from the mean (+ or -).
• What’s the 5th score have to be (hint=total has to be 0) so 4 numbers could be ANY number, while 1 number must be the same, n-1=4

14. Slide 14 - Degrees of Freedom

• If the mean of a set of 3 collected variables is 11 there’s an infinite number of options to get that mean (11,11,11..10,11,12)
• If I say one of those numbers is a 7 then it’s still infinite for the other two numbers
• If I tell you one is 7 an the other is 10 there is only one possible value for the third number
• So for this problem there is 2 degrees of freedom. Once two numbers are determined then the third number is fixed.
15. Slide 15 - Let’s Experiment…

• Mean shoe size is 8
• Frequency distribution of 5 people: 2, 5, 10, 11
• What does the fifth person’s size have to be?
• How many degrees of freedom in this problem?
• Formula for df:
• For one group df= n-1
• For two groups df = n-2
16. Slide 16 - T Test Please

• Dr. Williams wants to know if the two groups are different and her sample is small df=N-2 (because 2 groups).
• So if she has 26 people in the experimental (ketamine) group and 26 people in the standard (Prozac) group her df is?
• 50
• e.g., t(50)=4.3
• Note: Degrees of freedom in t test will tell you how many participants were in the study ;-)
17. Slide 17 - Just for Fun

• Interesting side story: The “inventor” of the T test, Gosset (1899), was motivated because he was brewing beer for Guinness in Ireland and wanted to know what kind of barley to use to make the product more consistent. He used the pseudonym “student” so Guinness wouldn't have to own up.
18. Slide 18 - Next Step:

• Between or within subjects design?
19. Slide 19 - Between vs. within subjects design

• Between subjects randomly (hopefully) assigns all participants to one group
• Either you get Ketamine or Prozac for the study
• Within subjects design allows everyone in the study to spend time in each condition/level of the IV
• You get one drug for a while, and then you get the other drug for a while
• This is a design issue that you need to know to tell your computer which version of T test you should run
20. Slide 20 - Assumptions of t test

• 1 Categorical IV (2 cats)
• 1 Continuous DV
• Robust to violations of assumptions in general… can be “liberal”
21. Slide 21 - Assumptions of t test

• N- unequal n problem for small sample
• Normality- Important for small sample size or low power; Not so important for moderate to large sample size.
• What’s small? Let’s use power analysis to decide…
22. Slide 22 - Homogeneity of variance

• Homogeneity of variance- a little important, but less so as sample gets larger
• Levene’s test will show up in SPSS output for you automatically
23. Slide 23 - Homogeneity of Variance

• In Independent Samples
• For between subjects design it is very important
24. Slide 24 - Violate Multiple Assumptions

• Homogeneity of Variance + Unequal n (two groups different sizes)
• If both violated, you have a real problem. Puts all your statistics in question.
25. Slide 25 - Self Test

• What is t-test doing?
• What are degrees of freedom?
• Come up with another example study that would be appropriate for a T test.
• Give example of heterogeneity of variance and unequal sample size
• Any questions?
26. Slide 27 - ANOVA does what?

• Categorical IV (2 or more cat)
• ONE continuous DV
• Are the groups different?
• Between or within subject design possible
27. Slide 28 - What am I doing?

• Low within group variability and high between group variability produces a significant effect
• High within group variability and low between group variability produces no effect
28. Slide 29 - Graph

• Ketamine
• Prozac
• High between group variability
• Mod low within group variability
• Ketamine
• Prozac
• Low between group variability
• High within group variability
• Depression severity score
• Depression severity score
29. Slide 30 - ANOVA

• What is it doing?
• It’s significant when?
• Mean and variance are both involved
30. Slide 31 - Assumptions

• Independence of Scores (scores not related within groups or between groups)
• Normal distribution-although ANOVA procedure is pretty robust and could transform variables if skewness is a big problem
• Equal sample sizes can be important, especially if not otherwise robust design
• Homogeneity of Variance (especially if unequal sample sizes)
• If really not passing the normality assumptions, & don’t want to transform would do another analysis (e.g., Kruskal-Wallis test)
31. Slide 32 - Homogeneity of Variance

• Levene’s test: SPSS default. Examines variance (Runs an ANOVA) based on deviations from the mean. Not significant is good.
• Brown-Forsythe test: Option in SPSS. Less deficiencies. Examines variance (Runs ANOVA) deviation from the group median.
• Welch test: Similar to Brown-Forsythe.
• Caveat: Brown-Forsythe & Welch not satisfactory if more than 4 groups
• Then do James’s second order method or two-stage method (Keppel), maybe
• Lack of agreement among statisticians of WHEN heterogeneity is a problem
32. Slide 33 - Reporting ANOVA

• Report this for T=test or ANOVA
• Size of each group
• Skewness & kurtosis of DV
• Test of homogeneity of variance
• Degrees of freedom
• Sum of Squares (rarely report in ANOVA)
• Mean of Squares (rarely report in ANOVA)
• F ratio (ANOVA), t-value (t-test)
• p value
• Group means (and standard deviations often)
33. Slide 34 - Follow-up Analyses 2,3,4,5… groups

• Once you know there are differences, how do you know which group is different than the others if you have more than 2?
• F test will not answer this question unless…
• If just two groups, just look at mean scores. Ex. Gender
• If more than two…
• Planned contrasts (orthogonal is best)
• Trend analyses
• Post-hoc tests: Tukey or Scheffe
• Examine means
34. Slide 35 - Planned Comparisons

• Two treatment groups vs. control group
• Parents of infants & toddlers vs. Parents of school age & high school children
• Must be orthogonal to compare treatment vs. no treatment: +1/2 standard + ½ experimental + -1 control group (the numbers rep. coefficients in formula)
• Must always sum = 0
• Planned not post-hoc
35. Slide 36 - Orthogonality Example

• Planned Comparisons OK
• Parents of infants groups plus parents of toddlers group vs. Parents of school age plus parents of teen group
• Parent of infants vs. parents of school age
• Parents of school age vs. parents of teens
• NOT OK
• Parents of school age and teens vs. parents of school age and toddlers
• No “double dipping”
36. Slide 37 - Post hoc Test options

• Tukey & Scheffe (slightly different stat methods)
• Both compare means for all groups and tell you which ones are sig. different
• Powerful & Robust analyses (don’t need to do Bonferroni’s for these)
• Will give you minimum sig. difference number
37. Slide 38 - Why not always do post-hoc?

• Method is not planned a priori
38. Slide 39 - Self test

• How is t test different than ANOVA?
• Come up with another example of an ANOVA with 3 levels of one IV
• How do you do apriori and posthoc tests when you have 3 or more groups in your IV?
• What assumptions should be tested for an ANOVA?
• There are two assumptions that if both are violated we have a bigger problem? What are they?
39. Slide 40 - What to Know from Lecture

• When to use stats
• Independent vs. paired samples
• Within/Between design
• Define terms
• Calculate a df
• Why is multiple t test bad?
• One tail vs. two tails
• What is an F ratio ?
• How to compare groups when you have 3 or more groups
• All assumptions
• Assumptions of all stats
• How to report stats
• When to use different stats for assumptions
• What do the numbers mean when reported
• F-ratio
• Follow-up options in ANOVA, when & what
40. Slide 41

• Prayer Requests and Questions?