Chapter Nine (Salkind) Significantly Significant: What It Means For You And Me Inferential Statistics We have turned a corner in the course now, and we are ready to move beyond just looking at descriptive statistics (statistics that “describe” the nature of the data set) The remaining chapters in our Salkind textbook focus on inferential statistics, the procedures used to analyze data after an experiment is completed in order to determine whether the independent variable has a significant effect (you can find that definition in your Smith and Davis textbook!) An Overview of This Chapter But in Chapter 9 (Salkind), we are first going to focus on better understanding the nature of significance testing and see how it applies to inferential statistics. Second, we delve into the errors (Type I and Type II) we can make when determining when (or if) two study conditions differ Finally, we’ll focus on choosing the most appropriate statistical test to use when running inferential statistics Good News! Here’s the good news … – No math formulas! Part One The Concept Of Significance The Concept of Significance We’ve briefly talked about the idea of significance, but in this chapter we will look more specifically at the idea of statistical significance Example study: does the temperature of the room influence exam performance? Randomly assign half of my students to a hot room (90 degrees), and the other half to a room temperature room (72 degrees). IV And DV The Independent variable is room temperature – the hot room is our “experimental” condition and the room temp room our “control” condition Our dependent variable is their exam scores Is It Statistically Significant? Let’s say that at the end of the study, we found that people in the room temperature room did better on the exam (the average score was an 86%) than the hot room (the average score was 75%). – This could suggest that heat makes people do worse. But, is this difference statistically significant? – Is this difference between exam scores just a fluke? – How likely were these findings? This is what we’ll be talking about today. Hypotheses In every study, we have two hypotheses: – The null hypothesis – The alternative hypothesis Remember, these can either be directional or non-directional. – A directional hypothesis has a specific expectation for the direction of effects. – A non-directional hypothesis does not have a specific expectation for the direction of effects. The Null Hypothesis The null hypothesis: the idea that there is NO affect of our IV on our DV. Our null hypothesis would be that exam performance would be the same in both rooms. The difference is not big enough to conclude that room temperature influences exam scores. Average exam score of the hot room = average exam score of the cold room. The outcome we expected did NOT happen. This means that we did not find what we thought we would find in our study. The Alternative Hypothesis Alternative hypothesis: The IV does influence the DV (that temperature influences exam performance). Students in the hot room will do worse on the exam than those in a cold room. – This is what we hope to happen in the study. – As you can see, this prediction focuses on a specific direction: exam scores will be lower hot room group. – As such, this is a directional hypothesis. – But consider a non-directional hypothesis … Non-Directional Hypotheses Recall that a non-directional hypothesis doesn’t focus on a specific direction … – “Exam scores will be significantly different (higher or lower) in the hot room than in the room temperature room” – With a non-directional hypothesis, we are still expecting that temperature influences exam scores, but we are not stating whether we think the heat will increase or decrease scores. We are just stating that they will be different. In the End At the end of our study, we will either conclude that the null hypothesis is true or that the alternative hypothesis is true. Let’s think about it this way…. For my room temp study, there are three possibilities that could happen at the end of my study: 1) There is no difference between hot and room temp exam scores (they are equal). 2) People in the hot room do better than people in the room temp room. 3) People in the hot room do worse than people in the room temp room. If The Hypothesis Is Non-Directional: 1) 2) 3) There is no difference between hot and room temp exam scores (they are equal). → Null Hypothesis People in the hot room do better than people in the room temp room. → Alternative hypothesis People in the hot room do worse than people in the room temp room. → Alternative hypothesis If we find 1, then we support the null. If we find 2 or 3, then we support alternative. If The Hypothesis Is Directional: 1) 2) 3) There is no difference between hot and room temp exam scores (they are equal). → Null Hypothesis People in the hot room do better than people in the room temp room. → Null hypothesis People in the hot room do worse than people in the room temp room. → Alternative hypothesis If we find 3, then we support alternative. But if we find 1 or 2, then we support the null. That is, we will only support the alternative hypothesis if the specific direction of effects is true! So, even if the people in the hot room do A LOT better than people in the cold room, we would still conclude that the null hypothesis was true for this study. Summary • • In short, the null hypothesis includes all the possible outcomes that we do not expect to happen in our study. The alternative hypothesis includes all the possible outcomes that we do expect to happen in our study. Let’s Practice! Let’s practice these concepts of null, alternative, and directional/non-directional hypotheses. • Keep in mind, that for the next few questions, there may be more than one right answer! • Be sure to read the question very carefully. • Pop-Quiz 1: Quiz Yourself A researcher wants to test if a new energy bar increases the speed at which participants run a marathon. She compares a group that ate the new bar (the experimental group) to a group that did not eat the bar (the control group) what is the alternative hypothesis? A). The group that ate the energy bar will run the marathon faster than the group that did not. B). The group that ate the energy bar will run the marathon slower than the group that did not. C). The group that ate the energy bar will run the marathon at the same speed as the group that did not. D). There is not enough information to answer this question. Answer 1: A A researcher wants to test if a new energy bar increases the speed at which participants run a marathon. She compares a group that ate the new bar (the experimental group) to a group that did not eat the bar (the control group) what is the alternative hypothesis? A). The group that ate the energy bar will run the marathon faster than the group that did not. → This is the alternative hypothesis (it is directional because she only wants to test if the energy bar INCREASES speed). B). The group that ate the energy bar will run the marathon slower than the group that did not. C). The group that ate the energy bar will run the marathon at the same speed as the group that did not. D). There is not enough information to answer this question. Pop-Quiz 2: Quiz Yourself A researcher wants to test if a new energy bar increases the speed at which participants run a marathon. She compares a group that ate the new bar (the experimental group) to a group that did not eat the bar (the control group) what is the null hypothesis? A). The group that ate the energy bar will run the marathon faster than the group that did not. B). The group that ate the energy bar will run the marathon slower than the group that did not. C). The group that ate the energy bar will run the marathon at the same speed as the group that did not. D). There is not enough information to answer this question. Answer 2: B & C A researcher wants to test if a new energy bar increases the speed at which participants run a marathon. She compares a group that ate the new bar (the experimental group) to a group that did not eat the bar (the control group) what is the null hypothesis? B). The group that ate the energy bar will run the marathon slower than the group that did not. C). The group that ate the energy bar will run the marathon at the same speed as the group that did not. → B & C would both be part of the null hypothesis here, because it’s a directional study so the null hypothesis includes all the possibilities that are NOT part of the expected outcome. So, even if the bar makes them run MUCH slower (B), the conclusion would be that the null hypothesis is true. Warning! Read the next two questions very carefully! • Notice that something about the study design has changed from the previous two questions… • Pop-Quiz 3: Quiz Yourself A researcher wants to test if a new energy bar influences the speed at which participants run a marathon. She compares a group that ate the new bar (the experimental group) to a group that did not eat the bar (the control group) what is the alternative hypothesis? A). The group that ate the energy bar will run the marathon faster than the group that did not. B). The group that ate the energy bar will run the marathon slower than the group that did not. C). The group that ate the energy bar will run the marathon at the same speed as the group that did not. D). There is not enough information to answer this question. Answer 3: A & B A researcher wants to test if a new energy bar influences the speed at which participants run a marathon. She compares a group that ate the new bar (the experimental group) to a group that did not eat the bar (the control group) what is the alternative hypothesis? A). The group that ate the energy bar will run the marathon faster than the group that did not. B). The group that ate the energy bar will run the marathon slower than the group that did not. → Both A & B would be the alternative hypothesis here because now the researcher is merely seeing if the energy bar influences the speed. She is testing whether it could make them run faster or slower, so it is non-directional. . Pop-Quiz 4: Quiz Yourself A researcher wants to test if a new energy bar influences the speed at which participants run a marathon. She compares a group that ate the new bar (the experimental group) to a group that did not eat the bar (the control group) what is the null hypothesis? A). The group that ate the energy bar will run the marathon faster than the group that did not. B). The group that ate the energy bar will run the marathon slower than the group that did not. C). The group that ate the energy bar will run the marathon at the same speed as the group that did not. D). There is not enough information to answer this question. Answer 4: C A researcher wants to test if a new energy bar influences the speed at which participants run a marathon. She compares a group that ate the new bar (the experimental group) to a group that did not eat the bar (the control group) what is the null hypothesis? A). The group that ate the energy bar will run the marathon faster than the group that did not. B). The group that ate the energy bar will run the marathon slower than the group that did not. C). The group that ate the energy bar will run the marathon at the same speed as the group that did not. → Now that this is a non-directional hypothesis, the null is simply that they are the same. In other words, that the energy bar has NO influence on speed. Does The IV Influence The DV? What we want to know: Is the difference between the groups significant? Does the IV influence the DV? – All other factors (we hope) are controlled, so the only thing that influences our dependent variable is our independent variable. – Well … Sources Of Error – Maybe the difference we observed between the exam scores was due to chance! 1). Maybe the people in the cold room were better prepared! 2).Maybe the people in the cold room took the exam after the people in the hot room, and they shared test questions with the people in the cold room, so they did better! 3). Maybe the people in the cold room got to take the exam in the afternoon, but the people in the hot room had to take it in the morning… What Are The Odds? It could be a million other things, just something OTHER THAN the IV. There is ALWAYS the possibility of error in research. What is the probability that I’ve observed these differences between my cold room and hot room, just by chance? Error Happens In our last chapter (Chapter 8, Salkind), we noted that extreme scores (scores at the upper or lower end of the normal curve) tend to be rare. But we must be mindful that extreme scores CAN and DO happen – Similarly, extreme differences between groups could happen just by chance. This error is problematic, as we can’t be sure if it is our IV or the error that affects our DVs Chance or Random Error I can’t emphasize this enough: There is ALWAYS a possibility that chance is responsible for differences in the DV. – If we found that the difference between the hot room and the room temp room was statistically significant, what we are saying is that the difference is so big that it would only happen by chance 5% of the time. – In other words, if we didn’t manipulate anything, and simply gave two groups of students two equal exams in the same room, how often would we see a difference between scores as big as the one we found in our study?  If it’s less than 5% of the time, then it’s significant! Statistical Significance We finally get to the formal definition of statistical significance: – The degree of risk you are willing to take that you will reject the null hypothesis when it is actually true (that you will say that there is an effect when there is not really one) We say there is a significant difference if the odds that our results happened by chance or random error is less than 5%. We use the normal curve to understand whether our findings are “big enough” to probably not be due to chance or random error. A 5% Possibility of Error (p = .05) In research, we allow 5% error in our designs. That is, we say “95 out of 100 times, the outcome we predict will occur” – This is our p < .05 philosophy, which is a risk level that the researcher defines at the outset of the study. – We’re okay with a 5% possibility that our results are a fluke. Why .05? Why not .01, or .10? Tradition sets it at 5%! Risk In significance testing, we focus on the idea of risk: That is, at what level do we set our risk of being incorrect in our outcome interpretation? How Do We Choose? Remember that we pit the null hypothesis (which states that our conditions do not differ) against our alternative hypothesis (our conditions do differ). – Which do we accept and which do we reject? Accepting The Null Hypothesis – A. Sometimes the null hypothesis is true (your samples do not differ significantly, thus your experiment failed) so you retain (keep) the null hypothesis This can be very frustrating, but it is a good decision Sometimes, people refer to this as failing to reject the null (this is the same as saying you are accepting the null) Rejecting The Null – B. Sometimes the null hypothesis is false (the samples differ and you succeeded!), so you can reject the null hypothesis This is what we usually want in research, letting us do our happy dance! Error! – C. Our next two conclusions are problematic. Sometimes you make a mistake in your results interpretation, and either conclude that 1) an effect occurred when it didn’t (Type I error) OR 2) say an effect did not occur when it really did (Type II error) Correct or Incorrect Conclusions? This gives us two correct conclusions – We reject the null hypothesis when we should reject – We retain the null hypothesis when we should retain It also gives us two incorrect conclusions – We reject the null hypothesis when we should retain Type I Error (a “false-alarm”) – We retain the null hypothesis when we should reject Type II Error (a “miss”) Error Table The World’s Most Important Table (For This Semester Only) Decreasing one actually increases the other…. Like The Boy Who Cried Wolf Crying wolf (saying there’s a wolf, when there isn’t) Missing a wolf that’s there Or A False Positive or False Negative False Positive False Negative Let’s Practice! Let’s practice identifying the null, alternative, Type I and Type II errors from research examples. Example Study Set Up Example study idea: Does aerobic exercise lower blood pressure? Randomly assign one group to exercise 3 times per week for a month, and a second group to not exercise at all. Before you move on to the next few slides, take some time on your own and try to identify the following things for this study: – Null Hypothesis – Alternative Hypothesis – Type I Error – Type II Error Null and Alternative Null hypothesis (H0): People who exercised will have the same blood pressure or higher blood pressure than those who did not exercise. – Exercise will have no impact on blood pressure or will increase blood pressure. Alternative hypothesis (H1): People who exercised will have lower blood pressure than those who did not. – Exercise will decrease blood pressure. – This is a directional hypothesis. Type I and Type II Error Type I Error: Researchers conclude that exercise decreases blood pressure, when in reality it does not! Researchers conclude the Alternative hypothesis (H1) is true, when in reality the Null hypothesis (H0) is true Type II Error: Researchers conclude that exercise does not decrease blood pressure, when in reality it does! Researchers conclude the Null hypothesis (H0) is true, when in reality the Alternative hypothesis (H1) is true Example Study 2 Set Up Example study idea: How does students’ performance differ in online versus face-to-face classes? Compare final exam scores between an online and face to face class. Before you move on to the next few slides, take some time on your own and try to identify the following things for this study: – Null Hypothesis – Alternative Hypothesis – Type I Error – Type II Error Null and Alternative 2 Null hypothesis (H0): There will be no difference between the online and face to face classes on the final exam. Alternative hypothesis (H1): There will be a difference between the online and face to face classes on the final exam. – The exam scores will either be higher or lower in the online class. – This is a non-directional hypothesis. Type I and Type II Error 2 Type I Error: Researchers conclude that there is a difference between the online and face-to-face exam scores, when in reality, there is not. Researchers conclude the Alternative hypothesis (H1) is true, when in reality the Null hypothesis (H0) is true Type II Error: Researchers conclude that the face-to-face and online class exam scores are not different, when in reality, they are. Researchers conclude the Null hypothesis (H0) is true, when in reality the Alternative hypothesis (H1) is true Alpha Level Our chances of type I error is set at .05 (this is known as the alpha level). – The chance that we will say there is a difference between our groups when, in reality, there is NOT. – The chance that we will observe a difference between our two groups, when in reality, there is not a difference between them. We observed this difference merely due to chance, not due to the independent variable. Why 5% – Why not 10% or 15%? Limiting error to 5% makes it harder to find significance. If it’s too easy to find significance, the chances of having a type 1 error goes up (of saying that we found an effect when we really didn’t). If we increase the threshold to higher than 5%, it means that it will be easier to detect effects that are not real! Trade Off Between Type I and Type II Type II Error – We also want to avoid claiming there is no significance if there is significance. So, why not 1% or .001%  Reducing the alpha makes it harder to find an effect, even if there is one. If we are too strict in setting our allowable error, we might make a Type II error in concluding that there are no differences when there really are differences Of course, study results can get below .05 error (.01 or .001 are common to see). It’s okay, but .05 is adequate P Values In Research Articles In journals, you will often see the .05 p level, but .01 and .001 are also common – The .01 indicates that there is a 1% chance you will reject the null hypothesis when you should retain it (you say there were differences when the groups were, in reality, the same) – The phrase “p < .01” (or p < .05) reads as, “The probability of observing that outcome is less than .01 (or .05)” – SPSS can give you more exact p values (p = .023). This is much more precise than p < .05, but the rounded version often makes it way into research article results sections! Population versus Samples – We’ve noted previously that we typically run studies using a sample (rather than a population). Do you recall why? When it comes to Type I and Type II errors, we need to see how error impacts sample / population decisions In research, we have a bit more control over Type I errors. After all, we control that error by setting our p value (we usually set it to p < .05) Type II errors, by contrast, decrease as our sample size increases. Characteristics of our sample better mimic the population when we have larger samples! Part Two Significance Versus Meaningfulness Is This Meaningful? McClellan and Woods (2001) found that it took salesclerks significantly longer to help hearing impaired customers than hearing customers. – But is that significance meaningful? – Imagine that after reading the results, the store manager decides to put salesclerks through an intensive sensitivity training program that takes time and costs money (hiring a seminar speaker, paying employees for overtime work, etc.) Are the study results worth all of this extra training? Maybe! But we’ll need some more research to know. Significance Versus Meaningfulness Just because a statistically significant result occurs doesn’t mean we should quickly alter our lives to align with the result – 1. First, one significant result is nice, but without replication you will never know if the result is reliable or just an anomaly – 2. Second, if a study lacks validity (it’s not really measuring what it purports to measure), the results are meaningless – 3. Third, the costs of changing based on a study may not be worthwhile – sensitivity training may cost more than it’s worth – 4. We test hypotheses. We do not prove them! – 5. Supporting the null hypothesis is not always a bad thing Can Something Be Sort Of Significant? Finally (and I know I am being a bit repetitive), our p value is based on research tradition, and thus it is inherently subjective in terms of acceptability – If p = .05 is significant, why is p = .051 not significant? Can’t we say that p = .051 trends towards significance? Sure! But that’s kind of like cheating. We set the threshold at .05. In fact, you might see authors discuss “significant trends” in their publications. Just be aware of the subjectivity aspect of the p < .05, even in science Significance Versus Meaningfulness Also be aware that there is a publication bias in that studies that are not significant may not get published. Yet retaining the null hypothesis (the Pause Problem!) might actually be good in some studies. Maybe you WANT to retain the null! Part Three In Introduction To Inferential Statistics Introduction To Inferential Statistics There’s that phrase again … Inferential statistics! – While descriptive statistics help describe the characteristics of a sample, inferential statistics let us infer something about the population based on the sample’s characteristics We will look more at how inferential statistics work in a study in later chapters (we’ll look at both t-Tests and ANOVAs, both of which are inferential tests) For now, we are going to look at populations versus samples Sample and Population In order to be of any use, characteristics of a sample should mimic the characteristics of a population – Lacking similarity (or representativeness) to a population, any results that we get from a sample in our study will be meaningless when we later compare it to the population – Conversely, if our study does a good job in mimicking the population, we are in a better position to infer that the results we get based on the sample are the same results we would get if we ran the study on the whole population Find A Sample 1. Our first task is to draw a sample that represents the larger population – The pool of participants in our sample should look a lot like the pool of participants in our population. – If the population and sample of people you are interested in studying should be similar in terms of age, sex, ethnicity, and other relevant characteristics. 2. Our second task is to introduce our independent variable 3. Our third task … run the study, analyze the results! Reach Conclusions 4. Our final task is to reach our conclusions – If our two samples differ significantly, can we say that the same finings would extend to the population? If it is representative, we can have more confidence in this assertion Selecting The Right Test  How To Select What Test To Use Selecting a test is a tough one. We have several inferential statistical tests that we can use. For example, there are: – t-Tests (correlation coefficient, dependent and independent) – Regression, factor analysis, canonical analyses – Analyses of Variance (ANOVA – correlated / independent) – Multivariate Analysis of Variance (MANOVA) – Analysis of Covariance (ANCOVA) – And the list goes on. But which one do we use and when? Your book (Salkind) gives us a handy chart (Figure 9.1) Questions To Ask (1) The handy chart gives you an indication of which test is best. It comes down to answering a few key questions – 1. Are you examining relationships between variables? If yes, you have to figure out how many variables you are dealing with If two variables, correlation coefficients are good If two +, use regression, factor analysis, or canonical correlations Questions To Ask (2) – 2. Are you examining differences between groups? If yes, are your participants tested more than once? If yes and you are dealing with only two groups, a tTest for dependent samples is best If yes and you are dealing with more than two groups, an ANOVA (repeated measures) is best If no and you are dealing with only two groups, a tTest for independent samples is best If no and you are dealing with more than two groups, an independent ANOVA is best Part Four An Introduction To Tests of Significance Introduction To Tests Of Significance  How A Tests Of Significance Works: The Plan There is an important relationship between the null hypothesis, the statistic you choose to test that hypothesis, the nature of the distribution (the normal curve) that is associated with each statistical test, the sample, and whether characteristics of the sample are different than you would expect by chance. Introduction To The 8 Steps Wow, quite a mouthful there. Let’s break this down more. In the remaining Salkind chapters that we will cover this first semester (which include Chapters 10, 11, 12, 13, and 15), we are going to come across a series of eight critical steps that we must take that compare the statistic that we obtained to the statistic that we need to overcome in order to say the test is statistically significant Don’t Forget! My advice? Memorize these eight steps, as we will come back to them time and time again. Step 1 (of 8) Step 1. State the null hypothesis. – This is the easiest step, right? We state that (µc = µt) In other words, we say that the control sample (µc) does not differ from the treatment sample (µt) – Contrast this with the alternative hypothesis (µc ≠ µt) Notice the ≠ sign: two samples are not equal (differ) This is a non-directional hypothesis We could also say that either (µc < µt) or that (µc > µt) The signs < or > indicate a directional hypothesis Step 2 (of 8) Step 2. Set the level of risk (the level of significance or Type I error) associated with the null hypothesis – This is what we talked about earlier in this chapter. Do we want our p < .05, p < .01, p < .001? Do we want to allow more chance in our interpretation, like p < .10? – In psychology, p < .05 is customary Step 3 (of 8) Step 3. Select the appropriate test statistic – Go back and see our chart (or see Salkind Figure 9.1) Do you need a t-Test, ANOVA, regression, etc. Choose! Step 4 (of 8) Step 4. Compute the test statistic value – The test statistic value (or “obtained value”) is the result of a specific statistical test. Remember when we computed z Scores? The z Score itself is one of many “obtained values”, though we also get “obtained values” for t-Tests, ANOVA tests (called F scores), and others. So what do you do with the obtained value? Well … Step 5 (of 8) Step 5. Determine the value needed for rejection of the null – We compare our “obtained value” with the “critical value” in a statistics table. If the “obtained value” exceeds the “critical value”, then our test is significant. When we looked at the z Score, we focused on the table values associated with those z Scores (Appendix B1, Salkind). These tabled values are the “critical values”, or the scores that your obtained value needs to surpass if the alternative hypothesis is true – One more thing about step 5 … How To Find Critical Values Step 5. Determine the value needed for rejection of the null – Other tests (like the t-Test) have similar tables. When we cover the t-Test in chapters 11 and 12, we will have critical value for the t-Test (The ANOVA has a table of values as well, which we will cover more next semester) Step 6 (of 8) Step 6. Compare the obtained value to the critical value – Okay, so we have our obtained value (like our computed z Score). We also have our critical value from the table. In Step 6, we simply compare our obtained value to the critical value – Again, t-Tests and ANOVAs work the same way. We compare our obtained value (t-Test or F test score) and compare it to the tabled “critical value” for each statistic Step 7 (of 8) Step 7. Decision: Reject the Null Hypothesis – If your “obtained value” exceeds the “critical value”, then you reject the null hypothesis and conclude that the alternative hypothesis is the best explanation If you have a t-Test obtained value of 1.80, you compare that to the critical value needed to say the score is significant at the p < .05 level. In this case, the critical value might be 1.76. Since 1.80 exceeds (is above) 1.76, we conclude that the t-Test outcome was significant Step 8 (of 8) Step 8. Decision: Retain the Null Hypothesis – If your “obtained value” does not exceed the “critical value”, then you retain the null hypothesis and conclude that the alternative hypothesis is the best explanation If you have a t-Test obtained value of 1.70, you compare that to the critical value needed to say the score is significant at the p < .05 level. In this case, the critical value might be 1.76. Since 1.70 does not exceed (is less than) 1.76, we conclude that the t-Test outcome was not significant Using Computer Software (SPSS) Steps 1 through 8 – The nice thing about SPSS is that it carries out a lot of these steps for you (especially steps 4, 5, and 6) – Even with a computer, though, you need to interpret the outcome. That is, you need to choose whether your data falls into step 7 (reject the null) or step 8 (retain the null) – As I said, memorize these eight steps. It will serve you well in the rest of the chapters this (and next!) semester Pop-Quiz 5: Quiz Yourself In order to determine whether or not you will reject the null hypothesis, the obtained statistic value must be compared against the ___________. A). Critical value B). Significance level C). z Score D). p value Answer 5: A In order to determine whether or not you will reject the null hypothesis, the obtained statistic value must be compared against the ___________. A). Critical value B). Significance level C). z Score D). p value It’s All About The Normal Curve Here’s The Picture That’s Worth A Thousand Words Now that you know the eight steps, let’s return for a moment to the normal curve to see how the critical and obtained values work in relation to determining significance Do you recall the normal curve? Well … Introduction To Tests Of Significance The Normal Curve and Critical Values – 1. The entire curve represents ALL possible outcomes regarding the null and alternative hypotheses (all possible obtained values) – 2. The critical value is the line on the left side of the curve. Any score to the right of that curve is extremely rare. In fact, it is so rare that we can conclude that any obtained value that falls to the right side of that line is not due to chance alone, but rather some other factor (our IV!) Introduction To Tests Of Significance Scores rarely fall in this region When To Retain The Null – 3. If the obtained value is to the left of (BIGGER THAN) that critical line, then we retain the null hypothesis. That is, any outcome or score that falls in this area is likely due to chance (not our IV) … Introduction To Tests Of Significance Scores often fall in this region To Review:  When the obtained value is bigger than the critical value, this means that we can sure that our results only happen 5% of the time at random. We can reject the null hypothesis. There is a 5% chance that our results are just a fluke.  This means it is statistically significant. We’re 5% sure we didn’t commit a type I error and state that our findings were significant when they were not! The critical value represents the point on the normal curve at which obtained values happen less than 5% of the time. Question 1 Scenario A. A researcher is interested in comparing the effectiveness of different forms of social media advertising. She has company A (Vance Refrigeration) create and deploy targeted ads, and she has Company 2 (Dunder Mifflin Paper Company) engage in ads via endorsements from relevant social media influencers. She then compares the sales rates of the two companies during the two ad campaigns. What is the null hypothesis for this scenario? -H0: sales rates at company 1 < sales rates at company 2 -H0: sales rates at company 1 ≠sales rates at company 2 -H0: sales rates at company 1 = sales rates a company 2 -H0: sales rates at company 1 > sales rates at company 2 Question 2 Scenario A. A researcher is interested in comparing the effectiveness of different forms of social media advertising. She has company A (Vance Refrigeration) create and deploy targeted ads, and she has Company 2 (Dunder Mifflin Paper Company) engage in ads via endorsements from relevant social media influencers. She then compares the sales rates of the two companies during the two ad campaigns. What is the alternative hypothesis for this scenario? Pay attention to whether it is directional or non-directional -H1: sales rates at company 1 < sales rates at company 2 -H1: sales rates at company 1 = sales rates at company 2 -H1: sales rates at company 1 > sales rates at company 2 -H1: sales rates at company 1 ≥ sales rates at company 2 -H1: sales rates at company 1 ≤ sales rates at company 2 -H1: sales rates at company 1 ≠sales rates at company 2 Question 3 Scenario A. A researcher is interested in comparing the effectiveness of different forms of social media advertising. She has company A (Vance Refrigeration) create and deploy targeted ads, and she has Company 2 (Dunder Mifflin Paper Company) engage in ads via endorsements from relevant social media influencers. She then compares the sales rates of the two companies during the two ad campaigns. What would be a type I error for this scenario? See the example question for guidance! Question 4 Scenario A. A researcher is interested in comparing the effectiveness of different forms of social media advertising. She has company A (Vance Refrigeration) create and deploy targeted ads, and she has Company 2 (Dunder Mifflin Paper Company) engage in ads via endorsements from relevant social media influencers. She then compares the sales rates of the two companies during the two ad campaigns. What would be a type II error for this scenario? Question 5 Scenario B. A pediatrician is interested in whether deep breathing can alleviate the distress that children experience when receiving routine vaccines during wellness checks. She randomly assigns her 2 year old patients to either receive brief instructions on how to use deep breathing techniques or to receive no instruction prior to receiving routine vaccines. She expects that the children who receive the deep breathing instruction will show less distress during the vaccines than the children who did not receive the instructions. Is this a directional or non-directional hypothesis? -Directional -Non-Directional Question 6 Scenario B. A pediatrician is interested in whether deep breathing can alleviate the distress that children experience when receiving routine vaccines during wellness checks. She randomly assigns her 2 year old patients to either receive brief instructions on how to use deep breathing techniques or to receive no instruction prior to receiving routine vaccines. She expects that the children who receive the deep breathing instruction will show less distress during the vaccines than the children who did not receive the instructions. What is the null hypothesis for this scenario? Question 7 Scenario B. A pediatrician is interested in whether deep breathing can alleviate the distress that children experience when receiving routine vaccines during wellness checks. She randomly assigns her 2 year old patients to either receive brief instructions on how to use deep breathing techniques or to receive no instruction prior to receiving routine vaccines. She expects that the children who receive the deep breathing instruction will show less distress during the vaccines than the children who did not receive the instructions. What is the alternative hypothesis for this scenario? Question 8 Scenario B. A pediatrician is interested in whether deep breathing can alleviate the distress that children experience when receiving routine vaccines during wellness checks. She randomly assigns her 2 year old patients to either receive brief instructions on how to use deep breathing techniques or to receive no instruction prior to receiving routine vaccines. She expects that the children who receive the deep breathing instruction will show less distress during the vaccines than the children who did not receive the instructions. What would be a type I error for this scenario? T-he pediatrician concludes that deep breathing does not work to alleviate distress, when in reality the distress levels were the same for the children who did and did not receive the deep breathing instruction. -The pediatrician concludes that deep breathing works to alleviate distress, when in reality it did not work. -The pediatrician concludes that deep breathing works to alleviate distress, when in reality the children who had the deep breathing instruction were less distressed than the children who did not. -The pediatrician concludes that deep breathing does not work to alleviate distress, when in reality it does work to alleviate distress. Question 9 Scenario B. A pediatrician is interested in whether deep breathing can alleviate the distress that children experience when receiving routine vaccines during wellness checks. She randomly assigns her 2 year old patients to either receive brief instructions on how to use deep breathing techniques or to receive no instruction prior to receiving routine vaccines. She expects that the children who receive the deep breathing instruction will show less distress during the vaccines than the children who did not receive the instructions. What would be a type II error for this scenario? -The pediatrician concludes that deep breathing does not work to alleviate distress, when in reality the distress levels were the same for the children who did and did not receive the deep breathing instruction. -The pediatrician concludes that deep breathing works to alleviate distress, when in reality it did not work. -The pediatrician concludes that deep breathing does not work to alleviate distress, when in reality it does work to alleviate distress. -The pediatrician concludes that deep breathing works to alleviate distress, when in reality the children who had the deep breathing instruction were less distressed than the children who did not. Question 10 Scenario C. David sees an ad for a new bug spray (bugsBEEgone) that promises to prevent bug bites better than the other leading product. He decides to test this out by assigning participants into two groups: one using the new bug repellent and another using a commercially available bug repellent as a control. The study will measure the number of insect bites received by each participant during outdoor activities. His goal is to test whether bugsBEEgone repels bugs better than his old brand. Is this a directional or non-directional hypothesis? -Directional -Non-Directional Question 11 Scenario C. David sees an ad for a new bug spray (bugsBEEgone) that promises to prevent bug bites better than the other leading product. He decides to test this out by assigning participants into two groups: one using the new bug repellent and another using a commercially available bug repellent as a control. The study will measure the number of insect bites received by each participant during outdoor activities. His goal is to test whether bugsBEEgone repels bugs better than his old brand. What is the null hypothesis for this scenario? -H0: Number of bites with bugsBEEgone > Number of bites with the other brand -H0: Number of bites with bugsBEEgone < Number of bites with the other brand -H0: Number of bites with bugsBEEgone ≠ Number of bites with the other brand -H0: Number of bites with bugsBEEgone ≥ Number of bites with the other brand Question 12 Scenario C. David sees an ad for a new bug spray (bugsBEEgone) that promises to prevent bug bites better than the other leading product. He decides to test this out by assigning participants into two groups: one using the new bug repellent and another using a commercially available bug repellent as a control. The study will measure the number of insect bites received by each participant during outdoor activities. His goal is to test whether bugsBEEgone repels bugs better than his old brand. What is the alternative hypothesis for this scenario? -H1: Number of bites with bugsBEEgone ≠ Number of bites with the other brand -H1: Number of bites with bugsBEEgone = Number of bites with the other brand -H1: Number of bites with bugsBEEgone ≤ Number of bites with the other brand -H1: Number of bites with bugsBEEgone < Number of bites with the other brand -H1: Number of bites with bugsBEEgone > Number of bites with the other brand -H1: Number of bites with bugsBEEgone ≥ Number of bites with the other brand Question 13 Scenario C. David sees an ad for a new bug spray (bugsBEEgone) that promises to prevent bug bites better than the other leading product. He decides to test this out by assigning participants into two groups: one using the new bug repellent and another using a commercially available bug repellent as a control. The study will measure the number of insect bites received by each participant during outdoor activities. His goal is to test whether bugsBEEgone repels bugs better than his old brand. What would be a Type I error for this scenario? Question 14 Scenario C. David sees an ad for a new bug spray (bugsBEEgone) that promises to prevent bug bites better than the other leading product. He decides to test this out by assigning participants into two groups: one using the new bug repellent and another using a commercially available bug repellent as a control. The study will measure the number of insect bites received by each participant during outdoor activities. His goal is to test whether bugsBEEgone repels bugs better than his old brand. What would be a type II error for this scenario? Question 15 Scenario D. A researcher is interested in examining whether elementary school students’ social skills changed after the pandemic Should we test this using a directional or nondirectional hypothesis? -Non-Directional -Directional Question 16 Scenario D. A researcher is interested in examining whether elementary school students’ social skills changed after the pandemic. What is the null hypothesis for this scenario? -H0: Social skills decreased after the pandemic. -H0: Social skills did change after the pandemic. -H0: Social skills did not change after the pandemic. -H0: Social skills increased after the pandemic. -H0: Social skills did were the same or higher after the pandemic. Question 17 Scenario D. A researcher is interested in examining whether elementary school students’ social skills changed after the pandemic. What is the alternative hypothesis for this scenario? -H1: Social skills increased after the pandemic. -H1: Social skills did not change after the pandemic. -H1: Social skills did change after the pandemic. -H1: Social skills decreased after the pandemic. Question 18 Scenario D. A researcher is interested in examining whether elementary school students’ social skills changed after the pandemic. What is a type I error for this scenario? -The researcher concludes that social skills changed after the pandemic, when in reality it did not change at all. -The researcher concludes that social skills stayed the same after the pandemic, when in reality it did change. -The researcher concludes that social skills changed after the pandemic, when in reality it did change. -The researcher concludes that social skills did not change after the pandemic, when in reality it stayed the same. Question 19 Scenario D. A researcher is interested in examining whether elementary school students’ social skills changed after the pandemic. What is a type II error for this scenario? -The researcher concludes that social skills did not change after the pandemic, when in reality it stayed the same. -The researcher concludes that social skills changed after the pandemic, when in reality it did not change at all. -The researcher concludes that social skills stayed the same after the pandemic, when in reality it did change. -The researcher concludes that social skills changed after the pandemic, when in reality it did change. Question 20 In this discussion you’re going to come up with your own research scenario (like the ones above). This must be your own original study idea. You may not copy and paste or get your answers from other internet sources. I. Post the following: a. create your own “Example Experiment” similar to the questions in the Individual Assignment. This can be about any topic you want. Take a look back at the individual assignment # 9 if you want some examples. Please write your own answers to this question, copying and pasting from the internet to get the answer will not be acceptable. II. Reply the following: a. Select a group member’s post and reply by stating the Type I and Type II error for that scenario. b. State whether their hypothesis is directional or non-directional.
Purchase answer to see full attachment