Post by halonachos on Dec 8, 2015 10:37:00 GMT
A lot of people on this site are of that age when studies are becoming important to them, whether or not they have to read them or if they have to interpret them to include their results and findings in a paper. Unfortunately a lot of universities fail to teach how to interpret studies and fail to teach how to carefully conduct them. So I, being the helpful type, am going to give a few basics of studying studies.
1) Rarely is a scientific study 100% conclusive in it's findings: No study is going to be the final determinant in what is true and what is not true as the number of errors in a study will typically prevent that. Also it is a poor study and a poor scientists who believes that their study proves anything without any doubt.
2) Values may seem large but may not be significant: Studies will typically set an alpha value, this value represents a level of results that are expected due to random chance. A good study will usually have an alpha value of .05, which translates to 5 times out of 100 tests. Does this mean that a result has to occur only 5 times out of 100 to be declared significant? No, it means that at most 5 out of the 100 tests have been influenced by chance alone.
3) Testing for alpha values can be one tailed or two tailed: If you want to know if something is better than something then you use a one tail test, if you want to see if it's better or worse then you use a two tail test. In a two tail test the alpha value may be .05, but it is split between both ends so the alpha value is a tad bit smaller for both sides.
4) POWER: Just because a result is significant doesn't mean it is powerful. For example if a study has a significant value but only has 10 subjects then it is less powerful than a significant value with a larger number of subjects. The most powerful kind of test is a test with a low alpha value(.01>.05), a larger sample size is much better, a one tailed test is better than a two tailed test, effect size helps show the power of the effect.
Now those are the basics for seeing whether or not a result is significant and whether or not it is a powerful significance, there are many other fun terms and equations used to determine these numbers (such as confidence intervals, z-scores, standard deviation, sample deviation, etc) but that's for a statistics class.
Interpreting correlations:
Correlations never prove causation, just a probability.
In a study, correlation is depicted with "r" and can range from 1 to 0, with 1 being 100% correlation and 0 being no correlation.
Correlations, such as those above, can show an incredible relationship and can be interpreted however a reader feels like interpreting them. For example, if true, then autism has an effect on organic food sales as well as Jim Carrey movies being made. I like to imagine that Jim Carrey movies cause autism, because he's a jackass when it comes to autism and vaccines. However, this is of course not an explanation for the rise and cause of autism but a 94% correlation is nothing to shake a stick at.
Generalizations.
This is one of the few times where a generalization is good.When looking at a study it is important to note what kind of study was done and who were the people responding to it. For example one thing that drew ire for me was someone posting studies showing that solitary confinement was bad for everyone.
However, the person failed to consider whether or not generalization could be applied in this situation. As an example many monks and certain other religious members see solitary confinement and isolation as a way to achieve enlightenment and these individuals do not come out any worse for wear as opposed to individuals in a prison system. Then again, even if the study conducted compared people in a prison system to others in a similar system and in solitary, would that be able to be generalized to the general public?
One thing most people do not realize is that not everyone is the same nor does every group or culture share similar values and habits that could impact a study. For example a study comprised of only a prison population may not represent the population as a whole nor would a US prison population represent a Syrian prison population. Consider whether or not a study can be generalized to you, your neighborhood, your country, or region.
We also know that some studies like to compare countries to countries, especially when it comes to education. We also know that these tend to place the US pretty low compared to the rest of the world. However, people fail to see the differences in the demographics of those being tested. For example, the US places under the UK, Germany, and many other similar nations... until we factor in demographics. Once demographics are taken into account and the results are corrected for it the US performs better than Germany, the UK, and France in reading and equally well as the UK in math. This is because low socioeconomic groups are over represented in the US test takers, and from previous studies socioeconomic level was shown to be a significant factor in determining education outcomes. Some studies even go so far as to conclude that race and culture related to it may be an important factor!
The next time you look at a study consider whether or not; the results are significant, the confidence value is high or low, the results are powerful, or the results can be applied to a wide spectrum of the population or a narrow spectrum. Of course always remember that correlation does not prove causation and that celebrities are usually not good sources of information and some may even cause Autism, like Jim Carrey.
Edit: Confidence values are 1-alpha value. If your alpha value is .05 then you have a .95 confidence value. The larger the alpha value, the less confidence you have that your results are not due to chance.
1) Rarely is a scientific study 100% conclusive in it's findings: No study is going to be the final determinant in what is true and what is not true as the number of errors in a study will typically prevent that. Also it is a poor study and a poor scientists who believes that their study proves anything without any doubt.
2) Values may seem large but may not be significant: Studies will typically set an alpha value, this value represents a level of results that are expected due to random chance. A good study will usually have an alpha value of .05, which translates to 5 times out of 100 tests. Does this mean that a result has to occur only 5 times out of 100 to be declared significant? No, it means that at most 5 out of the 100 tests have been influenced by chance alone.
3) Testing for alpha values can be one tailed or two tailed: If you want to know if something is better than something then you use a one tail test, if you want to see if it's better or worse then you use a two tail test. In a two tail test the alpha value may be .05, but it is split between both ends so the alpha value is a tad bit smaller for both sides.
4) POWER: Just because a result is significant doesn't mean it is powerful. For example if a study has a significant value but only has 10 subjects then it is less powerful than a significant value with a larger number of subjects. The most powerful kind of test is a test with a low alpha value(.01>.05), a larger sample size is much better, a one tailed test is better than a two tailed test, effect size helps show the power of the effect.
Now those are the basics for seeing whether or not a result is significant and whether or not it is a powerful significance, there are many other fun terms and equations used to determine these numbers (such as confidence intervals, z-scores, standard deviation, sample deviation, etc) but that's for a statistics class.
Interpreting correlations:
Correlations never prove causation, just a probability.
In a study, correlation is depicted with "r" and can range from 1 to 0, with 1 being 100% correlation and 0 being no correlation.
Correlations, such as those above, can show an incredible relationship and can be interpreted however a reader feels like interpreting them. For example, if true, then autism has an effect on organic food sales as well as Jim Carrey movies being made. I like to imagine that Jim Carrey movies cause autism, because he's a jackass when it comes to autism and vaccines. However, this is of course not an explanation for the rise and cause of autism but a 94% correlation is nothing to shake a stick at.
Generalizations.
This is one of the few times where a generalization is good.When looking at a study it is important to note what kind of study was done and who were the people responding to it. For example one thing that drew ire for me was someone posting studies showing that solitary confinement was bad for everyone.
However, the person failed to consider whether or not generalization could be applied in this situation. As an example many monks and certain other religious members see solitary confinement and isolation as a way to achieve enlightenment and these individuals do not come out any worse for wear as opposed to individuals in a prison system. Then again, even if the study conducted compared people in a prison system to others in a similar system and in solitary, would that be able to be generalized to the general public?
One thing most people do not realize is that not everyone is the same nor does every group or culture share similar values and habits that could impact a study. For example a study comprised of only a prison population may not represent the population as a whole nor would a US prison population represent a Syrian prison population. Consider whether or not a study can be generalized to you, your neighborhood, your country, or region.
We also know that some studies like to compare countries to countries, especially when it comes to education. We also know that these tend to place the US pretty low compared to the rest of the world. However, people fail to see the differences in the demographics of those being tested. For example, the US places under the UK, Germany, and many other similar nations... until we factor in demographics. Once demographics are taken into account and the results are corrected for it the US performs better than Germany, the UK, and France in reading and equally well as the UK in math. This is because low socioeconomic groups are over represented in the US test takers, and from previous studies socioeconomic level was shown to be a significant factor in determining education outcomes. Some studies even go so far as to conclude that race and culture related to it may be an important factor!
The next time you look at a study consider whether or not; the results are significant, the confidence value is high or low, the results are powerful, or the results can be applied to a wide spectrum of the population or a narrow spectrum. Of course always remember that correlation does not prove causation and that celebrities are usually not good sources of information and some may even cause Autism, like Jim Carrey.
Edit: Confidence values are 1-alpha value. If your alpha value is .05 then you have a .95 confidence value. The larger the alpha value, the less confidence you have that your results are not due to chance.