Originally posted by Deadline
You might be right, but you're probably lying.
well, it is actually really simple
all of the data you have presented is based on what is called a "null-hypothesis-significance-test" (NHST). in an NHST, you compare the data you obtain through an experiment to a "null" value that represents chance.
So, in most studies data is compared to a null hypothesis (H0) of 0, meaning there is no effect. In the cases you presented, instead of testing against no effect, they tested against a chance null, H0=25%. This is fine. Now, say I have results where my observed mean (u1) is 30. This means there is an absolute difference of 5% between my observed mean hit rate, and the assumed null chance hit rate (u0):
u1-u0=5
However, in my test, not every subject had a hit rate of exactly 30. Some were above and others were below. So, we then take an average of how much people are different from 30, known as the standard deviation, SD.
There are other things here, like probability distributions, etc, but for the sake of simplicity, just trust me that as a mathematical law, you can say what percentage of subject scores will fall within any number of standard deviations from the observed mean.
So, lets say that the SD in these results was 3. So, it is known that ~65% of all subjects will fall within one standard deviation of the mean. So, our 65% confidence interval would be 27-33.
now, there is something called an alpha value (a). alpha can be seen as, ummmmm, lets say the "opposite" of a confidence interval. So, the tradition in science is to use an alpha value of .05, or 5%, meaning that the typical CI that is used in experiments is 95%. basically, the CI representing all the values which are represented by your observed data, and alpha representing those that are outside.
a 95% confidence interval is all data within 2 standard deviations of the observed mean, so in this case, 30 +/- 6, or 24-36.
because 25 falls within this range, we cannot say it is statistically different from our null mean. Based on the variance in the data, the score of 25% would not be unexpected, therefore, the result is non significant. Let me know if this doesn't make sense.
Also, about the H0=25%. That null isn't the most appropriate, as in many cases, the probability of something occurring is based not simply on a raw percentage estimate, but can be influenced by previous trials and other mundane things. Think about it like this: I have a deck of cards and I say "deadline, predict the suit of the next card". Now, the raw probability is 1/4, or 25%. However, as we go through the deck, the suit of the previous cards can influence how probable it is another suit would come up. So, if you see a string of hearts, you as the experimental subject would then, even if subconsciously, know not to select hearts, as there are now fewer hearts in the deck than the other suits, making the probability of the other suits greater than 25%. This isn't nitpicking either. The studies I did in my undergrad had subjects distinguish between an L and a T on target objects. There was nothing important about the T or the L, but even in that case, they would often ask "why were there more Ts/Ls?". This type of probability is something our brains are intrinsically aware of, and could certainly cause difficulty in determining what a proper null percentage would be in these experiments. Another example of this is from previous stuff you showed me that said that a particular subject seemed to have a talent for remote viewing military installations. However, the studies were conducted on military bases (iirc), meaning that the context may have played a role in priming a certain type of response in the individual. However, it could be even more mundane, as cognitive biases like that could be produced from someone simply being a fan of Command and Conquer games.
Additionally, as I posted in the Atheism thread, when you apply more rigorous statistical methods, like Bayesian probability analysis, most of the significant results in psi have been seen to evaporate. There are a number of reasons for this. For one, a NHST does not tell you how likely it is that your hypothesis is true, but rather, how likely it is chance alone is responsible for your results. Studies have looked at the correlation between p-values (the probability of chance explaining your results) and true hypotheses, and found the R value to be just over .35 (extremely low), and this number drops when you restrict p-values to only those that would find significant results. (please ask if this doesn't make sense, I'm sure stats aren't as exciting to you as they are to me, but if you want to talk about double standards in science, you need to understand how stats work). These results are interesting, but really only show that, in a few experiments, the pattern of results aren't what would be expected due to chance alone.