Click HERE to see this thread with all of the graphics, features, and links.

Two interesting articles.

http://www.ics.uci.edu/~jutts/Sweden.pdf



How are anomalous cognition (ac) - remote viewing and
ganzfeld - results different from aspirin results?
�� If same standard applied, ac results are much stronger.
�� The aspirin studies had more opportunity for fraud and
experimenter effects than did the ac studies.
�� The aspirin studies were at least as frequently funded and
conducted by those with a vested interest in the outcome.
�� Both used heterogeneous methods and participants.



http://bigthink.com/ideas/24951

Whats really interesting is this and this sort of thing is mentioned in the first pdf.

By chance, then, the students should have been right exactly half the time. Instead, they predicted correctly just over 53 percent of the time. Not a big difference, but, as Melissa Burkley blogged last month, effects of that size are what support claims that aspirin can prevent heart attacks or that eating calcium helps build healthy bones.

So basically people just dictate to us what we should believe.

Well, lets hope this experiment can be repeated and that it's methodology was indeed as flawless as claimed.

I'd like to see hit rates of over 50% for four choices and 75% for 2 choices.





I've been right on dice rolls (50% chance with 2) as much as 20+ times in a row. I certainly do not have any "psychic" powers or latent psi abilities.



The tests would have to be run many days and thousands of times (tens of thousands) in order to be legit, to me.





In statistics, it's not unheard of for a professor to do the ol' coin flip 100 times homework. If you don't have at least 6 coin flips in a row, you fail the assignment because 100 coin flips should result in at leas one side being tossed 6 times in a row.

some of those stats aren't correct....

if you are comparing the observed mean to 25%, and the 95% CI contains the value 25, your p-value, by definition, can't be significant.

Originally posted by dadudemon


I've been right on dice rolls (50% chance with 2) as much as 20+ times in a row. I certainly do not have any "psychic" powers or latent psi abilities.



Maybe, but you do say alot of stuff.

Originally posted by dadudemon



The tests would have to be run many days and thousands of times (tens of thousands) in order to be legit, to me.


Yes because 100s of trials aren't enough, I see where you're coming from.



Originally posted by dadudemon


In statistics, it's not unheard of for a professor to do the ol' coin flip 100 times homework. If you don't have at least 6 coin flips in a row, you fail the assignment because 100 coin flips should result in at leas one side being tossed 6 times in a row.

Dunno about that.

Originally posted by inimalist
some of those stats aren't correct....

if you are comparing the observed mean to 25%, and the 95% CI contains the value 25, your p-value, by definition, can't be significant.

You might be right, but you're probably lying.


Originally posted by Bardock42
I am of course referring not to the discredited and abandoned older research, but that new research that your second link claims is so professional and accepted.

*shrug* Sound as if you're taking the piss.

Originally posted by Deadline
Maybe, but you do say alot of stuff.

No, not "maybe". A mystic would say I have "abilities" when hearing that story.

A statistician would say that it is a probability to get 100 in a row with millions of trials.



Originally posted by Deadline
Yes because 100s of trials aren't enough, I see where you're coming from.

Correct.


Some only had 100 in the data set.


Not only would you want to see thousands but you'd want to see results duplication. (Not really thousands...but something like that would need a lot od evidence to get the scientific community to believe)




Originally posted by Deadline
Dunno about that.

Well I do and it's true. The larger your "set", the higher the probability that you'll have 'straight' runs of one side or the other.

I'd be very happy if Psi would be real. That would be an enormous finding. I just think that a lot of the tests I have heard about are by no means valid.

Why do you think that, if there is such good evidence, that the scientific community tries to hide it? I mean, that's what you propose, yes? That there is a conspiracy to hide the significant findings in Psi from the general public.

woah... ummm, I think you guys just gave Cohen a stroke....


the way those statistical tests are done, the more subjects and the more trials actually biases in favor of a positive result. Increasing the N in an experiment, by definition, decreases the standard deviation, and thus reduces the 95% CI to a smaller range, that is therefore less likely to contain the null. It is one of the major problems with those statistical methods, and a full power analysis (something with its own problems) would be required to know exactly how many subjects and trials would be appropriate.

The idea that more people doing more trials is good for science is false, 100% false

Originally posted by dadudemon
No, not "maybe". A mystic would say I have "abilities" when hearing that story.

A statistician would say that it is a probability to get 100 in a row with millions of trials.


As far as I'm aware you usually have a bigger number than that in psi experiements. They don't just stop at 20 guesses and job done.


Originally posted by dadudemon

Correct.


Some only had 100 in the data set.


Not only would you want to see thousands but you'd want to see results duplication. (Not really thousands...but something like that would need a lot od evidence to get the scientific community to believe)






Well I do and it's true. The larger your "set", the higher the probability that you'll have 'straight' runs of one side or the other.

I kinda see what you're saying but the reason why these experiments were set this way was because they were within the protocols of accepted science and I might be wrong but other areas of reasearch follow the same criteria.

What you're saying now is eventhough the experiements followed scientific protocols it's not good enough. In fact theres evidence that psi experiments have actually better standards than other experiments.

I bet you any money that there other areas of science which don't accept the criteria you mentioned.

Originally posted by inimalist
The idea that more people doing more trials is good for science is false, 100% false

K.


I'll be over here, not doing "real" science with more trials. My quack statisticians like this thing called "effect size".

Originally posted by Bardock42
I'd be very happy if Psi would be real. That would be an enormous finding. I just think that a lot of the tests I have heard about are by no means valid.

Why do you think that, if there is such good evidence, that the scientific community tries to hide it? I mean, that's what you propose, yes? That there is a conspiracy to hide the significant findings in Psi from the general public.

facepalm

Originally posted by inimalist
woah... ummm, I think you guys just gave Cohen a stroke....


the way those statistical tests are done, the more subjects and the more trials actually biases in favor of a positive result. Increasing the N in an experiment, by definition, decreases the standard deviation, and thus reduces the 95% CI to a smaller range, that is therefore less likely to contain the null. It is one of the major problems with those statistical methods, and a full power analysis (something with its own problems) would be required to know exactly how many subjects and trials would be appropriate.

The idea that more people doing more trials is good for science is false, 100% false


If there were less subjects you would find something else to complain about. There was a telephone telepathy experiment done by Rupert Sheldrake and in those experiments they complained there weren't enough subjects. Damned if you do, damned if you don't.

Which is the point I've been making all along. You don't make constructive criticism you invent stuff to complain about. You've already made your mind up. If this were an experiment about something else you wouldn't be saying that.

Originally posted by dadudemon
K.


I'll be over here, not doing "real" science with more trials.

the most common scientific tests are t-tests or z-tests (ANOVAs and F-tests too, but they are much more complicated than I want to explain on a forum....).

In a t or z test, you compared your observed mean to a null mean, divided by either the standard deviation or standard error of your results:

t = (mean1-mean2)/SD

when calculating either SD or SE, the N (number of trials/participants) is the denominator:

SD = (whatever)/N

as N increases, the (whatever)/N value decreases, thus, SD decreases. As SD decreases, the t value increases, as the difference between the means is divided by an increasingly lower number. The higher your t value, the more likely the result is significant. it is a basic law of the equations used.

like, seriously, I'm not even exaggerating at this point, null hypothesis testing has major, major problems

Originally posted by Deadline
If there were less subjects you would find something else to complain about. There was a telephone telepathy experiment done by Rupert Sheldrake and in those experiments they complained there weren't enough subjects.

I wouldn't have made that complaint

Originally posted by Deadline
Which is the point I've been making all along. You don't make constructive criticism you invent stuff to complain about.

like, I'm working on a post that tries to explain confidence intervals to you, is this even worth my time?

Originally posted by Deadline
facepalm


I'm not trying to insult you. That is what you think, isn't it?

Every man and his dog is complaining about psi experiments not being repeatable? How the hell do you disprove that? By having more trials. Now somebodies trying to tell me more trials is bad?

If this isn't an example of trying to take the piss I don't know what is.

Originally posted by inimalist
the most common scientific tests are t-tests or z-tests (ANOVAs and F-tests too, but they are much more complicated than I want to explain on a forum....).

In a t or z test, you compared your observed mean to a null mean, divided by either the standard deviation or standard error of your results:

t = (mean1-mean2)/SD

when calculating either SD or SE, the N (number of trials/participants) is the denominator:

SD = (whatever)/N

as N increases, the (whatever)/N value decreases, thus, SD decreases. As SD decreases, the t value increases, as the difference between the means is divided by an increasingly lower number. The higher your t value, the more likely the result is significant. it is a basic law of the equations used.

like, seriously, I'm not even exaggerating at this point, null hypothesis testing has major, major problems


If I'm not mistaken, I was just referring to 2 side and 4 side probabilities and the likelihood of guessing results.

Is that not what those tests were?

There's also the literal requirement of a significant sample size. All the time I read articles about a new drug or a new method that has fewer than 20 samples and the journal concludes: "More testing will be required to determine if x is effective."

In z tests, you must have a significant sample size in order to overcome the null hypothesis...if you're trying to show the desired results. In fact, not having a large enough sample size in your z-testing can result in shitty results or wacky SDs (nuisance parameters).

In the medical community, 30 is usually considered acceptable for z-testing. It's the same for physics.

Originally posted by dadudemon

There's also the literal requirement of a significant sample size. All the time I read articles about a new drug or a new method that has fewer than 20 samples and the journal concludes: "More testing will be required to determine if x is effective."

Exactly. What on earth is he talking about?

Originally posted by Bardock42
I'm not trying to insult you. That is what you think, isn't it?

Go ahead. If trolling makes you feel good thats says alot about you. Yes and I do think there might be a 'conspiracy' but I think it may just be more like prejudice.

Originally posted by Deadline
Every man and his dog is complaining about psi experiments not being repeatable? How the hell do you disprove that? By having more trials. Now somebodies trying to tell me more trials is bad?

If this isn't an example of trying to take the piss I don't know what is.

oh, ok

there is a difference between more trials in a single experimental design (the N value) and replicating experimental designs using similar N values.

within a single experiment, yes, there are problems with increasing N, because of how it effects the calculation of standard deviation or standard error, however, across experiments this isn't the case, as SD and SE are calculated as unique values within individual experimental designs.

within a single experiment, increasing N will only have the effect of lowering your p value (increasing the likelihood of significant results), and is something I am personally loathe to do .

Replication, on the other hand, verifies previous findings and has no impact on their statistical results.

Do you know about meta-analysis?

Originally posted by dadudemon
If I'm not mistaken, I was just referring to 2 side and 4 side probabilities and the likelihood of guessing results.

Is that not what those tests were?

at the end of the day, its all t-tests

Originally posted by dadudemon
There's also the literal requirement of a significant sample size. All the time I read articles about a new drug or a new method that has fewer than 20 samples and the journal concludes: "More testing will be required to determine if x is effective."

In z tests, you must have a significant sample size in order to overcome the null hypothesis...if you're trying to show the desired results. In fact, not having a large enough sample size in your z-testing can result in shitty results or wacky SDs (nuisance parameters).

In the medical community, 30 is usually considered acceptable for z-testing. It's the same for physics.

no, totally

sorry, I'm never sure of how much people know about stats.

z tests have far more problems associated with them than t tests, like the one you mentioned above. There is also the problem of the SD in a z test assuming to represent a population mean rather than sample mean.

This really doesn't change the fact that inflating N gives more positive results. There is a theoretical "sweet spot" for any design based on the variance in the sample, however, there are only indirect and, imho, half assed ways to calculate it (power analysis, for instance). It is one of the main reasons why so many people have complained about NHST in recent years, and why I am teaching myself Bayes for my own data

Originally posted by inimalist
oh, ok

there is a difference between more trials in a single experimental design (the N value) and replicating experimental designs using similar N values.


My point.


I don't mean having someone "flex" their "psi" abilities 1000 thousand times in a single trial.


I'm referring to testing over and over again. Keeping bitches fresh for measurements.

Originally posted by inimalist

within a single experiment, increasing N will only have the effect of lowering your p value (increasing the likelihood of significant results), and is something I am personally loathe to do .

What experiment are you looking at? I think they mentioned the number of trials. Whats your def of an experiment 1 trial or many?


Originally posted by inimalist

Do you know about meta-analysis?

A bit.

Originally posted by dadudemon
My point.


I don't mean having someone "flex" their "psi" abilities 1000 thousand times in a single trial.

I'm not even sure they did that.

Originally posted by dadudemon

I'm referring to testing over and over again. Keeping bitches fresh for measurements.

Yes by having lots of trials.

Originally posted by Deadline
Go ahead. If trolling makes you feel good thats says alot about you. Yes and I do think there might be a 'conspiracy' but I think it may just be more like prejudice.

I don't mean to troll you, I would like to understand where you are coming from.

Originally posted by Deadline
What experiment are you looking at?

I'm actually just speaking in general here about stats more abstractly. These N problems apply to any science that uses null hypothesis testing

Originally posted by Deadline
I think they mentioned the number of trials.

they do, and I could probably look up and run a power analysis, but I'm not actually arguing that any of those results have an issue with N . I was more commenting on you and ddm talking about wanting to run more trials.

Though I may have confused what you guys were talking about, ie: running more experiments to replicate previous results, with simply just running thousands of trials within single experiments

Originally posted by Deadline
Whats your def of an experiment 1 trial or many?

I wouldn't define an experiment by the number of trials actually. for sure, there has to be more than one. Optimally, you just have to be smart as a researcher. Ive seen studies that had an N over 5000, but only 2 subjects (over 2500 trials per subject). They had awesome p-values, but only 2 subjects, no matter how many trials, isn't really good science. The opposite is true too, 1000 subjects with 2 trials each isn't worth much at all.

I'm actually trying to teach myself new statistical methods that aren't tied to things like N or the other problems in NHST, namely, Bayesian probability. I honestly do not trust p values anymore

Originally posted by Deadline
A bit.

ok, so the only time too much replication would be a problem is when someone does a meta-analysis. There is something called the "file-drawer" effect, meaning that studies that don't produce significant results don't get published. So, I can run statistical analyses over many studies in an attempt to see if there is any overarching pattern, however, if all that are published are the studies that are successful, this meta-analysis will be biased in favor of a positive result.

another issue with modern science is that you can never get null (negative) results published.

Originally posted by inimalist
I'm actually just speaking in general here about stats more abstractly. These N problems apply to any science that uses null hypothesis testing



they do, and I could probably look up and run a power analysis, but I'm not actually arguing that any of those results have an issue with N . I was more commenting on you and ddm talking about wanting to run more trials.

Though I may have confused what you guys were talking about, ie: running more experiments to replicate previous results, with simply just running thousands of trials within single experiments



I wouldn't define an experiment by the number of trials actually. for sure, there has to be more than one. Optimally, you just have to be smart as a researcher. Ive seen studies that had an N over 5000, but only 2 subjects (over 2500 trials per subject). They had awesome p-values, but only 2 subjects, no matter how many trials, isn't really good science. The opposite is true too, 1000 subjects with 2 trials each isn't worth much at all.

I'm actually trying to teach myself new statistical methods that aren't tied to things like N or the other problems in NHST, namely, Bayesian probability. I honestly do not trust p values anymore



ok, so the only time too much replication would be a problem is when someone does a meta-analysis. There is something called the "file-drawer" effect, meaning that studies that don't produce significant results don't get published. So, I can run statistical analyses over many studies in an attempt to see if there is any overarching pattern, however, if all that are published are the studies that are successful, this meta-analysis will be biased in favor of a positive result.

Yea I see what you mean but I don't think that applies to anything I posted.

Originally posted by inimalist

another issue with modern science is that you can never get null (negative) results published.

Dunno about that pretty sure I've seen failed replications of psi.

Originally posted by Deadline
Yea I see what you mean but I don't think that applies to anything I posted.

sort of, it does apply to how to replicate the research

the part about not trusting NHST is applicable /shrug

Originally posted by Deadline
Dunno about that pretty sure I've seen failed replications of psi.

fair enough

take that to mean they are rare. you seem them occasionally, but most journals wont publish nulls. You certainly don't see even a significant fraction of null results published.

Originally posted by inimalist
ok, so the only time too much replication would be a problem is when someone does a meta-analysis. There is something called the "file-drawer" effect, meaning that studies that don't produce significant results don't get published. So, I can run statistical analyses over many studies in an attempt to see if there is any overarching pattern, however, if all that are published are the studies that are successful, this meta-analysis will be biased in favor of a positive result.

another issue with modern science is that you can never get null (negative) results published.

Those meta-analyses are instantly criticized and some will not even get "funded" before they can commence the research.

Maybe I'm more of an optimist and think that department heads don't approve projects that have a propensity towards the publication bias.

IMO, it can't be considered a meta-analysis unless "inconclusive" and "negative" results are also included (where applicable).

But what about negative or inconclusive results that were considered "improperly" conducted as I often see from the primary researchers? They scream, "they didn't do the test according to the parameters so their results must be thrown out! RAWR!"



Other times, meta-analyses can reveal quite awesome information via the "aggregate conclusion" phenomenon.







About the other stuff.


The "N", especially when testing humans, HAS to be in separate "testing" events due to fatigue. You can't have someone do 500 trials in one sitting without expecting results to change towards the end.



Also, depending on what is being tested, it very well may BE legitimate to use 1000 subjects with only 1 or 2 trials for each. You should still see the realization of the central limit theorem in such a case. That really does depend on the test being done because you can't test to see how people will respond to 4 pictures, one the object, and then measure only one viewing while randomizing the object among them. Or can you?


DAMNIT! Soooo much headache.

Originally posted by Deadline
You might be right, but you're probably lying.

well, it is actually really simple

all of the data you have presented is based on what is called a "null-hypothesis-significance-test" (NHST). in an NHST, you compare the data you obtain through an experiment to a "null" value that represents chance.

So, in most studies data is compared to a null hypothesis (H0) of 0, meaning there is no effect. In the cases you presented, instead of testing against no effect, they tested against a chance null, H0=25%. This is fine. Now, say I have results where my observed mean (u1) is 30. This means there is an absolute difference of 5% between my observed mean hit rate, and the assumed null chance hit rate (u0):

u1-u0=5

However, in my test, not every subject had a hit rate of exactly 30. Some were above and others were below. So, we then take an average of how much people are different from 30, known as the standard deviation, SD.

There are other things here, like probability distributions, etc, but for the sake of simplicity, just trust me that as a mathematical law, you can say what percentage of subject scores will fall within any number of standard deviations from the observed mean.

So, lets say that the SD in these results was 3. So, it is known that ~65% of all subjects will fall within one standard deviation of the mean. So, our 65% confidence interval would be 27-33.

now, there is something called an alpha value (a). alpha can be seen as, ummmmm, lets say the "opposite" of a confidence interval. So, the tradition in science is to use an alpha value of .05, or 5%, meaning that the typical CI that is used in experiments is 95%. basically, the CI representing all the values which are represented by your observed data, and alpha representing those that are outside.

a 95% confidence interval is all data within 2 standard deviations of the observed mean, so in this case, 30 +/- 6, or 24-36.

because 25 falls within this range, we cannot say it is statistically different from our null mean. Based on the variance in the data, the score of 25% would not be unexpected, therefore, the result is non significant. Let me know if this doesn't make sense.

Also, about the H0=25%. That null isn't the most appropriate, as in many cases, the probability of something occurring is based not simply on a raw percentage estimate, but can be influenced by previous trials and other mundane things. Think about it like this: I have a deck of cards and I say "deadline, predict the suit of the next card". Now, the raw probability is 1/4, or 25%. However, as we go through the deck, the suit of the previous cards can influence how probable it is another suit would come up. So, if you see a string of hearts, you as the experimental subject would then, even if subconsciously, know not to select hearts, as there are now fewer hearts in the deck than the other suits, making the probability of the other suits greater than 25%. This isn't nitpicking either. The studies I did in my undergrad had subjects distinguish between an L and a T on target objects. There was nothing important about the T or the L, but even in that case, they would often ask "why were there more Ts/Ls?". This type of probability is something our brains are intrinsically aware of, and could certainly cause difficulty in determining what a proper null percentage would be in these experiments. Another example of this is from previous stuff you showed me that said that a particular subject seemed to have a talent for remote viewing military installations. However, the studies were conducted on military bases (iirc), meaning that the context may have played a role in priming a certain type of response in the individual. However, it could be even more mundane, as cognitive biases like that could be produced from someone simply being a fan of Command and Conquer games.

Additionally, as I posted in the Atheism thread, when you apply more rigorous statistical methods, like Bayesian probability analysis, most of the significant results in psi have been seen to evaporate. There are a number of reasons for this. For one, a NHST does not tell you how likely it is that your hypothesis is true, but rather, how likely it is chance alone is responsible for your results. Studies have looked at the correlation between p-values (the probability of chance explaining your results) and true hypotheses, and found the R value to be just over .35 (extremely low), and this number drops when you restrict p-values to only those that would find significant results. (please ask if this doesn't make sense, I'm sure stats aren't as exciting to you as they are to me, but if you want to talk about double standards in science, you need to understand how stats work). These results are interesting, but really only show that, in a few experiments, the pattern of results aren't what would be expected due to chance alone.

this was too long to include as part of the last post, but here are studies that have begun analyzing previous psi results using Bayesian methods

Originally posted by inimalist
hmmmmm, can't think of a better place to post this, so a little OT, but I think you guys might be interested.

So, in the field of statistical analysis, there is this new concept known as Bayesian Probability (BP), which can replace Null-Hypothesis significance testing (NHST), as a way of determining whether you have an effect in your data. I don't really understand it at this point (actually, the reason I found these articles was looking for intros to BP on pubmed for my own research), but if you look at statistical science, it is pretty much taken as a given that BP is superior to NHST for a number of reasons (in fact, NHST has massive and fatal problems, but for some reason, psychologists seem to be the last people to abandon it).

Anyways, in my search for such tutorials, I can across a pair of articles that will undoubtedly re-awaken some old debates, but that I feel most people here are going to get a kick out of.

They are both psi studies that reevaluated results found using NHST with BP. In this first, a series of 9 studies with over 1000 participants, which were all positive results using NHST, were found to be completely insignificant using BP:



http://www.ncbi.nlm.nih.gov/pubmed/21280965

The second, while still finding 3 out of 6 positive results using BP, overturned a 6 out of 6 positive result using NHST:



http://www.ncbi.nlm.nih.gov/pubmed/21713069

I'm really interested in reading the second, simply to see what evidence there is that still remains, but the take away from this post is that, even in the past where psi phenomena may have been discovered in tests, we see now, that superior statistical methods actually reduce, if not eliminate entirely, the salience of that evidence. This is nothing new, the same is typically found with tighter controls, etc, just something that tickled me in the right way this morning.

...

Originally posted by dadudemon
Those meta-analyses are instantly criticized and some will not even get "funded" before they can commence the research.

Maybe I'm more of an optimist and think that department heads don't approve projects that have a propensity towards the publication bias.

IMO, it can't be considered a meta-analysis unless "inconclusive" and "negative" results are also included (where applicable).

But what about negative or inconclusive results that were considered "improperly" conducted as I often see from the primary researchers? They scream, "they didn't do the test according to the parameters so their results must be thrown out! RAWR!"



Other times, meta-analyses can reveal quite awesome information via the "aggregate conclusion" phenomenon.

actually, the trick is that once you have a meta-analysis score, you can run some stats to see how many null results would have to be included in the meta-analysis to make it non-significant.

So like, if you find that your significant meta-analysis would only need one or two disconfirming studies to show there is no actual pattern in the data, you can probably conclude the meta-analysis is reflecting some type of file drawer effect. If the number is like 400000 or something, then you can probably be confident that there is a real effect. This is actually the case with the research done of violence in media and violence in people. The last such analysis I saw said that something like only 20 null results would reduce the effect to non-significance.

you are right about null results though, a lot aren't published because they just aren't good research versus just not being a good narrative. Its a double edge sword though, and also applies to research that finds different results, as stuff I did in undergrad was almost impossible to publish, because if provided significant evidence against prevailing theories.

Originally posted by dadudemon
About the other stuff.


The "N", especially when testing humans, HAS to be in separate "testing" events due to fatigue. You can't have someone do 500 trials in one sitting without expecting results to change towards the end.



Also, depending on what is being tested, it very well may BE legitimate to use 1000 subjects with only 1 or 2 trials for each. You should still see the realization of the central limit theorem in such a case. That really does depend on the test being done because you can't test to see how people will respond to 4 pictures, one the object, and then measure only one viewing while randomizing the object among them. Or can you?

sure, but now we are differentiating statistics from experimental design. 1 subject doing 10000000000 trials is going to net you an impressively low p-value, and will conform to general statistical rules, but in terms of being proper or well designed research, it is nonsense.

my own bias is that, no, 2 trials is not enough per participant. It is stuff like that which causes myself and other lab mates to laugh heartily at social psych research that depends on survey results and the like.

Originally posted by dadudemon
DAMNIT! Soooo much headache.

pfft, how can you not like stats?

Originally posted by dadudemon
Maybe I'm more of an optimist and think that department heads don't approve projects that have a propensity towards the publication bias.

man, if you only knew what a cluster**** funding is

Originally posted by inimalist
well, it is actually really simple

all of the data you have presented is based on what is called a "null-hypothesis-significance-test" (NHST). in an NHST, you compare the data you obtain through an experiment to a "null" value that represents chance.

So, in most studies data is compared to a null hypothesis (H0) of 0, meaning there is no effect. In the cases you presented, instead of testing against no effect, they tested against a chance null, H0=25%. This is fine. Now, say I have results where my observed mean (u1) is 30. This means there is an absolute difference of 5% between my observed mean hit rate, and the assumed null chance hit rate (u0):

u1-u0=5

However, in my test, not every subject had a hit rate of exactly 30. Some were above and others were below. So, we then take an average of how much people are different from 30, known as the standard deviation, SD.

There are other things here, like probability distributions, etc, but for the sake of simplicity, just trust me that as a mathematical law, you can say what percentage of subject scores will fall within any number of standard deviations from the observed mean.

So, lets say that the SD in these results was 3. So, it is known that ~65% of all subjects will fall within one standard deviation of the mean. So, our 65% confidence interval would be 27-33.

now, there is something called an alpha value (a). alpha can be seen as, ummmmm, lets say the "opposite" of a confidence interval. So, the tradition in science is to use an alpha value of .05, or 5%, meaning that the typical CI that is used in experiments is 95%. basically, the CI representing all the values which are represented by your observed data, and alpha representing those that are outside.

a 95% confidence interval is all data within 2 standard deviations of the observed mean, so in this case, 30 +/- 6, or 24-36.

because 25 falls within this range, we cannot say it is statistically different from our null mean. Based on the variance in the data, the score of 25% would not be unexpected, therefore, the result is non significant. Let me know if this doesn't make sense.

Also, about the H0=25%. That null isn't the most appropriate, as in many cases, the probability of something occurring is based not simply on a raw percentage estimate, but can be influenced by previous trials and other mundane things. Think about it like this: I have a deck of cards and I say "deadline, predict the suit of the next card". Now, the raw probability is 1/4, or 25%. However, as we go through the deck, the suit of the previous cards can influence how probable it is another suit would come up. So, if you see a string of hearts, you as the experimental subject would then, even if subconsciously, know not to select hearts, as there are now fewer hearts in the deck than the other suits, making the probability of the other suits greater than 25%. This isn't nitpicking either. The studies I did in my undergrad had subjects distinguish between an L and a T on target objects. There was nothing important about the T or the L, but even in that case, they would often ask "why were there more Ts/Ls?". This type of probability is something our brains are intrinsically aware of, and could certainly cause difficulty in determining what a proper null percentage would be in these experiments. Another example of this is from previous stuff you showed me that said that a particular subject seemed to have a talent for remote viewing military installations. However, the studies were conducted on military bases (iirc), meaning that the context may have played a role in priming a certain type of response in the individual. However, it could be even more mundane, as cognitive biases like that could be produced from someone simply being a fan of Command and Conquer games.

Additionally, as I posted in the Atheism thread, when you apply more rigorous statistical methods, like Bayesian probability analysis, most of the significant results in psi have been seen to evaporate. There are a number of reasons for this. For one, a NHST does not tell you how likely it is that your hypothesis is true, but rather, how likely it is chance alone is responsible for your results. Studies have looked at the correlation between p-values (the probability of chance explaining your results) and true hypotheses, and found the R value to be just over .35 (extremely low), and this number drops when you restrict p-values to only those that would find significant results. (please ask if this doesn't make sense, I'm sure stats aren't as exciting to you as they are to me, but if you want to talk about double standards in science, you need to understand how stats work). These results are interesting, but really only show that, in a few experiments, the pattern of results aren't what would be expected due to chance alone.

Look man it's debateable wether Bayesian analysis is more rigorous and I don't think it makes past results null and void. Bayesian analysis enables you to get what results depending on how biased you are. Don't think you have that with the other method. Think this is explained at the beginning here.

http://www.psy.unipd.it/~tressold/cmssimple/uploads/includes/ExtraordinaryClaim011.pdf

Too busy to read all of it.

Looks like a lot of medical experiments could have cognitive bias if you read comics.

Originally posted by Deadline
Look man it's debateable wether Bayesian analysis is more rigorous and I don't think it makes past results null and void. Bayesian analysis enables you to get what results depending on how biased you are. Don't think you have that with the other method. Think this is explained at the beginning here.

http://www.psy.unipd.it/~tressold/cmssimple/uploads/includes/ExtraordinaryClaim011.pdf

Too busy to read all of it.

Looks like a lot of medical experiments could have cognitive bias if you read comics.

I've read and, in fact, posted that study to you in the past... It shows that a Bayes analysis reduces iirc 6 significant results to 3... That isn't a strong positive for previous psi research. If you are really interested you can look up my summation of it and the experiments immediately preceding it in the "Atheism" thread.

can you explain how "Bayesian analysis enables you to get what results depending on how biased you are."? I haven't found that in the article or in any of the reading I have done on Bayes theory. I suspect you are confusing the assignment of initial probability with some type of bias, but even then, there are very conservative and non-subjective fixes to that abundant in the literature... Even then, however, after the first iteration of the analysis, that initial probability becomes essentially moot...

EDIT: WTF, lol, nvm dude, i re-posted that already, like 5 posts above this one. If you are too busy to read my argument, don't post a reply, it really makes you look sloppy...

Originally posted by inimalist
I've read and, in fact, posted that study to you in the past... It shows that a Bayes analysis reduces iirc 6 significant results to 3... That isn't a strong positive for previous psi research. If you are really interested you can look up my summation of it and the experiments immediately preceding it in the "Atheism" thread.

can you explain how "Bayesian analysis enables you to get what results depending on how biased you are."? I haven't found that in the article or in any of the reading I have done on Bayes theory. I suspect you are confusing the assignment of initial probability with some type of bias, but even then, there are very conservative and non-subjective fixes to that abundant in the literature... Even then, however, after the first iteration of the analysis, that initial probability becomes essentially moot...

EDIT: WTF, lol, nvm dude, i re-posted that already, like 5 posts above this one. If you are too busy to read my argument, don't post a reply, it really makes you look sloppy...

Haven't been on here for while but just need say somethings. I have little doubt that you're probably twisting things around. I also noticed you only started mentioning Bayesian analysis after I kept posting experiments with stats to back it up. I suspect you probably never even heard of Bayesian analysis and just started using it.

Anyway I don't have enough in-depth knowledge to challenge in detail (or time to read up) but again I've asked people who have more knowledge about this subject than you do and they don't agree with your opinion.

I really can't be bothered if I had time I'd probably find out you haven't read the pdf propely. That pdf was actually recommended, I can assume what you're going to do and sometimes it's not really worth the effort.

you outdo yourself every time

LOL @ Deadline.

Originally posted by inimalist
you outdo yourself every time

Dammm you inimalist!!! angrymob

I don't see what you want here deadline...? like, you seem determined to keep this active, but your posts amount to little more than accusations of lying and insults, whereas you are the first person to cry foul when you perceive even the smallest slight from other members. You clearly don't read what I'm writing, and have essentially refused to support any study or answer any claim I have made in this thread.

It really seems like you are looking for some kind of validation or vindication of your position from me, which, tbh, really isn't that important. If you believe this stuff, sure, that is cool, I don't judge you for that, however, if you want to substantiate this idea that psi phenomenon are being specifically ignored and there is a vast conspiracy among psychologists to suppress it, you might have to do better than suggesting I don't understand things that I've made (in retrospect far too) long posts describing.

I mean, you asked people who know better than I do, wtf do you care what I think? Obviously I don't know what I'm talking about. It's not like I do this for a living. Every. Day. Of. My. Life.

Originally posted by Deadline
Haven't been on here for while but just need say somethings. I have little doubt that you're probably twisting things around. I also noticed you only started mentioning Bayesian analysis after I kept posting experiments with stats to back it up. I suspect you probably never even heard of Bayesian analysis and just started using it.

You keep making accusations like this, but never back them up. I would think you'd enjoy someone taking as much time as inimilist has to engage the topic with you, regardless of whether or not you disagree. Because, contrary to what your posts suggest you think, most peoples' motivating factor in these discussions is not to win debates on the internet by making things up.

Originally posted by Deadline
Anyway I don't have enough in-depth knowledge to challenge in detail (or time to read up) but again I've asked people who have more knowledge about this subject than you do and they don't agree with your opinion.

Professed ignorance followed by an appeal to authority.

Originally posted by Deadline
I really can't be bothered if I had time I'd probably find out you haven't read the pdf propely. That pdf was actually recommended, I can assume what you're going to do and sometimes it's not really worth the effort.

Lack of effort followed by another veiled accusation (also unsupported).

Note: I haven't once commented on your position, just your methods of debate. You can change to a more amenable form of communication without rubbing everyone the wrong way, and without having to change your positions on any topic.

Originally posted by Deadline
I suspect you probably never even heard of Bayesian analysis and just started using it.



What's wrong with using an alternative and possibly superior method even IF he just started using it? I see no fault. One should be applauded for such an endeavor. Don't you agree?

Originally posted by Deadline
Anyway I don't have enough in-depth knowledge to challenge in detail (or time to read up) but again I've asked people who have more knowledge about this subject than you do and they don't agree with your opinion.

Who? It's basic stuff you'll cover in your college stats class. There's no much to disagree on other than the semantics inimalist and I were arguing over...and that argument did not really affect the overall conclusion of how this data can be used (or not used).

Originally posted by dadudemon
Who? It's basic stuff you'll cover in your college stats class. There's no much to disagree on other than the semantics inimalist and I were arguing over...and that argument did not really affect the overall conclusion of how this data can be used (or not used).

Bayes doesn't get taught in most undergrad stat classes since its so anti-intuitive and people are already terrified of math classes so I'm not sure I'd call it "basic stuff". You might get Bayes Theorem mentioned but that's about it. It would be hard to get into graduate level psychology (or any other field that's heavy on data analysis) in the last ten years without being familiar with it, though.

Originally posted by Symmetric Chaos
Bayes doesn't get taught in most undergrad stat classes since its so anti-intuitive and people are already terrified of math classes so I'm not sure I'd call it "basic stuff". You might get Bayes Theorem mentioned but that's about it. It would be hard to get into graduate level psychology (or any other field that's heavy on data analysis) in the last ten years without being familiar with it, though.

Look at the portion of his post that I quoted. The words I typed to that portion of his post have nothing directly to do with Bayesian probabilities nor did the section of his that I quoted.

Edit - Additionally, Bayesian stats is covered in my 3000 level class (junior) so I am not sure what you're talking about. It would depend on your degree and the institution you attend.

Originally posted by dadudemon
Look at the portion of his post that I quoted. The words I typed to that portion of his post have nothing directly to do with Bayesian probabilities nor did the section of his that I quoted.

Deadline's whole post looked like it was about Bayes.

Originally posted by dadudemon
Edit - Additionally, Bayesian stats is covered in my 3000 level class (junior) so I am not sure what you're talking about. It would depend on your degree and the institution you attend.

I certainly wouldn't count Junior level stats as "basic". srug

At that point your probably in a major where stats are really going to be important.

Originally posted by Symmetric Chaos
Bayes doesn't get taught in most undergrad stat classes since its so anti-intuitive and people are already terrified of math classes so I'm not sure I'd call it "basic stuff". You might get Bayes Theorem mentioned but that's about it. It would be hard to get into graduate level psychology (or any other field that's heavy on data analysis) in the last ten years without being familiar with it, though.

actually, we don't even get it at a graduate level, hence why I was trying to teach myself.

among the sciences though, psychology is still behind the times in terms of stats. I've had people in graduate level stats courses argue that they aren't important because you can just look at a graph and tell what is significant (and, fields like ABA, personality/social, etc, are often loath to use real statistics).

From the readings I do, Bayes is actually really rare, to the point where it might actually be more difficult to get publications with Bayes, as the editors wont know it and people are very territorial when it comes to change.

Originally posted by Symmetric Chaos
Deadline's whole post looked like it was about Bayes.

I disagree, but I could be wrong.

Originally posted by Symmetric Chaos
I certainly wouldn't count Junior level stats as "basic". srug

I wouldn't count junior level stats as post-grad coursework, either.

However, I looked it up and it is a 2000 level course (sophomore), not a 3000 level.

Originally posted by Symmetric Chaos
At that point your probably in a major where stats are really going to be important.

Cyber Security and Digital Forensics. But after contemplating your point, you're right: Business Stats uses "Bayesian statistics" more so than most other applied stats courses that I know.


Originally posted by inimalist
From the readings I do, Bayes is actually really rare, to the point where it might actually be more difficult to get publications with Bayes, as the editors wont know it and people are very territorial when it comes to change.

I believe it was you who I was talking to about how "cliquish" publishing work can be. It is a bit irritating, imo, and part of the "subjective" portion of science that shouldn't really be there. I do understand the funding and "audience" requirements, however...

Originally posted by Digi
You keep making accusations like this, but never back them up. I would think you'd enjoy someone taking as much time as inimilist has to engage the topic with you, regardless of whether or not you disagree. Because, contrary to what your posts suggest you think, most peoples' motivating factor in these discussions is not to win debates on the internet by making things up.



Professed ignorance followed by an appeal to authority.



Lack of effort followed by another veiled accusation (also unsupported).

Note: I haven't once commented on your position, just your methods of debate. You can change to a more amenable form of communication without rubbing everyone the wrong way, and without having to change your positions on any topic.

Thats nonsense and you know it. Already made a post which showed you're hypocrisy and and I'm pretty sure you decided to ignore it. Obvoulsy what you're trying to do now is to get everybody to gang up on me.

Don't bother because I can easily prove you're one of the hypocrites as well. I have better things to do however.


Originally posted by inimalist
I don't see what you want here deadline...? like, you seem determined to keep this active, but your posts amount to little more than accusations of lying and insults, whereas you are the first person to cry foul when you perceive even the smallest slight from other members. You clearly don't read what I'm writing, and have essentially refused to support any study or answer any claim I have made in this thread.

It really seems like you are looking for some kind of validation or vindication of your position from me, which, tbh, really isn't that important. If you believe this stuff, sure, that is cool, I don't judge you for that, however, if you want to substantiate this idea that psi phenomenon are being specifically ignored and there is a vast conspiracy among psychologists to suppress it, you might have to do better than suggesting I don't understand things that I've made (in retrospect far too) long posts describing.

I mean, you asked people who know better than I do, wtf do you care what I think? Obviously I don't know what I'm talking about. It's not like I do this for a living. Every. Day. Of. My. Life.

That last post with the smilies was supposed to be tongue in cheek. Look man you're basically a liar from the beginning you've argued that there is no evidence for the paranormal and I've discussed this shit with you for years. Whats clear is that you're biased, for example it seems pretty obvious that you didn't even know that there was actually evidence in favour of the paranormal or know any of the scientists who believe it exists.

On the face of it my post looks like a cop out but it just comes from experience of your debating style. Look didn't really want to argue with you, just explaining my reasons for not replying.





Originally posted by dadudemon


What's wrong with using an alternative and possibly superior method even IF he just started using it? I see no fault. One should be applauded for such an endeavor. Don't you agree?



Who? It's basic stuff you'll cover in your college stats class. There's no much to disagree on other than the semantics inimalist and I were arguing over...and that argument did not really affect the overall conclusion of how this data can be used (or not used).


Theres nothing wrong with that, but basically his MO is to try and debunk as much as he can hes not doing it with open mindness. He also exaggerates about how much he knows about this subject. He sets himself up as an expert but what beceome apparent over the years is that he really doesn't even know the subject well either.

You just get offended too easily. Half the time you're spouting off about hypocrisy or something and we're just confused, trying to have a conversation with you. Your apparent persecution complex doesn't serve you well here; we're just dudes trying to talk about sh*t, not one-up each other or gang up on anyone.

Anyway, I was just trying to help you out with my last post, even if it did include some criticisms of your debate style. I'm not "against" you in any sense here, and haven't even been a part of this conversation on experiments.

Originally posted by Digi
You just get offended too easily. Half the time you're spouting off about hypocrisy or something and we're just confused, trying to have a conversation with you. Your apparent persecution complex doesn't serve you well here; we're just dudes trying to talk about sh*t, not one-up each other or gang up on anyone.

You're 100% incorrect. Maybe what I'll do is post your obnoxious post again. Basically you athiests take yourself too serioulsy.

Originally posted by Deadline
You're 100% incorrect. Maybe what I'll do is post your obnoxious post again. Basically you athiests take yourself too serioulsy.

Generalization, sarcasm, and insults.

I've got something like 30K posts on KMC. If you decide to hate me, you can find some isolated incidents of idiocy. It's basically unavoidable. The vast majority of my history on this forum is behavior that I'm proud of, though - either debating rationally and respectfully with others, or sharing my views in a candid manner.

"Taking too seriously" I would think would be something more along the lines of holding grudges over months and years. I can't even remember our fights - seriously, I can't. I don't hold grudges. If I burned a torch for everyone who ever insulted or disliked me (usually just because I'm a mod, once in a while with legit reason), I'd never sleep.

If I've ever offended you in any way, you have my sincere apologies. It doesn't mean I'm a fan of how you're handling yourself right now, but it means I don't want bad blood between us.

Originally posted by Deadline
That last post with the smilies was supposed to be tongue in cheek. Look man you're basically a liar from the beginning you've argued that there is no evidence for the paranormal and I've discussed this shit with you for years. Whats clear is that you're biased, for example it seems pretty obvious that you didn't even know that there was actually evidence in favour of the paranormal or know any of the scientists who believe it exists.

On the face of it my post looks like a cop out but it just comes from experience of your debating style. Look didn't really want to argue with you, just explaining my reasons for not replying.

you are correct, it is entirely my fault that you are both unable and unwilling to defend your position.

look, you don't need me to validate what you believe, just be a happy person confident in the knowledge that I'm obviously wrong.































****ing douche

Originally posted by Deadline
Theres nothing wrong with that, but basically his MO is to try and debunk as much as he can hes not doing it with open mindness. He also exaggerates about how much he knows about this subject. He sets himself up as an expert but what beceome apparent over the years is that he really doesn't even know the subject well either.

There's this internet video out there that explains that concluding just one thing instead of keeping and open mind to many other explanations is actually being closed minded. Meaning, you're concluding one thing while inimalist is only addressing that one particular approach. He doesn't mean he is closed minded, at all.

HOWEVER...I have no idea what possibilities he is willing to entertain nor do I know if he is open minded. I'm just giving you a tip on what open-mindedness can mean for different people.


Me? Man, you know I'd love for those studies to show some sort of ESP. But I'd want some very very solid evidence before I believe the ESP conclusions. More likely, I would look for reasons that proved the stats were wrong or that the studies themselves were conducted with suspect. I consider myself fairly open minded by I also consider myself a skeptic.


More on your point, only an idiot would try and rationalize away clear evidence in favor of a conclusion. I am not too sure these studies do that for us. One or two did but it would have created a shit-storm if it was legit. So it is more likely that I am missing something or am not away of the criticisms of those two studies that appeared to have nifty results.


Originally posted by inimalist
****ing douche

Awww. Come on. He's just an irritated kid.

Originally posted by dadudemon
Awww. Come on. He's just an irritated kid.

I'm not of the opinion that pity is a more positive emotion than anger

Fair enough. I just don't want to see you in trouble for a kid's immature rage posts.

Originally posted by inimalist
you are correct, it is entirely my fault that you are both unable and unwilling to defend your position.

look, you don't need me to validate what you believe, just be a happy person confident in the knowledge that I'm obviously wrong.































****ing douche

I just decided to explain why I didn't respond.

Originally posted by dadudemon
Fair enough. I just don't want to see you in trouble for a kid's immature rage posts.

Cheers.

Originally posted by dadudemon
There's this internet video out there that explains that concluding just one thing instead of keeping and open mind to many other explanations is actually being closed minded. Meaning, you're concluding one thing while inimalist is only addressing that one particular approach. He doesn't mean he is closed minded, at all.



Um if you actually look at our discussions it's actually not about me being right. I argue in favour of the paranormal but don't neccesarily believe that the evidence is conclusive. His actual problem is he always assume that his atheistic point is the most rational explanation. ie if our current understanding according to science is that a certain state indicates that a person is dead and experiences an NDE then the person must be alive. So it doesn't matter what science says but because he thinks that his viewpoint is superior he'll assume that they have to be alive.

You see heres the problem. His debating style makes it impossible to prove the paranormal. So basically in the case of the dead person no matter how much evidence is supplied he keeps arguing that there is a 'rational' explanation. My point isn't that I neccesarily disagree with their point of view the problem is that his debating style makes it impossible. The irony of it is that I could actually make a better case for the support that there is no life after death than he can. So no you're 100% wrong there.

In all fairness you're who was arguing that all Vikings that die go to Vallhalla and being a complete douche about it as well, so really not so sure if you wanna be pointing fingers ( also stated you knew heathens not sure if I believe that). If you must know lil b has pointed out his bias as well.

Originally posted by Digi
Generalization, sarcasm, and insults.

I've got something like 30K posts on KMC. If you decide to hate me, you can find some isolated incidents of idiocy. It's basically unavoidable. The vast majority of my history on this forum is behavior that I'm proud of, though - either debating rationally and respectfully with others, or sharing my views in a candid manner.

"Taking too seriously" I would think would be something more along the lines of holding grudges over months and years. I can't even remember our fights - seriously, I can't. I don't hold grudges. If I burned a torch for everyone who ever insulted or disliked me (usually just because I'm a mod, once in a while with legit reason), I'd never sleep.

If I've ever offended you in any way, you have my sincere apologies. It doesn't mean I'm a fan of how you're handling yourself right now, but it means I don't want bad blood between us.

Not sure if I'm holding a grudge, you responded to my post I told you what I thought. Telling you what I think = grudge?

Originally posted by Deadline
Telling you what I think = grudge?

Issuing unprovoked insults based on at least months-old posts (probably years). I'd say so.

Originally posted by Digi
Issuing unprovoked insults based on at least months-old posts (probably years). I'd say so.

Well there not insults its the truth. Therefore telling like it is = grudge. Sure the post was old, had nothing better to do so I responded to an old post I was meaning to, not sure if I sounded that pissed. Hell inimalist called me a douche I didn't...yes of course he was justified

how hard it must be to be victimized so unfairly

Originally posted by inimalist
how hard it must be to be victimized so unfairly

Everyone creates their own hell.

Like I said, the persecution complex doesn't suit you. If you've developed animosities with others on the forums, it's never a one-sided affair.

Originally posted by Digi
Like I said, the persecution complex doesn't suit you. If you've developed animosities with others on the forums, it's never a one-sided affair.

Um you're an obnoxious hypocrite lets leave it at that.

Originally posted by Deadline
Um you're an obnoxious hypocrite lets leave it at that.

Your Karma is clear.

Originally posted by inimalist
how hard it must be to be victimized so unfairly

Not sure thats what happening here. I'm pointing out your flaws Team Athiest ain't having it, this is just a predictable course of events. With the exception of Shaky of course but y'know.....

lol, seriously, eat a dick

Originally posted by inimalist
lol, seriously, eat a dick

Sure but just so there isn't any misunderstanding. My personal opinion is that I don't think that there is enough scientific evidence to support the paranormal, so really you not believing in the paranormal isn't the issue.

My problem with you is that you don't understand that you're athiestic viewpoint isn't inherently rational and in some case you haven't even done the reasearch, in some cases I could make a better argument against the paranormal than you can.

Alot of the time I'm just trying to point out how certain viewpoints aren't productive to science.

http://cdn4.diggstatic.com/story/cool_story_bro_pic/o.png