This is originally a comment made on the AI Philosophy Facebook group, about the book Evidence for Psi: Thirteen Empirical Research Reports edited by respected AGI researcher Ben Goertzel.
When I commented earlier that “we are now looking for Bigfoot”, I meant to criticize scientific research aimed at finding evidence for widespread beliefs. Scientific discoveries in general are counter-intuitive and are driven by a handful of specialists with better knowledge than the general population; hypotheses that are correctly formed by scientists are not what the general public believes in, whatever that is.
While I acknowledge that promising to do research on naive beliefs has been on occasions a good way of obtaining funding and getting useful research done (alchemy, astrology, search for extra-terrestrial intelligence, …), I do not want to spend time debunking faith-driven science, however serious and honest it is.
My personal experience as a post-doc studying the structure of proteins was the following: the boss had the strong impression that the 3D structure of bacterial proteins was more compact, less irregular and fuzzy than their homologs found in eukaryotes. That was the belief. I looked at a lot of structures and couldn’t see anything, other than known results about the eukaryotic sequences having longer inserts than their bacterial homologs. I looked at a number of metrics that would show differences in the compactness of those proteins but no difference was found between eukaryotes and bacteria. However, using those tools, I found strong differences between high-quality structures determined by X-ray crystallography and low-quality structures obtained by X-ray crystallography. Another technique for determining 3D structures of proteins, NMR, however did not come with a way of knowing the “resolution” of the structure as it was the case for crystallography. Well, it turned out that the metric I came up with while trying to measure how loose a protein structure was, simply showed that NMR structures were very loose, just like the set X-ray structures of low resolution that were considered unreliable. In a set of 50% of X-ray structures and 50% of NMR structures, our new metric would guess correctly in 90% of the cases which technique was used to determine the 3D structure of the protein - just by a geometric analysis on the positions of the atoms.
Unfortunately, the friendly lab next door was focused on NMR analysis and structure determination of proteins. The results I had obtained were highly inconvenient, as they demonstrated both that (1) the initial belief on difference between bacteria and eukaryotes was unfounded and (2) that their experimental technique to determine 3D structures in the first place was greatly limited. My boss delayed the paper forever; I eventually left the lab and ended up in the software industry.
In the end, I would recommend against trying to find evidence for other people’s beliefs, however numerous, powerful, nice, serious or respected these people are.