-
-
Re: Big Science Is Broken
Good read, vt, thanks. -
Re: Big Science Is Broken
Originally posted by vt View Post
It's largely hogwash, and the author is clueless.
The irony seems to escape Mr. Gobry that the 'flaws' in the "existing scientific research paradigm" have been 'discovered' using, gasp, the existing scientific research paradigm. Worse, one of the biggest purported culprits, social sciences, has been applied to 'discover' these flaws. It's like saying: "Don't trust anything I say because it is all a lie."
As an example of failure in physics research, Gobry points to the retraction of an article on superluminal neutrinos. In my opinion, this retraction is an example of the system actually working: A group of researchers came across an experimental result that seemed to contradict known laws of physics. Try as they may, they were not able to resolve the discrepancy within their group. So, rather than sweep it under the rug, they announced the result to the broader community in hope that someone may offer an explanation consistent with said known laws of physics. And, indeed, someone did. Puzzle solved. System worked.
And what is Gobry's proposed resolution of the 'crisis?' Give the largest grant money to 22-years olds, and as they grow older, shift them to "harmless activities" such as teaching. So, first of all, who is to decide, and what is the basis for the decision as to who of the 22-years olds gets the grants? (We cannot afford to give large grants to all who ask for the money.) Should other tweens make the decision? Based on what? Twitter feed? And calling teaching a "harmless activity" is beyond stupid.
If you think there are problems with scientific research, look at journalism.
In reality, scientific research, including 'big science' is inextricably tied with the society and its values. If the society values $$ more than honor and integrity, as exemplified by its 'leaders' and 'community organizers,' then such values are unavoidably reflected in the quality of scientific research.Comment
-
Re: Big Science Is Broken
I'm with Jam on this one, 100%.Originally posted by Jam View Post
I'll just add a bit on why I think stories like this keep popping up, in the hopes that it helps the reader parse them in the future:
What so many journalists fail to understand about science is that the "peer-review process" is not, as many seem to assume, merely the review of research submitted for publication, which would thus end once the paper passes the editorial process, and is published. That may be how journalism works, but it is not at all what scientific publishing is like. In science, the editorial process is not there to catch errors, but to set minimum standards of evidence. The consequences of this difference are huge.
Publication in a technical journal merely indicates that work meets the minimum standard of evidence, and deserves to be READ by the broader community. But it is the community that has the burden of deciding for itself if the ideas are thought of as valid. The debate between opposing views in the literature thus IS part of the peer review process -- in fact, the most important part.
That's how, once something has been published, the peer-review continues. People try to reproduce results, or otherwise subject the publication to scrutiny. If they get different answers, they can then publish those. And back and forth, for as long as one likes, until a universal or near-universal consensus is reached. OF COURSE people will find problems in each others' publications, and lots of them! The need to find those problems is WHY the papers get published in the first place!
In other words, the fact that something has been published does NOT indicate that it has been verified, tested independently, or that it should be regarded as "true".* It just means that it has been deemed to be sufficiently well-supported that it contributes to the scientific conversation. In journalism, the implied claim is that what is published has been verified by fact-checkers and vetted by editors, but journalists extrapolate falsely into other areas when they assume the same holds in non-journalistic publications.
One consequence of this is that when something can't get published (at all) it's usually not because it's wrong, but because it is so obviously and overwhelmingly bad that it isn't even wrong:
Perhaps the experiment was obviously not conducted correctly, or an experiment that should have had a control was instead conducted without one, or essential data to reach the proposed conclusion was simply absent. While not all work makes it into the most prestigious journals of a given field, it has to be pretty awful indeed for it to be rejected from all journals in a field. And the scientific readers of each journal know that every publication is to be read with all the skepticism that this broad acceptance range demands. Published work is a suggestion of one interpretation concerning a given experimental outcome. Nothing more.
(This is why climate-change deniers or other conspiracy theorists are silly to complain of "bias" when they can't get published; there isn't even a "validation" step that could be used to insert that bias into the scientific publishing process. If you're not getting published anywhere, it's because a minimal standard for evidence isn't being met.)
Once one understand how this paradigm differs from journalism (where the publication of an article IS considered a final, verified, product) the confusion, and the "scandal" goes away on its own. Journalists see retractions and reversals as "problems" because in journalism, they are a sign of something terrible going wrong. ("You didn't get the fact-checking right! No excuse for that!") In science, these same actions are (as Jam indicated) signs that the system is working as intended. No publication is ever seen as "final" because more refinement can always be added to make it better. The scientific process is based on a continual updating of our understanding, and there should be no expectation that any given moment's snapshot can ever be "authoritative" into perpetuity.
So what's linked in the original post is simple: a journalist who doesn't understand the first thing about scientific publishing wrote yet another expose about "problems" in scientific publishing, thus exposing the shocking fact -- that he doesn't understand the first thing about scientific publishing.
To be fair, it cuts both ways. Sometimes a journalist or politician takes some random publication that sounds really cool, assumes that it therefore must be perfect gospel, and makes a big deal about the implications -- when that isn't at all justified. And other times they "discover" a larger debate raging in the literature, notice that the ideas contradict each other, and declare the fact that a debate exists at all to be a huge problem.
Neither mistake is a novel "expose". In fact, they're practically cliche. But they are based on the same misunderstanding about what a published article represents, and how to understand and weight its evidence.
Science is a process, which advances via the aggregate total of all publications, and the continuously evolving consensus of what they all signify. And it does advance because of -- not in spite of -- the fact that each individual publication will one day be supplanted with better knowledge. That is taken as a given.
Finding errors in the literature is thus not only not a problem, it is actually the solution. It is the central process of science itself.
* Footnote: To be strictly accurate, there is in fact one journal that I know of that does try to replicate the validity and yield of each proposed chemical syntheses in a different researcher's lab prior to publication, but it is a notable exception to the rule.Comment
-
Re: Big Science Is Broken
Excellent PostIt's the Debt, stupid!!Comment
For a concise, readable summary of iTulip concepts developed over the past 16 years and a vision of a challenging next decade and how to navigate it, read Eric Janszen's book "Post Catastrophe Economy".
Join the discussion of today's events with a wide range of professionals with an interest in economics and finance. Register to join our 50,000 plus member registered community from 78 countries today.
Subscribe to iTulip Select for access to the longest running, deep, accurate, and unvarnished macro economic trends analysis and forecasting available, since 1998.
- If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.
Working...
Tweet
Comment