In our post-Kuhnian times as philosophers and historians of science, it is important to remember that interests besides the holy, disinterested pursuit of Truth, Progress and Knowledge affect the development of scientific thought and practice. There is now a blossoming field of study devoted to understanding how scientific communities, personal and social values, and similar human concerns shape scientific developments. Recently, some people have taken another, quite pressing, (possibly?) nonscientific concern that affects the development of science: money.
This post is inspired by a group of people who are doing what might be called the Philosophy of the Business of Science: they are investigating the practicalities of funding and grant processing using philosophical tools for the purpose of trying to figure out how these aspects of the scientific endeavor shape changes in the landscape of science, and whether there are better systems that could, if implemented, improve this landscape. I had a chance to see a symposium on peer review in the sciences put on by a group of these people at the 2010 Philosophy of Science Association (PSA) biennial meeting in Montreal last weekend, and I want to share some of what I learned there with you. I also want to tell you a little bit about another pair of philosophers whose project is aimed more directly at funding structures in the sciences. But before I get into the gory details, let me orient you to the problems at hand.
Here are a few stories about money flow from the history of science:
- Leibniz’s early travels to Paris, where he met Huygens and became interested in the math that would lead to the calculus, were funded by Baron von Boineburg, and came with instructions to carry out the Duke’s political agenda.1
- The development of nuclear science and the Manhattan Project relied heavily on an influx of military money.
- In 1932, Warren Weaver was put in charge of assigning grants for the natural sciences division of the Rockefeller Foundation, one of the largest private grant institutions in the United States. He decided not to fund any project unless it was related in some way to biology or psychology. One of the grant proposals he rejected belonged to Linus Pauling, who then revised his grant to work on organic chemistry and the structure of proteins. This work on the structure of proteins led to the discovery of the alpha-helix protein structure, which paved the way for Watson, Crick, and Franklin’s discovery of the double helix structure of DNA. (For more information, see Thomas Hager, Force of Nature, pp. 181-199 and the Structure of DNA series at paulingblog.wordpress.com.)
- Advocates and naysayers to stem cell research, recreational controlled substances, carbon taxes, and medical uses of nanotechnology (among others) all pour money into their various political causes. This money is then sometimes used to fund research into the benefits and drawbacks of each of these causes. It is a common news story that the findings of research sponsored in this way has been impacted by the promise of financial gain. For an example, see Deborah Barnes’ and Lisa Bero’s report on conflict-of-interest and Tobacco-funded research through the Center for Indoor Air Research
This list is intended to demonstrate that money flow, or funding structure, has an impact on how scientific ideas develop. The question now is how, as philosophers and historians, are we supposed to analyze the impact of funding structures on the development of science?
One way to perform the analysis is to play the counterfactual-history game, trying to answer questions like:
- Would we have calculus without Boineburg’s political designs?
- Would we have the atom bomb without the 1943 withholding tax on wages act?
- Would we have the double helix if Weaver hadn’t forced Pauling to work on organic projects?
This method can produce entertaining stories and provide some insight into the importance of funding structures, and it is guiding a recent movement in historiography known as counterfactual history.2 But it has three big drawbacks. First, it does not produce concrete data and can be reliant on the intuition of the author, who is performing a necessarily speculative, if not uninformed, analysis. Second, it is not projectible, and so it cannot provide much guidance about how to consciously shape funding structures in the future. Third, it is not systematic, so it cannot give us a bird’s-eye view of how funding structures have shaped science as a whole; rather, it is confined to individual events and stories.
Recently, Kyle Stanford (University of California–Irvine) and Kevin Zollman (Carnegie Mellon University) have proposed an alternative means of analyzing funding structure that does not face the drawbacks of the speculative-history method. Rather than just investigating individual cases, they are engaging in a more systematic study of the professionalization of science. This includes a historical analysis of the peer review process through which most contemporary scientists receive the majority of their funding, as well as a formal model based on game theory of changes in funding structure. Their project is concerned with the institutionalization of scientific theories and ideas, and they aim to investigate whether today’s peer-review processes and other professional-incentive structures inhibit the introduction of particularly creative or revolutionary ideas into the scientific community.
It is no industry secret that some scientists oppose the peer review process because they believe it hinders their creativity as researchers, as well as taking up a sizeable percentage of their time. Sentiments like those of Luis Alvarez that “no group of peers would have approved my building the 72-inch bubble chamber,” for which he won the 1968 Nobel Prize in Physics, are not at all uncommon. But until now, they have been quite difficult to analyze. Stanford and Zollman are using history and game theory to try to pin down the nature of peer review and seek improvements to the current design. The members of the symposium on peer review at the PSA conference are taking a different set of approaches. Here’s a little play-by-play:
The symposium began with an overview orientation to the process of peer review in the sciences as it stands today. The history of peer review is an interesting one, and Ed Hackett (Cornell University) took care to point out that today’s peer review is quite different from the activity that might have been called peer review during the scientific revolution. There are two main differences to take away from Hackett’s detailed presentation. First, something like peer review happened throughout the 17th-19th centuries in scientific societies, but it was a meeting of minds to discuss and evaluate the results of past scientific research. Today, on the other hand, much peer review consists in evaluating plans for future research. Second, evaluation of results in scientific societies did not have all that much impact on money flow for the gentlemen scientists of earlier centuries. Today, on the other hand, whether a graduate student is hired or a laboratory built can frequently depend on the results of a peer-reviewed grant application—the evaluation of one scientist’s plans by a group of her peers.
Hackett’s points in hand, Fred Kronz (National Science Foundation) gave an insider’s perspective on some of the problems concerning the current state of the peer review process. Among other things, he talked about the benefits and drawbacks of sharing identities of reviewees with reviewers: reviewers may not be able to accurately assess a reviewee’s potential to do quality research without knowing who the person is, but they are inevitably biased once they know who the person is. Kronz’s research on how this bias works will interest philosophers, psychologists, and grant-writers alike. And his conclusion that this bias is inescapable is a convincing, if mildly depressing, one. Luckily, Kronz has a proposal for a hybrid review system where some but not all reviewers know the identity of the reviewees.
Carole Lee (University of Washington) spoke next, and she gave the best talk I saw in the whole conference. Rather than try to review the incredible volumes of research that went into her presentation, I’ll try to summarize briefly by saying that she demonstrated, using utterly fascinating psychological and anthropological evidence, that we need to figure out why groups of reviewers are so likely to disagree about the merit of a particular project. She framed this demonstration in Kuhnian terms, pointing out that while reviewers may agree on what is valuable in scientific research, this does not imply that they will automatically agree on the merit of a particular project. While personality differences may account for some of this disagreement, differences in expertise and in how they apply evaluative criteria also play a major role. It is a fairly straightforward point when put this way, but its implications are staggering: figuring out what reviewers think is good and how they determine which proposals count as good is one of the biggest challenges that faces any theory of peer review or any improvements that can be made to the peer review process. Lee suggested that current peer-review processes might do well to take this diverse range of divergences into account, and that philosophers could help by creating models of peer-review processes that do just this.
Since my fawning over Lee’s presentation has caused me to overrun my editors’ desired word count, I’ll try to wrap up quickly. Helen Longino (Stanford University) finished out the symposium by considering how peer review, as a social process, functions in larger process of academic discourse and generates objectivity-via-intersubjectivity. I was quite taken by a study she mentioned that used 21st-century dialogue techniques to generate peer review and evaluate scholarly ideas. In the study, which you can read about here, Shakespeare Quarterly posted reviewers’ comments online along with submitted articles and readers were invited to assess both the articles and the reviews. Readers’ assessments were factored into the editors’ publication decisions.
In sum, the symposium, and Stanford and Zollman’s work, exemplify a new and exciting realm of philosophical discourse, this philosophy of the business of science. The issues these people are addressing are central to shaping the development of science in the near and distant future, because the structure of peer review processes in particular, and funding of the sciences more generally, will unquestionably impact what research does and doesn’t happen.