At the end of April Lamar S. Smith, Republican chair of the congressional committee on Science, Space, and Technology, made headlines first by drafting legislation aimed at reforming the National Science Foundation’s merit review process, and then by writing a letter to the Acting Director of the NSF asking for details about the review of six proposals, all from the social sciences.
Absent context, Smith’s request seems entirely reasonable:
Based on my review of NSF-funded studies, I have concerns regarding some grants approved by the Foundation and how closely they adhere to NSF’s “intellectual merit” guideline. To better understand how NSF makes decisions to approve and fund grants, it would be helpful to obtain detailed information on specific research projects awarded NSF grants.
Smith’s committee is tasked with holding the NSF accountable to its merit review process, and thus it makes sense to request details about how proposals are judged in order to facilitate proper oversight. What could be wrong with seeking to understand the process better? But this request occurred in the context of congress already gutting the funding for political science research from the NSF and comments from fellow Republican Bill Posey that, “It’s just hard to conceive how those are important to our national security or our national interest.” These are not disinterested politicians doing their due diligence.
First to respond was the ranking Democrat on the committee, Eddie Bernice Johnson. Johnson accused Smith of taking “the first step on a path that would destroy the merit-based review process at NSF” and charged that “Interventions in grant awards by political figures with agendas, biases, and no expertise is the antithesis of the peer review process. By making this request, [Smith is] sending a chilling message to the entire scientific community that peer review may always be trumped by political review.”
This exchange set off a flurry of outraged blog posts from bloggers and news organizations, and even warranted a mention from President Obama. To a large extent I agree with the criticisms levelled by Johnson and others—we don’t want to live in a world where politicians, whether or not they have sufficient expertise to do so, are making judgments about the merit of individual research projects. However, I think many, rushing to reflexive defines of the NSF and its peer review process go too far and fail to acknowledge legitimate concerns about the accountability of academics to society, particularly academics directly funded by public institutions such as the NSF. Further, many of these defences promote an untenable view of science and peer review as objective and apolitical.
We’ve been here before
This is not the first time the social value of scientific research has been challenged. Many scientists at the 1931 Second International Congress of the History of Science in London were deeply impressed by the Soviet system of science, directly organized to solve social problemsh. Inspired by this and concerned about the ongoing depression of the 1930s, physicist J.D. Bernal argued that British science could better serve social ends, and even its own internal ends, if it were reformed along socialist lines. Bernal’s plan was to create hierarchically-organized research centres targeted at particular areas of science, ultimately answerable to political overseers. In response, chemist-turned-philosoper Michael Polanyi argued that science was an intricate spontaneously organized system, one that would inevitably fail given any attempts at centralized control. No matter how well meaning, proposals such as Bernal’s would backfire, destroying all possible social benefit of science in their attempts to improve it.
Director of the wartime Office of Scientific Research and Development Vannevar Bush echoed many of Polanyi’s sentiments in Science, the Endless Frontier, a report authored for President Roosevelt in 1945. Bush argued that scientific progress was essential for national health, national security, and the public welfare in general. The best way to promote scientific progress, he argued, was to provide funding to scientists while maintaining their autonomy:
The publicly and privately supported colleges, universities, and research institutes are the centers of basic research. They are the wellsprings of knowledge and understanding. As long as they are vigorous and healthy and their scientists are free to pursue the truth wherever it may lead, there will be a flow of new scientific knowledge to those who can apply it to practical problems in Government, in industry, or elsewhere (my emphasis).
Bush proposed the foundation of a new government agency to promote scientific progress, and in consequence the National Science Foundation was founded in 1950.
Perhaps the most cited predecessor to Smith is former Democratic senator William Proxmire, famous for his “Golden Fleece Awards” that highlighted what he judged to be the most egregious misuses of public funds. A frequent target for Proxmire was social science research funded by the NSF. Notable examples include spending $84,000 “to find out why people fall in love” in 1975, $46,100 to “study the effect of scantily clad women on the behavior of Chicago’s male drivers” in 1976, and $39,600 “to study ‘Himalayan mountaineering, Social change, and the Evolution of [the Buddhist] Religion among the Sherpas of Nepal’”. As far as I know, however, Proxmire never made any substantial efforts to interfere with the operation of the NSF. He was merely an outspoken critic, one who highlighted waste wherever he perceived it.
The NSF’s Merit Review Process
Smith has been criticized for presuming that he has the expertise to judge the merit of individual research proposals. Granting that he does not have the expertise to judge what the NSF calls “intellectual merit”, these critics miss that the NSF merit review process involves another component: broader impacts. Since 1997, peer reviewers have had to report on not just the traditional internal disciplinary value of proposed projects, but also their “broader impacts”. The NSF website explains:
Examples could include, among others: innovations in teaching and training (e.g., development of curricular materials and pedagogical methods); contributions to the science of learning; development and/or refinement of research tools; computation methodologies, and algorithms for problem-solving; development of databases to support research and education; broadening the participation of groups underrepresented in science, mathematics, engineering and technology; and service to the scientific and engineering community outside of the individual’s immediate organization.
Recently the NSF has introduced “Broader Impacts 2.0” which strengthen the assessment and reporting requirements:
The NSF’s revisions more explicitly integrate the Broader Impacts and Intellectual Merit Criteria, require a separate Broader Impacts section in grant proposals, and mandate a detailed account and assessment of a project’s broader impacts in grantee reports.
When this component of the NSF review process was introduced, many scientists were resistant and reviewers often either ignored or didn’t take it seriously. The accepted view in science, and academia, seems to be that internal, intellectual merit is all that really matters and the rest is just jumping through hoops. This attitude may be changing, but the indignant reactions to Smith’s query demonstrate just how entrenched it still is. It’s true that Smith probably isn’t able to judge whether the potential broader impacts claimed by these proposals are credible. However, I doubt this is what he was trying to do. Rather, he wanted to see what broader impacts were predicted, and as an elected representative he is qualified to judge whether those impacts are in his constituents’ best interests.
Up until very recently, scientists were either gentlemen of independent means conducting whatever research captured their interest or had to convince a wealthy patron to fund their investigations. There are of course many problems with such a funding model; it is elitist almost by definition. However I think something important has been lost in not requiring scientists who rely on others’ financial resources for their work to convince their patrons of the value of that work. Although Smith may be unable to exercise competent judgment of any of the questioned projects, I think it is reasonable to hold that the onus should be on scientists to convince those such as Smith, who have been elected to represent the public interest, of the value of their work.
As ably explored by Mark Brown in Science in Democracy, the charge of “politicizing science” has been levelled by those of all ideological persuasions. It is not a tactic restricted to the left, the right, or any other political faction. No science is removed from politics. All knowledge is entwined with values. But there are still valid concerns when politicians directly involve themselves with the production of scientific knowledge.
First, we must be concerned when those such as Smith and Posey equate social benefit solely in terms of national security or economic benefit. Canadian academics have especially suffered from this overly-narrow interpretation of the social good. Perhaps the most vivid example of this is the federal budget’s directive that scholarships granted by the Social Sciences and Humanities Research Council (SSHRC) should focus on “business-related degrees.” Partly academics are the victims of history here; proponents of government funded scientific research such as Vannevar Bush and Kenneth Arrow emphasized the economic potential of scientific research and argued that this potential wouldn’t be fully realized without government assistance. It was the potential of social sciences such as sociology and psychology to help win the Cold War that led to dramatic increases in their public funding.
Second, we must be concerned that politicians may judge research projects unsuitable not because they are irrelevant to the social good but because their findings might be ideologically problematic. When Republicans complain about the Liberal Elite in Ivy League humanities departments, it’s not because they deem their work irrelevant to the social good—it’s because they see them as a threat, contributing to the moral corruption of society. In a letter to NSF director Subra Suresh this March, Republican Senator Tom Colburn wrote, “Studies of presidential executive power and Americans’ attitudes toward the Senate filibuster hold little promise to save an American’s life from a threatening condition or to advance America’s competitiveness in the world.” This does not seem like Senators solely concerned with saving the public money, but Senators unhappy with the potential results of research.
Finding a Balance
There is a delicate balance to be struck between ensuring that public funds are spent in the public interest and shielding important research from parochial political interests. A potential guide to striking this balance comes from Heather Douglas’s Science, Policy, and the Value-Free Ideal. Douglas argues that all scientific judgments involve values—there is no such thing as “value-free science”—but that values should enter the scientific process prospectively, not retrospectively. That is, in setting significance thresholds for what will count as a “significant result”, it is appropriate to consider social, political, or moral factors. This must be done, however, before the results are known. Otherwise there is the danger that we will create a moving target, and will conveniently set thresholds to avoid ideologically troublesome results. Similarly, it seems appropriate for political bodies such as congressional committees to only prospectively set criteria for funding research projects, not retrospectively set criteria in judging existent proposals. This is pretty much the current practice. The “Broader Impacts 2.0” criteria resulted from Congressional worries over the previous set of criteria’s lack of effectiveness in demonstrating the social value of research.
However, the problem remains that the review of individual proposals is done solely by academic peers. Although scientists may gradually be taking broader impacts criteria more seriously, as long as the review process is completely internal it will be almost impossible to ensure proper oversight. After all, peer reviewers are just as susceptible as politicians are to the value biases that Douglas described. Since we don’t want to directly involve Congressional representatives, I think the best solution is to involve lay juries in the process. A third pillar of the NSF review process could be a randomly selected jury of ordinary citizens who evaluate the potential social impact of prospective research projects. In this, the onus would be on scientists to explain their research to members of the public, not on the public to try to understand complex scientific jargon. This harkens back to the patronage relations of past scientific eras and I think captures an important aspect of those relations that has been lost in the current system.
It might be objected that it is not simply scientific jargon that is too complex for the public to understand, but that the underlying scientific concepts themselves, and how they relate to the rest of the science in their field, are too difficult for non-experts to comprehend. We should view such objections skeptically. Studies such as Steven Epstein’s Impure Science demonstrate just how proficient members of the public can become in evaluating scientific research. Citizens’ Assemblies, such as the Citizens’ Assembly on Electoral Reform in BC have also demonstrated the ability of the public to understand and evaluate complex proposals. Criminal juries are constantly asked to evaluate complex technical arguments with far more at stake than the funding of scientific research. Will non-expert members of the public be able to evaluate the technical merits of research projects as well as experts? No, of course not, but I am not proposing to replace expert judgment with public juries, but rather to supplement it.
In conclusion, although there is some merit to the concerns of Republicans such as Smith and Posey, I agree that they should not be exercising direct judgment over scientific research proposals. All science involves value judgment, but not all value judgments are appropriate at all points in the scientific process. The best role for Congress is to set prospective criteria for funding. However, this is not sufficient to ensure proper democratic oversight of publicly funded research. For that we need to involve members of the public directly, and the best way to do that is through lay juries. I think this would force scientists to be accountable to society for their research funding without subjecting it to political meddling of the wrong sort.