Tag Archives: research methods

(Following an interesting interchange on Twitter with Cameron Brick and Dave Pietraszewski about essentialism in psychology and the hazards it creates for scientific progress, I thought I would re-post this 2017 blog entry, which might be useful and/or interesting to some, and perhaps even entertaining for an extremely small subset of that small group. I daresay the concerns I raise here aren’t any less concerning in 2021. ~M)

TWO years ago, I idly surfed my way to a harmless-seeming article from 2004 by Denny Borsboom, Gideon Mellenbergh, and Jaap van Heerden entitled The Concept of Validity. More than a decade had passed since its publication, and I had never heard of it. Egocentrically, this seemed like reason enough to surf right past it. Then I skimmed the abstract. Intrigued, I proceeded to read the first few paragraphs. By that point, I was hooked: I scrapped my plans for the next couple of hours so I could give this article my complete attention. This was a paper I needed to read immediately.

I’ve thought about The Concept of Validity every day for the past two years. I have mentioned or discussed or recommended The Concept of Validity hundreds of times. My zeal for The Concept of Validity is the zeal of an ex-smoker. The concept of validity in The Concept of Validity has led to a complete reformatting of my understanding of validity, and of measurement in general—and not just in the psychological sciences, but in the rest of the sciences, too. And those effects have oozed out to influence just about everything else I believe about science. The Concept of Validity is the most important paper you’ve probably never heard of.*

The concept of validity in The Concept of Validity is so simple that it’s a bit embarrassing even to write it down, but its simplicity is what makes it so diabolical, and so very different from what most in the social sciences of have believed validity to be for the past 60 years.

According to Borsboom and colleagues, a scientific device (let’s label it D) validly measures a trait or substance (which we will label T), if and only if two conditions are fulfilled:

(1) T must exist;

(2) T must cause the measurements on D.

That’s it. That is the concept of validity in The Concept of Validity.

This is a Device. There are invisible forces in the world that cause changes in the physical state of this Device. Those physical changes can be read off as representations of the states of those invisible forces. Thus, this Device is a valid measurement of those invisible forces.

What is most conspicuous about the concept of validity in The Concept of Validity is what it lacks. There is no talk of score meanings and interpretations (à la Cronbach and Meehl). There is no talk of integrative judgments involving considerations of the social or ethical consequences of how scores are put to use (à la Messick). There’s no talk of multitrait-multimethod matrixes (à la Campbell and Fiske), nomological nets (Cronbach and Meehl again), or any of the other theoretical provisos, addenda, riders, or doo-dads with which psychologists have been burdening their concepts of validity since the 1950s. Instead, all we need—and all we must have—for valid measurement is the fulfillment of two conditions: (1) a real force or trait or substance (2) whose presence exerts a causal influence on the physical state of a device. Once those conditions are fulfilled, a scientist can read off the physical changes to the device as measurements of T. And voila: We’ve got valid measurement.

Boorsboom and colleagues’ position is such a departure from 20th century notions of validity precisely because they are committed to scientific realism—a stance to which many mid-20th-century philosophers of science were quite allergic. But most philosophers of science have gotten over their aversion to scientific realism now. In general, they’re mostly comfortable with the idea that there could be hidden realities that are responsible for observable experience. Realism seemed like a lot to swallow in 1950. It doesn’t in 2017.

As soon as you commit to scientific realism, there is a kind of data you will prize more highly than any other for assessing validity, and that’s causal evidence. What a realist wants more than anything else on earth or in the heavens is evidence that the hypothesized invisible reality (the trait, or substance, or whatever) is causally responsible for the measurements the device produces. Every other productive branch of science is already working from this definition of validity. Why aren’t the social sciences?

For some of the research areas I’ve messed around with over the past few years, the implications of embracing the concept of validity in The Concept of Validity are profound, and potentially nettlesome: If we follow Borsboom and colleagues’ advice, we can discover that some scientific devices do indeed provide valid measurement, precisely because the trait or substance T they supposedly measure actually seems to exist (fulfilling Condition #1) and because there is good evidence that T is causally responsible for physical features of the device that can be read off as measurements of T (fulfilling Condition #2). In other areas, the validity of certain devices as measures looks less certain because even though we can be reasonably confident that the trait or substance T exists, we cannot be sure that changes in T are responsible for the physical changes in the device. In still other areas, it’s not clear that T exists at all, in which case there’s no way that the device can be a measure of T.

I will look at some of these scenarios more closely in an upcoming post.

Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111, 1061-1071.

*Weirdly, The Concept of Validity does not come up in Google Scholar. I’ve seen this before, actually. Why does this happen?

Human Oxytocin Research Gets a Drubbing

3 Replies

There’s a new paper out by Gareth Leng and Mike Ludwig1 that bears the coy title “Intranasal Oxytocin: Myths and Delusions” (get the full text here before it disappears behind a pay wall) that you need to know about if you’re interested in research on the links between oxytocin and human behavior (as I am; see my previous blog entries here, here, and here). Allow me to summarize some highlights, peppered with some of my own (I hope not intemperate) inferences. Caution: There be numbers below, and some back-of-the-envelope arithmetic. If you want to avoid all that, just go to the final paragraph where I quote directly from Gareth and Mike’s summary.

brain-OTFig 1. It’s complicated.

  1. In the brain, it’s the hypothalamus that makes OT, but it’s the pituitary that stores and distributes it to the periphery. I think those two facts are pretty commonly known, but here’s a fact I didn’t know: At any given point in time, the human pituitary gland contains about 14 International Units (IU) of OT (which is about 28 micrograms). So when you read that a researcher has administered 18 or 24IU of oxytocin intranasally as part of a behavioral experiment, bear in mind that they have dumped more than an entire pituitary gland’s worth of OT into the body.
  2. To me, that seems like a lot of extra OT to be floating around out there without us knowing completely what its unintended effects might be. Most scientists who conduct behavioral work on OT with humans think and of course hope that this big payload of OT is benign, and to be clear, I know of no evidence that it is not benign. Even so, research on the use of OT for labor augmentation has found that labor can be stimulated with as little as 3.2 IU of intranasal OT during childbirth by virtue of its effects on the uterus. This is saying a lot about OT’s potential to influence the body’s peripheral tissues because that OT has to overcome the very high levels of oxytocinase (the enzyme that breaks up OT) that circulate during pregnancy. It of course bears repeating that behavioral scientists typically use 24 IU to study behavior, and 24 > 3.2.2
  3. Three decades ago, researchers found that rats that received injections of radiolabeled OT showed some uptake of the OT into regions of the brain that did not have much of a blood brain barrier, but in regions of the brain that did have a decent blood brain barrier, the concentrations were 30 times lower. Furthermore, there was no OT penetration deeper into the brain. Other researchers who have injected rats with subcutaneous doses of OT have managed to increase the rats’ plasma concentrations of OT to 500 times their baseline levels, but they found only threefold increases in the CSF levels. On the basis of these results and others, Leng and Ludwig speculate that as little as 0.002% of the peripherally administered OT is finding its way into the central nervous system, and it has not been proven that any of it is capable of reaching deep brain areas.
  4. The fact that very low levels of OT appear to make it into the central nervous system isn’t a problem in and of itself—if that OT reaches behaviorally interesting brain targets in concentrations that are high enough to produce behavioral effects. However, OT receptors in the brain are generally exposed to much higher levels of OT than are receptors in the periphery (where baseline levels generally range from 0 to 10 pg/ml). As a result, OT receptors in the brain need to be exposed to comparatively high amounts of OT to produce behavioral effects—sometimes as much as 5 to 100 nanograms.
  5. Can an intranasal dose of 24 IU deliver 5 – 100 nanograms of OT to behaviorally relevant brain areas? We can do a little arithmetic to arrive at a guess. The 24 IU that researchers use in intranasal administration studies on humans is equivalent to 48 micrograms, or 48,000 nanograms. Let’s assume (given Point 3 above) that only .002 percent of those 48,000 nanograms is going to get into the brain. If that assumption is OK, then we might expect that brain areas with lots of OT receptors could—as an upper limit—end up with no more than 48,000 nanograms * .00002 = .96 (~1) nanogram of OT. But if 5 – 100 nanograms is what’s needed to produce a behavioral effect, then it seems sensible to conclude that even a 24 IU bolus of OT (which, we must remember, is more than a pituitary gland’s worth of OT) administered peripherally is likely too little to produce enough brain activity to produce a behavioral change—assuming that it’s even able to get into deep brain regions.

Leng and Ludwig aren’t completely closed to the idea that intranasal oxytocin affects behavior via its effects on behaviorally relevant parts of the brain that use oxytocin, but they maintain a cautious stance. I can find no better way to summarize their position clearly than by quoting from their abstract:

The wish to believe in the effectiveness of intranasal oxytocin appears to be widespread, and needs to be guarded against with scepticism and rigor.

1If you don’t know who Gareth Leng and Mike Ludwig are, by the way, and are wondering whether their judgment is backed up by real expertise, by all means have a look at their bona fides.

2A little bet-hedging: I think I read somewhere that there is upregulated gene expression for oxytocin receptors late in pregnancy, so this could explain the uterus’s heightened sensitivity to OT toward the end of pregnancy. Thus, it could be that the uterus becomes so sensitive to OT not because 3.2 IU is “a lot of OT” in any absolute sense, but because the uterus is going out of its way to “sense” it. Either way, 3.2 IU is clearly a detectible amount to any tissue that really “wants”* to detect it.

*If you’re having a hard time with my use of agentic language to refer to the uterus, give this a scan.

 

The Myth of Moral Outrage

2 Replies

This year, I am a senior scholar with the Chicago-based Center for Humans and Nature. If you are unfamiliar with this Center (as I was until recently), here’s how they describe their mission:

The Center for Humans and Nature partners with some of the brightest minds to explore humans and nature relationships. We bring together philosophers, biologists, ecologists, lawyers, artists, political scientists, anthropologists, poets and economists, among others, to think creatively about how people can make better decisions — in relationship with each other and the rest of nature.

In the year to come, I will be doing some writing for the Center, starting with a piece I that has just appeared on their web site. In The Myth of Moral Outrage, I attack the winsome idea that humans’ moral progress over the past few centuries has ridden on the back of a natural human inclination to react with a special kind of anger–moral outrage–in response to moral violations against unrelated third parties:

It is commonly believed that moral progress is a surfer that rides on waves of a peculiar emotion: moral outrage. Moral outrage is thought to be a special type of anger, one that ignites when people recognize that a person or institution has violated a moral principle (for example, do not hurt others, do not fail to help people in need, do not lie) and must be prevented from continuing to do so . . . Borrowing anchorman Howard Beale’s tag line from the film Network, you can think of the notion that moral outrage is an engine for moral progress as the “I’m as mad as hell and I’m not going to take this anymore” theory of moral progress.

I think the “Mad as Hell” theory of moral action is probably quite flawed, despite the popularity that it has garnered among may social scientists who believe that humans possess “prosocial preferences” and a built-in (genetically group-selected? culturally group selected?) appetite for punishing norm-violators. I go on to describe the typical experimental result that has given so many people the impression that we humans do indeed possess prosocial preferences that motivate us to spend our own resources for the purpose of punishing norm violators who have harmed people whom we don’t know or otherwise care about. Specialists will recognize that the empirical evidence that I am taking to task comes from that workhorse of experimental economics, the third-party punishment game:

…[R]esearch subjects are given some “experimental dollars” (which have real cash value). Next, they are informed that they are about to observe the results of a “game” to be played by two other strangers—call them Stranger 1 and Stranger 2. For this game, Stranger 1 has also been given some money and has the opportunity to share none, some, or all of it with Stranger 2 (who doesn’t have any money of her own). In advance of learning about the outcome of the game, subjects are given the opportunity to commit some of their experimental dollars toward the punishment of Stranger 1, should she fail to share her windfall with Stranger 2.

Most people who are put in this strange laboratory situation agree in advance to commit some of their experimental dollars to the purpose of punishing Stranger 1’s stingy behavior. And it is on the basis of this finding that many social scientists believe that humans have a capacity for moral outrage: We’re willing to pay good money to “buy” punishment for scoundrels.

In the rest of the piece, I go on to point out the rather serious inferential limitations of the third-party punishment game as it is typically carried out in experimental economists’ labs. I also point to some contradictory (and, in my opinion, better) experimental evidence, both from my lab and from other researchers’ labs, that gainsay the widely accepted belief in the reality of moral outrage. I end the piece with a proposal for explaining what the appearance of moral outrage might be for (in a strategic sense), even if moral outrage is actually not a unique emotion (that is, a “natural kind” of the type that we assume anger, happiness, grief, etc. to be) at all.

I don’t want to steal too much thunder from the Center‘s own coverage of the piece, so I invite you to read the entire piece over on their site. Feel free to post a comment over there, or back over here, and I’ll be responding in both places over the next few days.

As I mentioned above, I’ll be doing some additional writing for the center in the coming six months or so, and I’ll be speaking at a Center event in New York City in a couple of months, which I will announce soon.

Why Not Use Cap and Trade to Reduce False Positives In Science? An Elaboration

1 Reply

This post is a longer-form treatment of the Cap and Trade idea for controlling false positives in science that Dave Kelly and I outlined in our brief letter, which appeared in this week’s issue of Nature. It provides more background and additional details that we simply couldn’t cover in a 250-word letter.

First, the background. For the past several years, as many readers are surely aware, a replication crisis has been roiling the sciences. The problem, quite simply, is that some (probably large) proportion of published scientific findings are false. Many remedies have been proposed for addressing the replication crisis, including (1) system-wide changes in how researchers are trained in statistics and research methods; (2) exhortations to greater statistical and methodological virtue among researchers; (3) higher editorial standards for journal editors and reviewers; and (4) journal reforms that would require more transparency from investigators about hypothesis formulation, research methods, data collection, and data analysis as a condition for publication.

Most of these remedies are sensible, but Nature has suggested here and here that NIH officials have been contemplating an even more radical measure: Some sort of audit system in which independent laboratories would be tasked with trying to reproduce recently published scientific results from particular fields. An audit-based system would have its merits, but a cap and trade system might work even better. Our proposal rests on the idea that false positives are a form of pollution: I call it false positive pollution.

False Positives are Pollution

False positives fit the standard economic definition of pollution: They impose opportunity costs on others when they are emitted into the environment. If all published research findings were correct (i.e., if the false discovery rate were zero), then any single conclusion from any single research paper (“Drug X is safe and effective,” say, or “Cap and trade systems reduce false positives in scientific literatures”) could form the basis for confident decision-making. You could read a published paper and then take action on the basis of its conclusions, knowing that those conclusions reflected true states of the world.

However, the more false positive pollution a literature contains, the more costly, on average, it becomes to make decisions on the basis of any published finding. The recent Tamiflu debacle provides a vivid case study: The reason drug companies, governments, and individuals got so excited about Tamiflu as a treatment for flu was that their decision-making was distorted by irreproducible research results. The Tamiflu misadventure features false positive pollution doing what it does best: imposing costs on others, to the tune of $20 billion in wasted public expenditures (not to mention the harm the drugs might have done to their consumers, and the opportunity costs associated with not pursuing possible alternatives).

Likewise, if a published scientific article led you erroneously to believe that a particular laboratory technique was a good way to manipulate some variable in your research, and then you went on to base your PhD work on that technique—only to find that it did not work for you (because it actually doesn’t work for anybody)—then false positive pollution would have caused you to devote time and resources to hocus-pocus rather than the pursuit of something that could have produced actual scientific knowledge. This is one of the costs of false positive pollution that should really bother graduate students, post-docs, and anyone who cares about their career development: Trainees with just as much scientific promise as any other end up wasting their valuable training time on illusions. False positive pollution sets careers back.

A cap and trade system might be useful for reducing false positive pollution in the same way that cap and trade systems have, over the past 45 years, helped to reduce sulphur dioxide, nitrogen oxide, lead additives in gasoline, and even over-fishing. Below, I outline some of the steps we’d need to undertake as a society to implement a cap and trade system to control false positive pollution.

Step 1: Measuring Existing Levels of False Positive Pollution

The first step forward could be to estimate how much false positive pollution is emitted annually, which would require independent replications of random samples of published findings from the prior year. What we would be trying to estimate is the proportion of published experiments, out of the 1.5 million or so that are published each year worldwide, whose results cannot be independently reproduced even when the original protocols are followed exactly. I rather admire the way this was done in the Many Labs Replication Project: Several lab directors agree on the experimental protocol [ideally in collaboration with the investigator(s) who originally published the study] and then go back to their labs and re-run the experiment. The results from all of their independent attempts to replicate are then statistically aggregated to determine whether the original result is a true positive or a false positive.

Expensive, yes, but don’t let the expense distract you for the moment. Good research takes money, and we’re already hemorraging money through the production of false knowledge (keep the image of those warehouses full of Tamiflu vividly in your mind). Why not invest in trying to understand how much money we’re actually wasting and what we might do about it?

Step 2: Determining An Optimal Level of False Positive Pollution

Once we had an estimate of much false positive pollution is emitted annually, we’d need to figure out how much false positive pollution we’d like to live with. A 100% pollution-free research literature would be nice. So would 100% pollution-free air. However, “100% pollution-free air” is an unrealistic goal. Compliance would be too expensive, and it would come with too many undesirable side effects. Likewise, a research literature that’s 100% free of false positive pollution sounds great, but that’s a goal that cannot be attained without adversely affecting the overall research enterprise. False positives are going to happen—even by scientists who have done their best to avoid them (after all, there is no such thing as a study with 100% statistical power). There must be some amount of false positive pollution we can tolerate.

One way to set an acceptable level of false positive pollution would be to measure the costs and benefits associated with the average false positive emission. How much money is wasted each time a researcher emits an erroneous “finding?” And how much would it cost to prevent such an event? These benefits and costs are likely to vary quite a lot from field to field, so I see good, plentiful work for economists here. In any case, with those data in hand, it should be possible to estimate the optimal amount of false positive pollution that we should be willing to tolerate—that is, the amount that maximizes society-wide benefits relative to costs.

But there’s actually a simpler way to set an acceptable level: Society tacitly endorses the idea that we can live with a 5% false positive pollution rate each time we accept the results of a study in which the p value was set at .05. That’s what p < .05 actually means: “In a world in which the null hypothesis is true, we’d only get results as extreme as those we obtained in this study in 5 out of 100 exact replications.” We could simply make a 5% FPP emissions rate our explicit society-wide ideal.

Step 3: Setting Goals

Once key stakeholders have agreed upon an acceptable annual level, whether that acceptable level is derived by measuring costs and benefits (as outlined above), or by the “5% fiat” approach, an independent regulatory body would be in a position to set goals (with stakeholder input, of course) for reducing the annual FPP emissions rate down to the acceptable level. (In the United States, the regulatory body might be the NIH, the NSF, or some agency that does the regulatory work on behalf of all of the federal agencies that sponsor scientific research; an international regulatory body might resemble the European Union’s Emissions Trading System.)

I’ll illustrate here with a simplified example that assumes a global regulatory agency and a global trading market. Let’s assume that the global production of scientific papers is 1,500,000 papers per year. Now, suppose the goal is to reduce the global false positive emission rate from, say, 50% of all research findings (I use this estimate here merely for argument’s sake; nobody knows what the field-wide FPP emission rate is, though for some journals and sub-fields it could be as high as 80%) to 5%, and we want to accomplish that goal at the rate of 1% per year over a 45-year period. (In our Nature correspondence, space limitations forced Dave and me to envision a move from the current emission levels to 5% emissions in a single year. The scenario I’m presenting here is more complex, but it’s also considerably less draconian.)

Our approach relies on the issuance of false positive pollution (FPP) permits. These permits allow research organizations to emit some false positive pollution, but the number of available permits, and thus, the total amount of pollution emitted annually, is strictly regulated. In Year 1, the Agency would distribute enough FPP permits to cover only 49% of the total global research output (or 1,500,000*.49 = 735,000 false positive permits). The number of permits distributed to each research-producing institution (universities are canonical examples of research-sponsoring institutions, as are drug companies) would be based on each institution’s total research output. Highly productive institutions would get more, and less productive ones would get fewer, but for all institutions, the goal would be to provide them with enough permits to allow a 49% emissions rate in Year 1. After the agency distributes the first year’s supply of FPP permits, it’s up to each individual research-sponsoring institution to determine how it wants to limit its false positive pollution to 49%. In Year 2, the number of permits distributed would go down a little further, a little further in the year after that, and so on until the 5% ideal was reached.

By the way, there are lots of ways to make the distribution process fair to small businesses, independent scientists, and middle-school science fair geniuses (including, for example, exempting small research enterprises and individuals, so long as the absolute value of their contributions to FPP are trivially small) so it’s not fair to dismiss my idea on the basis of such objections. Cap and trade systems can be extremely flexible.

Step 4: Monitoring and Enforcement

Once the FPP permits have been distributed for the year, the regulatory agency would turn to another important task: Monitoring. In the carbon sector, monitoring of individual polluters can be accomplished with electronic sensors at the point of production, so the monitoring can be extremely precise and comprehensive. In the research sector, this level of precision and comprehensiveness would be impossible. We’d have to make do with random samples of research-producing institutions’ research output from the prior year. (Yes; some research studies would be difficult to replicate because the experiment or data set is literally unrepeatable. Complications like these, again, are just details; they don’t render a cap-and-trade approach unworkable by any means). If the estimated FPP emission rate for any research-sponsoring institution substantially exceeded (by some margin of error) the number of FPP permits the institution possessed at the time, the institution would be forced to purchase additional permits from other institutions that had done a better job of getting their FPP emissions under control. If you, as a research institution, could get your FPP emissions rate down to 40% in Year 1, you’d have a bunch of permits available to sell on the market to institutions that hadn’t done enough to get their emissions under control. In a cap and trade system, there is money to be made by institutions that take their own false positive pollution problems seriously.

The Virtues of a Cap and Trade System

Cap and trade systems have many virtues that suit them well to addressing the replication crisis. Here are a few examples:

  • Cap and trade systems use shame effectively. On one hand, they enable us to clearly state what is bad about false positives in a way that reduces righteous indignation, shame-based secrecy, and all of the pathologies these moralistic reactions create. On the other hand, were we to make information about institutions’ sales and purchases of false positive permits publicly available, then institutions would face the reputational consequences that would come from being identified publicly as flagrant polluters. Likewise, permit-sellers would come to be known as organizations whose research was relatively trustworthy. These reputational incentives would motivate all institutions—even Big Pharma and universities with fat endowments, which could afford to buy all the excess permits they desired on the open market—to get their emissions problems under control.
  • Cap and trade systems don’t rely on appeals to personal restraint, which are subject to public goods dilemmas. (Fewer false positives are good for everyone, of course, but I’m best off if I enjoy the benefits of your abstemiousness while I continue polluting whenever I feel like it.) Cap and trade systems do away with these sorts of free-rider problems.
  • Cap and trade systems encourage innovation: Each research-sponsoring institution is free to come up with its own policies for limiting the production of false positives. Inevitably, these innovations will diffuse out to other institutions, increasing cost-effectiveness in the entire sector.
  • A cap and trade system would be less chilling to individual investigators than a simple audit-and-sanction system would be because a cap-and-trade system would require institutions, and not just investigators, to share in the compliance burden. Research-sponsoring institutions take the glory for their scientists’ discoveries (and the overhead); they should also share the responsibility for reform.
  • Most importantly; cap and trade systems reduce pollution where it is cheapest to do so first. All of the low-hanging fruit will be picked in the first year; and harder-to-implement initiatives will be pursued in the successive years. This means that we could expect tangible progress in getting our problems with false positives under control right away. Audit systems do not possess this very desirable feature.

Wouldn’t a Cap and Trade System Be Expensive?

Elizabeth Iorns estimated that it costs $25,000 to replicate a major pre-clinical experiment that involves in vitro and/or animal work. I don’t know that well-conducted laboratory-based behavioral experiments are that much cheaper (at least, once you’ve factored in the personnel time for running the study, analyzing the data properly, and writing up the paper). So all of those replications goal-setting and monitoring purposes are going to cost a lot of money.

But bear in mind, as I already explained, that false positives are expensive, too—and they produce no societal benefit. In fact, what they produce is harm. It costs as much money to produce a false positive as it does to produce a true positive, but the money devoted to producing a false positive is wasted. (If it’s true that the United States spends around $70 billion/year on basic research, then if even 10% of the resultant findings are false positives (which is almost surely a gross underestimate), then the U.S. alone is using $7 billion dollars per year to buy pollution). Also, Tamiflu. What if we used some of the money we’re currently using to buy pollution to make sure that the rest of our research budget is spent not on the production of more pollution, but instead, on true-positives and true-negatives—that is, results that actually have value to society?

Cap and Trade: Something For Everyone (In Congress)

Here’s the final thing I like about the cap-and-trade idea: It has something for both liberals and conservatives. (I presume that enacting a project this big, which would have such a huge impact on how federal research dollars are spent, would require congressional authorization, and possibly the writing of new laws, but perhaps I am wrong about that). Liberals venerate science as a source of guidance for addressing societal problems, so they should be motivated to champion legislation that helps restore science’s damaged reputation. Conservatives, for their part, like market-based solutions, private sector engagement, and cutting fraud, waste, and abuse, so the idea should appeal to them as well. In a congress as impotent as the 113th U.S. congress has been, can you think of another issue that has as much to offer both sides of the aisle?