Imagine a mid-career social psychologist who, in 2018, watched her lab's flagship longitudinal study lose a grant competition to a pilot proposal using a smartphone app. The reviewer comments praised the pilot's "clean" design, its "tractable" timeline, and its "clear" predicted effect. The longitudinal study, which followed 400 families over three years, was described as "ambitious but risky." This is a hypothetical scenario, but it reflects a pattern reported by several early-career researchers in interviews. One postdoc, who asked to remain anonymous, described a similar pivot: their original project planned to measure daily mood, stress, and social interactions using a diary method over six months. Reviewers praised the ecological validity but worried about attrition and compliance. The postdoc revised the proposal to use a two-week experience-sampling protocol with smartphone reminders. The scores improved, and the grant was funded. These stories illustrate a systemic trend: across behavioral science, the scoring rubric used by grant reviewers is quietly rewriting what counts as a legitimate research question, and the datasets that answer them.

The Rubric That Decides What Counts as Science

Grant review rubrics rarely mention scientific depth explicitly. They score proposals on significance, innovation, approach, investigator, and environment. But the approach section, weighted heavily, is where feasibility dominates. Reviewers ask whether the design can be completed in three years, whether the sample can be recruited, whether the measures are validated, and whether the analysis plan will yield interpretable results. Each of these questions pushes toward smaller, simpler, faster studies. A proposal that promises a 10-year cohort with multiple measurement waves is scored lower on feasibility than one that promises a 2-week online experiment with 500 participants.

The pressure is not malicious. Reviewers are overworked, often reading dozens of proposals in a cycle. They rely on heuristics. A clean pilot dataset signals that the investigator can execute. A complicated design raises the specter of unanticipated confounds and failed analyses. The result is a systematic bias: proposals that fit the rubric's implicit template are more likely to be funded, and that template favors studies that can be completed quickly with high statistical power.

Behavioral datasets shrink to fit this template. Longitudinal panels become cross-sectional snapshots. Multi-dimensional questionnaires are reduced to single-item scales. Observational measures are replaced by self-report checklists. The research question itself is often reframed to match the available data. A lab that once asked how childhood adversity shapes adult decision-making now asks whether a 10-minute priming task changes risk preference in undergraduates. Both are legitimate questions, but they are not the same question.

The incentives are structural. Tenure and promotion depend on publications, which depend on grants. A grant that funds a small, quick study produces three papers in three years. A grant that funds a large, slow study might produce one paper in five years, and only if the longitudinal data survives attrition. For a junior investigator, the choice is rational. But the aggregate effect is a scientific literature increasingly built on narrow, decontextualized findings.

Reproducibility Crisis Meets Funding Pressure

The pressure to produce quick, clean results collides with the reproducibility crisis that has gripped psychology and other behavioral fields. The Open Science Collaboration's 2015 effort to replicate 100 psychology studies found that only about 36 percent of replications produced significant results, compared with 97 percent of the original studies. That gap is often attributed to questionable research practices, but funding incentives play a role. When grants reward novelty and speed, researchers are more likely to engage in p-hacking, selective reporting, and HARKing (hypothesizing after results are known).

The rubric does not explicitly encourage these practices. It rewards "innovation" and "impact." But in practice, a novel, surprising result is easier to publish than a robust, incremental one. Reviewers and editors alike prefer findings that overturn conventional wisdom. The replication crisis revealed that many such findings were statistical flukes. Yet the funding system has not adapted. Reviewers still score proposals on the novelty of the hypothesis, not on the rigor of the design or the robustness of the anticipated effect.

Behavioral tasks become proxies for real-world phenomena. A lab might use a delayed-gratification task to measure self-control, a trust game to measure cooperation, or a Stroop task to measure cognitive control. These tasks are convenient, cheap, and easily administered online. But they capture only a thin slice of the construct they claim to measure. The gap between the task and the real-world behavior is often ignored, because the task produces clean data that fits the rubric's demand for interpretable results.

Some researchers argue that the reproducibility crisis is partly a funding problem. If grants rewarded replications, preregistration, and open science practices, the incentive structure would shift. But grant review rubrics rarely include these criteria. A proposal that promises to replicate a published finding is often scored lower on innovation. A proposal that preregisters its analysis plan is sometimes seen as less flexible, and therefore less likely to discover something new. The tension between novelty and rigor is built into the rubric.

One Lab's Data, Rewritten by a Scorecard

The lab I mentioned earlier is a composite based on interviews with several early-career researchers. One postdoc told me about a project that originally planned to measure daily mood, stress, and social interactions using a diary method over six months. The grant reviewers praised the ecological validity but worried about attrition and compliance. The postdoc revised the proposal to use a two-week experience-sampling protocol with smartphone reminders. The scores improved, and the grant was funded.

The postdoc's experience is common. Labs pivot to simpler paradigms because the rubric rewards them. Longitudinal measures are dropped because they require more effort, more time, and more statistical complexity. Sample sizes are recalculated to detect only large effects, because small effects require larger N, which costs more money. The rubric demands clean statistics, meaning analyses that are straightforward and unlikely to produce ambiguous results. This often means dichotomizing continuous outcomes, excluding outliers, or using convenient but questionable corrections.

The cost is scientific depth. A six-month diary study captures the ebb and flow of mood in response to real events. A two-week smartphone study captures a brief snapshot, possibly during an unusual period. The resulting dataset is narrower and less generalizable. But it is more likely to be funded, published, and cited. The lab builds its reputation on these quick studies, and the long-term questions are abandoned.

Not all labs feel this pressure equally. Established investigators with large grants and permanent positions can afford to take risks. Early-career researchers, adjunct faculty, and those at teaching-heavy institutions are most vulnerable. They must win grants to survive, and the rubric's feasibility criteria become a straitjacket. The result is a two-tiered system where the most creative, ambitious research is increasingly the province of a privileged few.

The Instrument That Measures What Funders Want

Alongside the pressure to simplify, new instruments have emerged that promise to measure behavior passively and at scale. Wearables, smartphone logs, and social media data offer a way to collect rich behavioral data without the burden of self-report or laboratory visits. These instruments are attractive to funders because they are cheap, scalable, and produce large datasets that can be analyzed with machine learning.

For example, a study on sleep and mood might use a fitness tracker to record sleep duration and a smartphone to capture daily mood ratings. The data are collected passively, reducing participant burden and attrition. The dataset is large and relatively clean, with timestamps that allow precise temporal analyses. Such a study is more feasible within a two-year grant cycle than a traditional sleep lab study with polysomnography.

But these instruments also narrow the scope of what is measured. A fitness tracker captures movement and heart rate, but not the subjective experience of fatigue. A smartphone log captures call duration and app usage, but not the content of conversations. The richness of human behavior is reduced to the signals that sensors can detect. This is not inherently bad, but it changes the questions researchers can ask. The instrument becomes the measure, and the measure shapes the construct.

Passive data also raise privacy concerns. For instance, a proposal to mine social media data for psychological traits might be scored highly on feasibility but could raise ethical questions about informed consent and data security. While grant review rubrics often include sections on human subjects protection, they may not always explicitly address the unique challenges of passive data collection, such as the difficulty of obtaining meaningful consent for data that is collected continuously and invisibly. As these instruments become more common, the field must grapple with the trade-off between convenience and depth.

What Gets Lost When Feasibility Drives Design

When feasibility drives design, context-dependent effects vanish. Many behavioral phenomena are highly sensitive to context: a finding that holds in a laboratory may not hold in a classroom, and a finding that holds in one culture may not hold in another. The pressure to use standardized tasks and homogeneous samples (often college students) strips away this context. The result is a literature that describes how a narrow slice of humanity behaves in artificial settings.

Social dynamics are oversimplified. A trust game with anonymous strangers online tells us little about how trust develops between colleagues, friends, or family members over time. A cooperation task in a lab with immediate payouts misses the long-term reciprocity that sustains real-world cooperation. These simplifications are not necessarily wrong, but they are incomplete. The rubric rewards designs that are easy to execute, and easy designs often abstract away the very complexity that makes behavior interesting.

Publication bias worsens. Journals are more likely to publish positive, surprising results than null or incremental findings. When the rubric favors novel, quick studies, the field produces a larger proportion of such studies, increasing the risk of false positives. The replication crisis is partly a consequence of this incentive structure. Replication studies, which are inherently less novel, struggle to get funded. The result is a self-reinforcing cycle: novel findings get published, replications fail, but the novel findings are already part of the literature.

Theory development is stunted. Theories are built on a foundation of varied, robust findings. When the literature consists of narrow, decontextualized effects, theories become fragmented and unmoored. A researcher trying to build a general theory of self-control must reconcile dozens of conflicting findings from different tasks and populations. The rubric that favors feasibility does not explicitly discourage theory, but it does discourage the kind of systematic, multi-method research that theories require.

Reforming the Rubric: Trade-offs and Counterarguments

Reform is possible, but it comes with trade-offs. Some funding agencies are experimenting with two-stage review processes, where the first stage evaluates the scientific question and the second stage evaluates feasibility. This separation allows reviewers to score a proposal's intellectual merit without being swayed by methodological simplicity. The idea is to fund the best questions, not just the easiest ones. However, a two-stage process adds time and administrative burden, and some argue that it could disadvantage early-career researchers who may not have the resources to develop detailed feasibility plans in the second stage.

Other agencies are encouraging exploratory pilot phases. A grant might fund a small, risky study specifically to test a novel hypothesis or a new measurement approach. If the pilot shows promise, the investigator can apply for a larger grant. This structure reduces the pressure to produce a "clean" dataset in the first round, allowing researchers to develop their methods and refine their questions. But critics point out that pilot funding can be seen as a way to underfund risky research, leaving investigators with insufficient resources to complete the work.

Preregistration is another tool. By requiring investigators to preregister their hypotheses, methods, and analysis plans, funders can reduce the incentive to p-hack. Some review rubrics now include a section on transparency, rewarding proposals that commit to open data and materials. This shift is gradual, but it is happening. The National Science Foundation and the European Research Council have both made open science practices a criterion in some calls. However, some researchers worry that rigid preregistration can stifle exploratory analyses, which are often the source of unexpected discoveries. A balance must be struck between flexibility and accountability.

Diverse review panels also help. When reviewers come from different subfields, with different methodological traditions, they are less likely to share a single implicit template of what a good proposal looks like. A sociologist might appreciate a qualitative approach that a psychologist would dismiss as "uncontrolled." A behavioral economist might value a field experiment that a lab-based researcher would see as too noisy. Diversity in review panels can counteract the homogenizing pressure of feasibility. But assembling diverse panels is not always straightforward, and reviewers may still defer to dominant paradigms.

Finally, valuing long-term infrastructure is essential. Some research questions simply require time and resources. Longitudinal cohorts, cross-cultural collaborations, and multi-site studies do not fit neatly into a three-year grant cycle. Funders can create separate funding streams for such infrastructure, with longer timelines and larger budgets. Yet such investments are expensive and may be seen as less efficient in producing immediate publications. The economics of research are always entangled with its content, and the same is true here.

These changes will not be easy. The current system is entrenched, and the incentives are powerful. But the cost of inaction is a behavioral science that is increasingly shallow, fragmented, and untrustworthy. Researchers who care about depth must advocate for rubrics that reward rigor, context, and long-term thinking. A single, patient observation can reshape a field, but only if the funding system allows it to happen. The future of behavioral science depends on whether funders can see beyond the bottom line of feasibility. A rubric that only counts what is easy to measure will produce a science of easy things. The hard questions, the ones that matter most, will be left unasked. That is a trade-off no scoring sheet can justify.