How do we mistake biological plausibility for clinical relevance?

The case for Blindness to Marginal Utility

In the past two decades, we have witnessed a transformation in critical care research. Small, unicentric, low-budget, researcher-oriented trials gave place to massive, multicentric, methodologically sound, robust studies. Nevertheless, results never came. Our specialty is evolving by subtraction, not addition, of therapies. There must be a fundamental problem in the way we conceive research hypotheses. How do we mistake biological plausibility for clinical relevance?¹

The problem

It is notorious that randomized clinical trials (RCT) in critical care are prone to return negative results, especially if the study outcome is mortality. A bias towards treatment effects overestimation is a leading cause of negative trials. Prior estimates of treatment effect in recent RCTs commonly reached a 10% reduction in mortality, an estimate proved optimistic when confronted with the actual studies results and prior estimates provided by clinicians. Authors have even raised the hypothesis that most therapies for critical illness may be, in fact, inefficacious.

This article introduces a conceptual framework to discuss the plausibility of finding efficacious therapies in a multi-comorbidities, multi-intervention environment.

The Additive Paradigm

In 1946 a British Medical Research Council special committee gathered to plan clinical trials of streptomycin in tuberculosis. In the period of a few years before the meeting, researchers had built a strong case for the biological plausibility of successfully treating tuberculosis with the newly discovered antibiotic. They showed potent in vitro inhibition of the bacilli and promising results in animal models of tuberculosis. They also advanced the drug to early clinical trials, finding encouraging results. The 1948 British Medical Journal article reporting the subsequent larger clinical trial is something to be read by today's researchers and clinicians to set higher standards. The quality and sheer openness of the paper struck me. Anyway, the trial of adding streptomycin to bed rest versus bed rest alone for pulmonary tuberculosis treatment had a plausible potential for producing a noteworthy reduction in mortality.

Now, let us examine a trial including patients with Acute Respiratory Distress Syndrome (ARDS) already receiving (1) oxygen, (2) positive pressure ventilation, (3) positive end-expiratory pressure (PEEP), (4) a high-PEEP, low tidal volume strategy, (5) vasopressors as necessary, (6) hemodialysis if indicated, (7) protocol-directed sedation, (8) early neuromuscular blockade, (9) a restrictive fluid strategy, (10) prone positioning if severely hypoxemic, (11) antibiotics as necessary, (12) respiratory care, (13) prophylactic measures for nosocomial infections, (14) prophylactic measures for venous thrombosis, (15) early mobilization, (16) nutritional support, and (17) glycemic control. The RCT protocol randomizes these patients to receive or not receive a candidate therapy. Considering the combined effect of concurrent therapies on mortality, is there a remaining effect to be reaped? Can the addition of the 18th intervention cause a 10% decrease in mortality? The Additive Paradigm is the assumption that each added therapy bears a significant effect and that researchers should, therefore, test the efficacy of additional therapies.

Marginal Utility

The hypothetical ARDS trial and the early streptomycin trial may have the same design - both enrolled a specific group of patients to test the effect of an additional therapy versus the current treatment without the additional therapy. But they differ in one point: the ARDS trial was performed under the Additive Paradigm.

In order to study the efficacy of adding new therapies we can borrow the concept of Marginal Utility from Economics. Utility refers to the benefit or satisfaction resulting from a unit of a good or service. Marginal utility is the benefit of an additional unit of the good or service. Consider the benefit of having a glass of water after three days in the desert. The second glass would also be very satisfactory. The marginal (residual) utility (benefit) of the 10th straight glass would be close to zero or even negative, causing more harm than good. The returns are incrementally smaller, i.e., diminishing, as units of the good or service add. In the research, as well as in the clinical scenario, the utility of every new therapy is expected to be incrementally smaller. In the limit, the effect becomes insignificant.

Reductio ad absurdum

The idea above is graphically demonstrated in Figures A, B, and C. Consider a graph with a y-axis representing mortality (or another outcome) and an x-axis designating the number of therapies. Let the axes cross at x=0 and y=0. Under the additive paradigm, the addition of therapies translates into a simple equation [y = e1 + e2 + e3 +... ei] where "y" stands for the net clinical effect, "e" is the effect for each therapy already present in both the intervention and control groups and ''ei" the assumed effect size of the intervention under study. Reductio ad absurdum, the Additive Paradigm eventually reaches absurd consequences such as less than 0% mortality as treatments keep adding (Figure A).

To avoid overestimation, an estimate of the treatment effect of additional interventions should observe that as the number of interventions increases, marginal utility shrinks. Hence, the marginal utility of the 18th intervention sits between the marginal utility of the 17th intervention and zero. The same applies whenever other intervention stacks up. The process fits best as an exponential function with any exponent below one and above zero, e.g. [y = iˆ0.45], where “y” stands for the net clinical effect and "i" stands for the number of therapies (Figure B)

The Edge of Irrelevance