I will be talking a bit about the randomized clinical trial. As mentioned previously, I have no financial disclosures relevant to this topic and I haven't figured out a way to get paid to talk about it. But I do live the life and try to spread the message even while driving to and from work. So the hierarchy of evidence is a concept that underlies evidence-based medicine quite thoroughly and is maintained in its most formal form by the Center for Evidence-Based Medicine at Oxford. And when we talk about interventional studies, the highest level of evidence is a systematic review of a number of randomized clinical trials. A randomized trial or even an observational trial with dramatic therapeutic effect and so on. So the randomized trial sits at the top of the hierarchy of evidence and we are going to talk a little bit about why. So what are the fundamental characteristics of a randomized trial? It is a prospective comparison of best current therapy to an innovative treatment with random allocation between current therapy and innovative treatment and best possible bias control. A best possible bias control is not unique to the randomized trial. In every study we do, we want to try to minimize the likelihood that bias is affecting the result. But it's a critical part. So if we were going to construct an ideal way of evaluating a therapeutic intervention, we would want to minimize many forms of bias. We want to minimize the bias of time by using concurrent controls. We want to minimize the bias of observation by blinding observers and analysts. We want to minimize the bias of susceptibility that the two groups that we are comparing have different susceptibility to the outcome and this requires careful study design and equal application of our evaluation measures to both groups. But then we'd like to minimize the bias of unknown factors that might affect the outcome. And this is where randomization comes in and plays its unique role. It is the only technique we know of that equalizes the chance of both known and unknown factors that might influence the outcome of the study. In many ways it's magic. It allows you to control something you can't see, can't measure, and don't even know exists. So it's an incredibly powerful tool and that's why it elevates the randomized trial to the top of the evidence hierarchy. So it has all the characteristics of the best designed observational therapeutic research studies, but also controls for unknown sources of bias and that's why it's the gold standard for therapeutic evaluation. Well then what are the difficulties with the randomized trial? If it's the best thing, why don't we use it all the time? Well there are a number of issues to be considered and the first are ethical. Now there are some silly ethical arguments that are made like it's unethical to deny new treatment to anyone. It's unethical to use placebos when there's proven active treatment. It's unethical to substitute a random process for clinical judgment. We hear these arguments all the time, but the fact of the matter is that it's unethical to apply an unproven treatment outside of some form of investigation that will allow you to determine whether it works or not. The comparison in a well designed trial should always be to best proven therapy. There's no way you should deny best proven therapy to anyone. The substitution of randomization for informed guessing is in fact a judgment in itself. A judgment that obtaining the highest quality of evidence is better than perpetual informed guessing. So it's easy to dispense with those ethical issues, but there are some real ethical issues that affect, that are important in considering the use of randomized trials and the first is equipoise. There really must be true uncertainty as to which treatment is best in order to ethically conduct such a trial, but that applies to clinical practice as well. The study must ask an appropriate question and be well designed to answer it. It's unethical to put people into a trial that is so poorly designed that it won't give an answer or that's asking a question that's unimportant. And it must be practical. There must be a reasonable likelihood that the study can be successfully completed. The second issue is that of generalizability. The results of a randomized trial can only be reliably applied to patients who are very similar to those who are studied. That seems pretty obvious. The ability of the study to answer its question is best when the comparison groups are very similar to each other. And these two things create a tension between the scientist in us and the doctor in us. The scientist wants the treatment randomly allocated and wants everything else to be the same in the two groups. And there's this tendency, if we think of it purely scientifically, to try to create such rigid eligibility criteria that the only thing that's different is the treatment that you randomly allocate. That makes the interpretation very easy. But as a practicing neurosurgeon, you go out and look at that study and you say, that's great. This operation works really well in people between the age of 35 and 40 who are female South Pacific natives. And you have troubles actually applying the results to the broad population that you see. So these issues need to be thought about very carefully. The complaint, the criticism that randomized trials are complex and expensive is true. So is surgery, science, and wrong answers. I love it when a neurosurgeon who operates in this kind of environment says, boy, we can't do a randomized trial. It's hard. It's particularly galling when they also run a very complicated basic science laboratory. There are ways to reduce these problems. And one of the best, one of the strongest that is practiced much more effectively outside of the United States than in the United States are large, simple trials. So a good example of this was the CRASH trial. These large, simple trials have costs that are measured in hundreds of dollars per patient rather than thousands or tens of thousands of dollars per patient. They collect more data much faster. And they're very good for practical clinical questions. In the CRASH trial it was, is there a small but definite benefit to corticosteroid administration after significant head injury? Because meta-analyses of the existing data suggested that there could be. And the results are much more easily generalizable to large patient populations because large patient populations are studied and there's much less restriction on the eligibility criteria. This, you're not expected to read those. This is the total data load for CRASH-1, which was able to produce a very clear, very definite answer about the use of corticosteroids in significant head injury, which was in fact that it increased mortality. They were able to enroll 10,000 patients in five years. But we actually, I was on the Data Monitoring Committee, we were actually able to stop the enrollment after about 2,000 patients had been analyzed. And the fact that 10,000 people ended up enrolled was simply the lag between enrollment and analysis and then getting the Data Monitoring Committee together. So the data was being collected actually very rapidly and very efficiently. So there are ways to get around the complexity and cost, particularly for pragmatic randomized trials. And then there's very good evidence that randomized trials are actually an efficient way to answer clinical questions. Here's an example from digestive surgery where they looked at a number of 96 randomized trials and 180 observational studies in the same area. They found significant outcome heterogeneity in 41% of the observational trials and only 11% of the randomized trials. And in 20 to 25% of the time, the outcomes were different, significantly different between the observational studies and the randomized trials, indicating that we can't just automatically substitute good observational studies for randomized trials. This was accomplished while studying 10,493 patients in randomized studies as opposed to 90,000, more than 90,000 patients in observational studies. That's 8.6 observational patients to every randomized clinical trial patient. The ratio study or area to area was from 0.3 to 60. And that suggests that in most cases, fewer patients are exposed to inferior treatment in using randomized trials than using observational studies. This is true in neurosurgery as well. Now, these are historically important questions, not currently important questions. But it's by studying the history that we can understand it. So in the area of antifibrinolytic therapy following subarachnoid hemorrhage, which was very hot about 30 years ago, there were a number of uncontrolled studies, nonrandom controlled studies, small clinical trials, inconclusive studies, almost 3,400 patients studied in this way and total controversy about the outcome, 479 patients in a single randomized trial, clear reduction in re-bleeding, chemonucleolysis, chymopapping. It was an incredibly controversial topic. Over 20,000 patients were reported in uncontrolled observational series, and nobody could agree. 234 patients in three small randomized trials gave a clear and definitive answer about effectiveness. It worked. A lot of other reasons we don't still do it, but it was a therapeutically effective treatment. BCIC bypass trial is well-known. Uncontrolled studies, 2,600 at the time the trial was started. Very clear result from 1,300 patients in the trial, clear controversial results. Antibiotic prophylaxis, believe it or not, was a hot topic 20 or 30 years ago. And there are a large number of uncontrolled series, 13,787 patients, nobody could agree. A few relatively small clinical trials, clear answer favoring antibiotic prophylaxis. And the endarterectomy trials. Again, 17,000 operations reported from 75 to 85 in uncontrolled series. The controversy was so severe that the volume of carotid endarterectomy was plummeting in the United States because of concerns raised by the neurology community. Three relatively small trials, clear results supporting the operation. So in this case, you can see that the ratio of patients reported in uncontrolled trials not resulting in a clear conclusion about the therapeutic efficacy to the number of patients required in the randomized scenario to get a clear conclusion ranges, in this case, from about 2 to well over 80. So it's an efficient and effective way to answer clinical questions. Randomization controls unknown sources of bias better than any other technique and answers important clinical questions with fewer patients exposed to inferior treatment. But sometimes they still seem unreal. And one of the major issues about this is this question of intention-to-treat analysis where the patients are analyzed in the group to which they are randomized regardless of what treatment they actually received. That sort of thing makes your head spin. It just doesn't make good common sense to say, I put somebody in a trial of an operation, they got randomized to have the surgery, they didn't have it, and we count that result as a surgery result. Just seems wrong. Well, how do those things happen? Well, it happens if somebody screws up. Again, following Mike's advice, we're going to assume everybody's intentions are good. Somebody screws up and violates the protocol. The fact is that it's not always true. There's a famous anticoagulation trial where the envelopes that were used for the randomization allocation were so thin that if you held them up to light you could see what the allocation was, and there were physicians who were doing that because they wanted their patients to get anticoagulated. Those things happen. But if you have a long time in a surgical trial, a long time between randomization and the actual operation, a lot of things can happen. They can have a car accident on the way to surgery. They run into somebody they know who says, oh, no, no, no, never let them operate, that kind of thing. And so you can have a change in heart. Some other event intervenes and prevents an operation from happening. Or, as is probably more common, you get the treatment that you're randomized to, you're not happy with the result, and you switch over and get the other treatment. Those things all interfere with the understanding of the results of the study. And in particular, they reduce the ability of randomization to protect you from bias. Because if the change is done in a biased way, then obviously the protection that randomization provides is lost. And it's very easy to construct ways in which that crossover or movement from one treatment, from the allocated group to the other, can be done in a biased way. What's the practical impact of this? If protocol violations, crossovers, and losses to follow up are minimized, they don't materially affect the analysis. A well-designed study that's well-conducted doesn't have this problem. If they're large, the study is suspect regardless of what you do in the analysis. But it's a real problem. Mike provided an example. In this case, looking at clove fibrate and niacin in coronary artery disease, only about half of the randomized trials claimed to include all the randomized patients in their intention to treat analysis. And of studies claiming to adhere to that model, up to a third did not. And it has practical effects. So the pooled data not analyzed by adherence to the protocol showed no difference between clove fibrate and placebo. If you look at whether or not the patients actually took this stuff, you find out that those that didn't had a much higher mortality than those who did. And when you look at those two groups separately, there's not a lot of difference between clove fibrate and placebo mortality in the non-adherers and the adherers. So it can make an important difference in getting a result that's real. So sometimes the randomized trial seems unreal because we know the results are wrong. Some important examples, the ECIC bypass trial, the SPORT trial, the vertebroplasty trial that was reported in the New England Journal of Medicine. Sometimes, though, we know the results are right. We were very happy with nemotipine, the carotid endarterectomy trials, and the antibiotic prophylaxis trials. There was very little complaint about those trials. There's a very interesting characteristic. The ones we knew were wrong all said no, something we do isn't working. And the ones we were very happy with all said, yes, what we're doing works. So bias is a problem not only in conducting the trials, but in interpreting them. And we're back to evidence that facts eventually win. You can't change the facts. You can just try to obfuscate them. So we've talked about the essential components of the randomized trial and the advantages and disadvantages. When is it OK not to do a randomized trial? When is it all right not to do our best? Well, we have to remember that different kinds of questions require different kinds of studies. And the randomized trial is a therapeutic evaluation. If we're talking about patient assessment, examinations, imaging, pathology, scores and scales, we're talking about validity and reproducibility. If we're talking about diagnosis, sensitivity, specificity, likelihood ratios. If we're talking about prognosis, we're looking at outcome over time. And if we're talking about safety, harm, and risk, we're talking about risk and odds ratios, prediction, and questions of causation. It's when we're talking about intervention. Treatment A versus treatment B. So patient assessment is agreement among observers. Diagnosis is agreement with a gold standard. Prognosis, objective observation over time. Safety, harm, and risk dependence of outcome on factors. Each a different kind of question requiring a different kind of study. Intervention, an unbiased comparison of outcome. This is where the randomized trial fits. So one reason not to do a randomized trial is that it's not the appropriate study paradigm for the question you're asking. A second reason is that the condition is too rare to allow enough patients to be studied. So this really happened. This is 1988. And the first two surgical series on optic nerve decompression in osteopetrosis get published. One in the Journal of Neurosurgery and one in Neurosurgery. Ours got published in Neurosurgery because the reviewer in the Journal of Neurosurgery, where we sent it first, honestly wrote me back a letter and said, Dr. Haynes should know better than to present a case series. This needs to be evaluated in a randomized clinical trial. The total world experience is 11 patients and 22 nerves. It is a totally inappropriate comment from a reviewer. Fortunately, I don't know who that was. So conditions can be too rare to apply the paradigm. And finally, when the treatment effect is so dramatic that it overcomes any conceivable bias, it's unnecessary and probably unethical to do a trial. And you've been reminded about this paper. It's not a study. It's a paper. From two lazy obstetricians who decided to assault evidence-based medicine on the basis that the parachute should not be subjected to a randomized clinical trial. Now, I agree with that conclusion. But they didn't do any work to demonstrate it. It turned, and they said, no randomized trial so far, no randomized clinical trial. It turned, and they said, no randomized trial supports parachutes. Randomized trials would be unethical. And therefore, insisting on randomized trials would be silly. I agree. But it turns out that dramatic treatment effects, while they're very unusual, do exist in the real world of neurosurgery. And there's good prognostic information in the literature about patients with epidural hematomas who are deteriorating. And it turns out there's very good prognostic information about jumping out of airplanes that comes from the British and American Parachute Associations and from the FAA. So the first thing that surprised us when we started looking at the data was the mortality of jumping out of an airplane and having your parachute fail to work. Would someone hazard a guess as to what that number is? Somebody. Nearly 100%. It's actually about 74%. It's a shockingly low number. So even after you fall out and it won't open, there is some hope. The databases maintained by the Parachute Associations, however, are really good. And produce a very good estimate of the likelihood of getting in trouble if you jump out with a parachute. And it's really good. So the absolute risk reduction by using a parachute actually turns out to be about 74%. You prevent virtually all the fatalities. And by pooling the good prognostic data on epidural hematoma, it turns out that the absolute risk reduction of operating in the face of an expanding epidural with progressive neurologic deterioration is about 72%. The relative risk reductions look a little different. But the number needed to treat is almost identical. So this is a situation. The surgical treatment of an expanding epidural in a neurologically deteriorating patient, where I believe it's both unethical and unnecessary to do a trial. Because that treatment effect, based on good quality prognostic data, is so overwhelming that it's impossible to imagine a bias that would have produced it. So is the randomized trial gravitationally challenged? No, it just isn't always necessary. Mike, that paper is going to come out in neurosurgery in a few months. It's been a lot of fun, actually. So the problem is that there isn't any agreement about how big that therapeutic effect needs to be in order to avoid a trial. You can express it in non-quantitative terms. The outcome of interest should be serious, like dying, and unambiguous, as death usually is. There should be multiple observational series of high quality that should be very consistent in demonstrating the outcome of interest in a very high proportion of cases. And here, there's no agreement in the profession about how high that should be. We're proposing about a 70% absolute risk reduction. That's going to be a very unusual circumstance, but it needs to be a fairly rigorous standard. Others have suggested relative risks of greater than 5% or rate ratios of greater than 10%. But I get really worried when we start talking about relative risk, because you can get very large differences in relative risk with very small absolute risk reductions. And the risk of bias affecting those remains real. And finally, what is the appropriate role of the randomized trial in the evaluation of therapeutic interventions? Well, there really are two qualities of randomized trial. One is the experiment, the explanatory randomized trial that is useful when you have a conceptually new treatment that needs to be validated. And because it's conceptually new, we don't know that much about it. It hasn't been out in use. We haven't collected a great deal of data about it. The condition to be treated should be important. It should not be so rare that the trial is impractical. And the estimated treatment effect is not so large, as we just discussed, that you can avoid the trial. The purpose of a trial like this is to establish whether the intervention works. And these tend to be fairly rigorous eligibility and exclusion criteria. So the groups are very similar. The random allocation takes care of a lot of the unknown stuff because it's new. And you find out if this intervention actually works in a carefully controlled circumstance. The other kind of randomized trial is a pragmatic trial. This is when you have an intervention that's likely to be widely applied. So broad generalizability is important. It tends to be a relatively common condition. But the estimated treatment effect is relatively small or moderate in size. So there's a lot of noise in the data when you're trying to be broadly generalizable. And so you need the randomization to refine that so that you can pick a small but important treatment effect out of all the noise, and then be confident in generalizing that broadly to a population. And that's where the large, simple trials like CRASH become very, very useful. So there's two circumstances, I think, where the randomized trials are important. So the objectives here were to be able to discuss the essential components of the randomized trial, the advantages and disadvantages, the unique role, and the appropriate use. That was prospective comparison of best current therapy to innovative treatment with random allocation to treatment, and best possible bias control are the essential components of the randomized trial. When you're publishing the results of a randomized trial, most journals now will require you to, or at least strongly recommend that you use the consort statement as a way of, as a kind of checklist of, have you done all the right things to do the study well? It's a consensus statement about reporting results There are 22 items on the checklist. And as I said, many of the journals are now requiring it. It's worth using those kind of guidelines in designing the trials. Remember, the advantage, the primary advantage is that this tool gives you control over things you can't understand, measure, see, or know about. It's unbelievably powerful. Disadvantages, ethics, generalizability, complexity, and expense, and sometimes lack of reality. Remember, they're not needed when we're not comparing treatments. When the condition is rare, the question isn't important, or when the treatment effect is very dramatic. And finally, that there are two kinds of randomized trial that have specific purposes. And when those purposes are met, the randomized trial remains an incredibly useful and powerful tool for evaluating therapy. Thanks. Thank you.

randomized clinical trials

evidence-based medicine

random allocation

bias control

ethical considerations

intention-to-treat analysis

pragmatic trials