Spinal CSF Leak Foundation in conversation with Dr. Timothy Amrhein
Dr. Amrhein took some time to discuss with us his most recent publication, “Efficacy of Epidural Blood Patching or Surgery in Spontaneous Intracranial Hypotension: A Systematic Review and Evidence Map.”
This study—the first of its kind for SIH and spinal CSF leak—reviewed 139 papers from the last decade, and found that no published studies met Level 1 evidence. About 92% of these studies were retrospective cohort or case series, and only just over ten percent compared the efficacy of different treatments. Most used objective methods for the diagnosis of spontaneous intracranial hypotension, but 37% did not clearly meet the International Classification of Headache Disorders-3 criteria, and almost 78% did not specify the type of CSF leak. Nearly all reported patient symptoms using unvalidated measures, and outcomes were rarely collected at uniform prespecified time points.
Dr. Amrhein’s paper concludes: “Evidence gaps demonstrate a need for prospective study designs, clinical trials, and comparative studies. We recommend using the International Classification of Headache Disorders-3 diagnostic criteria, explicit reporting of CSF leak subtype, inclusion of key procedural details, and using objective validated outcome measures collected at uniform time points.”
Dr. Amrhein is on the medical advisory board of the Spinal CSF Leak Foundation and is an Associate Professor in the Department of Radiology at Duke University Medical Center, where he also serves as the Director of Spine Intervention.
Transcript:
Andi Buchanan
Hello and welcome. I am Spinal CSF Leak Foundation Executive Director Andi Buchanan. And with me today is Dr. Tim Amrhein. Dr. Amrhein is an associate professor in the department of radiology at Duke University Medical Center, where he also serves as the Director of Spine Intervention. Dr. Amrhein is here today to discuss with us his recently published paper titled Efficacy of Epidural Blood Patching or Surgery in Spontaneous Intracranial Hypotension, A Systematic Review and Evidence Map. Dr. Amrhein, thank you so much for being here.
Dr. Tim Amrhein
Thank you so much for the opportunity to join everybody today, and I appreciate your interest in this, Andi.
Andi Buchanan
Yeah, absolutely! So first of all, congratulations on this important paper and can you just tell me a little bit about—what is a systematic review and what is an evidence map, and why are they so important?
Dr. Tim Amrhein
Yeah, that’s a great question. So a systematic review is a type of research where it’s an attempt at kind of an evidence synthesis, so, what do we know about what’s in the medical literature? And so there’s a formalized process for doing that, for reviewing all of the world’s medical literature. This was actually a very rigorously done study that was driven by a protocol that we developed beforehand, an a priori.
And you kind of go through the process of pulling all of the potential articles that could meet a certain criteria, in this case, articles that we’re discussing the efficacy of epidural blood patching or surgery in SIH. And you work with a team to try and determine which articles actually meet particular inclusion and exclusion criteria.
That then allows you to have a whole cohort of papers, a whole group of papers that discuss that. In this case, it was all the papers from I think October of 2021 throughout the history of time, and we reviewed all those papers and then were able to come up with summary statements about what’s in the literature and what do we know and what do we not know.
And so that’s the overarching goal of a systematic review. There are some subsets to that. So a lot of times you’ll hear “systematic reviews” and “meta-analysis.” And so what a meta-analysis is, is you do some fairly complicated statistics to try to get at an understanding of effect sizes and how well certain treatments work so that you can get a better estimate of an effect by pooling a bunch of data from a bunch of different papers.
We didn’t do that in this case because I think what we found really is that the state of the literature in our world, of SIH and efficacy is really not quite ready yet for meta-analysis. We can kind of get into a little bit more of that as we, as we talk, but instead what we did was an evidence map.
And so that’s a type of systematic review where we kind of map the evidence and sort of say, here’s what’s out there, this is what we’re seeing, and draw some general conclusions about it and recommendations for what we might do moving forward.
Andi Buchanan
Has this kind of research or this level of research ever been done before in terms of SIH and spinal CSF leak?
Dr. Tim Amrhein
I don’t think so. I think this is the first systematic review and evidence map. There were some— there are some systematic reviews at the time of the publication that we wrote here. There were three that had been previously done. Those included meta-analyses. Some were very narrow in scope. And some were kind of broader and more general. But I don’t think anybody had done a scoping review or evidence map like this before.
Andi Buchanan
So in the paper you investigate the efficacy of epidural blood patching or surgery. What does “efficacy” mean in this context? Is there a medical definition that’s different from the common understanding of the word as “something that works”? If we know that something works, why do we need to quantify it? And then also, I guess, crucially, how do paper authors determine if a treatment works or not?
Dr. Tim Amrhein
Yeah, I think it’s a real critical question. I think the definition is a little more complicated than might meet the eye at the surface.
I think it’s very reasonable to assume that efficacy means the general understanding that you know, how well something works, does a treatment work. The medical definition, though, is a little bit more nuanced. And there are kind of two sides to the coin: efficacy and effectiveness.
So what efficacy means is how well does a treatment work in an expert’s hand in an ideal situation, right? So if you come to a major center like Duke or the Mayo Clinic or Cedar Sinai, and one of the experts is treating you using their techniques that they do all the time—how well does it work? Does that procedure actually work?
On the other side of the coin is effectiveness, and so effectiveness research is how well does a treatment work in the real world. If you don’t go to an expert and you’re just getting it done generally in the community and there’s all these other variables that are involved that are not controlled for—Does that treatment work or not? And I think those are slightly different questions that sort of get at the same concept. At the end of the day, it’s really about: are we applying treatments that are actually working? Right? Because if we’re applying treatments that don’t work, then that’s not going to get anybody better.
Andi Buchanan
Right. I imagine when you’re looking through these, these papers to try to figure out the efficacy of these things it’s probably important to know what kind of a CSF leak a person had. I think in the paper you noted that many authors did not specify the subtype. Can you speak a little bit about that?
Dr. Tim Amrhein
Yeah. You know, one of the things that I think was apparent to the author group, which actually included authors from the Mayo Clinic and and at Duke is that when we critically appraise the literature and read through it, as we’re trying to decide what are the best things to do for our patients, it becomes apparent that there are some papers where it’s unclear whether or not the patients actually have SIH.
So there’s some heterogeneity there in the papers. But the other thing that’s really clear is in many of the papers, they’ll describe the outcomes of how patients did after an applied treatment, be it surgery or blood patching, or now the embolization procedures for cerebral CSF venous fistula. But in some of those papers, they’re not describing what subtypes of leaks are being treated. Right?
And so I think that’s important to know, because there may be differences in how certain types or subtypes of leaks respond to certain treatments. So for instance, it may be that we find out fistulas don’t respond to blood patching, or maybe they do. But if we don’t know who we’re treating, we may not be able to understand that that subgroup analysis, which I think is really critically important because there are fundamental pathologic differences between the different CSF leak types.
In the paper, we looked at—I’m looking at it now—is something, somewhere around 79 out of the total studies—130-something studies—did not really report what type of CSF leak they were looking at or the subtype. So it’s a really very common problem, I think, with the literature, and one that’s I think easily correctable in the future.
Andi Buchanan
So how does a paper like this ultimately affect patient care? Are there specific recommendations that came out of this paper?
Dr. Tim Amrhein
Yeah, I, I think there are. So I think that what this paper allows us to do is sort of look at the literature as a whole. And there’s a couple of things that we can kind of touch on as we go through.
But let me try and find, we did have some summary comments here—so, we recommended that future research confirm that patients have SIH and so we should be applying the, the ICHD-3 criteria or whatever diagnostic criteria is current at the time for whether or not a patient has SIH.
Because if you’re trying to talk about how are patients with SIH responding to a treatment, we need to know that’s who we’re actually treating, right, and not have a heterogeneous group of some people who may have other causes of headache or other causes for their symptoms. Otherwise, we’re sort of getting a very biased view or uncertain view to of how those treatments work for SIH patients.
That was one important piece. We also thought that people should explicitly report the subtypes of CSF leak, and I think we just spoke on that a little bit. But it would be helpful to not only talk about who’s being treated, but how those different subtypes are responding to the treatment in question. So, subgroup analyses—how do fistulas respond? How do nerve root sleeves respond? How do osteophytes spur type leaks respond?
The other thing is some of the important key procedural details are often missing in the literature, and I think there’s an opportunity for improvement in future publications as well.
So for instance, in many of the papers that we reviewed, it was difficult to determine whether or not an epidural blood patch was targeted to the site of a CSF leak or non-targeted, right? And, and many of us believe that could be a real critical difference, right? Similarly, whether or not fibrin glue was used, or the type of patching material, or, in the surgical literature, what type of surgery was done. Right? So if we don’t report those details, it’s hard to know how the treatment that we’re going to apply for one individual patient or a group of patients at our individual centers will respond. Right? Because we, we can’t really generalize the conclusions in the literature.
And then I think the other thing that’s really important is to use some sort of objective outcome measures. And so I think this was very pervasive in the literature is that a lot of the papers that we reviewed would say that patients did better. But that was also often a subjective interpretation, either based on the patient’s reporting, or in some cases based on the physician’s opinion about how the patient was doing.
And so there are some fundamental problems with that because it introduces bias. And if there’s an opportunity for us to use objective measures that have been validated ,you know, some sort of headache measure score or composite score for SIH symptoms, I think that’s really critical, right? And we can talk more about that, but those are some of the major recommendations that came out of this paper.
Andi Buchanan
Yeah. I imagine following up at specific times for a specific period of time would also be really useful in terms of knowing the true efficacy.
Dr. Tim Amrhein
Absolutely. You know, I think there’s a fundamental difference between following a patient one to two weeks after a procedure, or even, if they’ve gotten a blood patch and they’re still in the hospital, within 72 hours and saying “everybody does great after blood patches” versus following them out a year later. And I think those are the sorts of things that are really important. It’s also really important, I think, [in order to] be able to compare between groups, is to collect that outcome measure at the same time. Right? And so if we look at everybody at three months, then we might be able to compare different treatments—you know, how is embolization working versus epidural blood patching for fistulas, for instance.
Andi Buchanan
Are there any studies like that on the horizon?
Dr. Tim Amrhein
Yeah, I think, I think a lot of the groups that are interested in this are working toward that. And so, you know, one of the things that became clear as we looked at sort of the levels of the evidence and the different types of studies that are out there is there is very little in terms of prospective studies and there are no clinical trials.
And so, you know, one of the goals of this evidence map is really not to be critical of the literature. That’s actually not the point. It’s to identify areas of knowledge and then knowledge gaps. And so one of the points of this paper is to point out, here are the gaps, right? Here are the opportunities for us to fill those gaps, and how we might do that moving forward.
And so, yeah, I think that’s the next step actually, is to do a lot of these prospective studies where we kind of do a lot of comparative assessments of different treatments for different patients who have SIH.
Andi Buchanan
And when you say a prospective study, what, what does that mean and how is that different from the kinds of studies that already exist in the literature?
Dr. Tim Amrhein
Yeah. And so in the paper we have these nice bubble plots that, that sort of show, with each bubble representing an individual study. And actually I might, can I share my screen? Maybe we can do that and I can show you what one of these look like. Let’s see how this works.
Great. So this is from the paper and here’s an example of what one of the bubble maps looks like. And we did a series of these. Each of these individual circles represents a separate paper that we reviewed, and you can see the size of the circle is directly correlated to the number of patients that were involved in this study. So the larger the circle, the larger the number of patients. And so for this map, what we’re looking at is epidural blood patching in patients who had SIH and the type of guidance used—was the imaging guided or not guided?
And then we looked at the levels of evidence. And so what we’re seeing here is there’s different studies that could be done. At the lowest level of evidence is expert opinion. So you get an expert from a center—maybe Dr. Schievink or something gets up and says, this is what I think we should be doing.
And expert opinion is great, but is dependent on, you know, your trust in that expert. And even the best of experts, like somebody like Dr. Schievink could potentially have a bias or could be incorrect because you’re limited to only what you see at your center and in your experiences. So what we know from the medical literature is that there are different study designs that increase our confidence that what we’re finding is a real finding and is the truth.
And they begin at the lowest level with things like case series. So we report a series of CSF devenous fistula for the first time, which is like that sentinel paper that came out from the Cedars Sinai group in 2014. Those are great papers, right? They can really change practice like that [one] did, but they are limited in their scope. They only tell you about a couple of different patients.
As we go up the different study designs, you can see one of the most popular ones here is actually a retrospective cohort study. And so what that is, is we take a series of patients and they are followed over time. So we maybe see them before a procedure, we ask them how they’re doing or collect some imaging data, do the procedure, and then follow them at a specified time period afterwards typically, and say, Hey, what’s the change? How are they doing? Are they getting better? That’s what a cohort study is.
The prospective and retrospective is a little bit about planning and the timing behind it. So, with a retrospective cohort, what we’re doing is we’ve already collected those data and then we’re going, “Hey, you know what would be great is, let’s go back and look and see how we did with all those patients.” When we do it retrospectively, it’s a lot easier to do. However, we’re entirely dependent on how well we did in collecting those data, right, and if we don’t do it in a systematic fashion upfront, we end up with very heterogeneous data with a lot of uncertainty, like collections at different time points or subjective outcomes, assessments, et cetera. And so there’s a lot of bias that could potentially be introduced with retrospective studies.
When we move forward with prospective studies, they’re a little more complicated because we have to get patient consent beforehand, and usually there’s a protocol written out about how we’re going to do things, but that gives us the opportunity to collect the same measures at specified time points. And then we get some really clear understanding of, let’s say for instance, we’re going to use something like a HIT-6 or a pain score.
We might be able to collect that from a patient right before we do the procedure and then always collect it at two months afterwards. And then we’ll have a whole group of patients where we can say, Hey, at two months, this is what we’re seeing. So those are really powerful studies.
As we kind of move up the ladder, eventually we get to things like randomized controlled trials, and the specifics of that, where we randomize two groups, is that that allows for removing a lot of confounding, which are some variables that we don’t know about that could affect the study.
And so at the end of the day, really what we want to do is move up the evidence chain here to try and improve our certainty that there is efficacy to patching or to other procedures. And then do that in a fashion where we’re comparing different procedures to be able to make, you know, definitive statements about what’s best for patients.
Andi Buchanan
So, you know, we recently announced that we’re partnering with NORD to build a patient registry. It seems like something like that might be really helpful in aiding this kind of research.
Dr. Tim Amrhein
I think, yeah, I think a patient registry is going to be supremely important, it’s going to provide a lot of information. It’ll get us I think a little bit closer to better understanding, you know, the incidence and the prevalence of SIH, which is probably much, much higher than it’s currently being reported.
And I think it’ll give us an opportunity to gather kind of more nationwide or even international information about what’s happening to SIH patients—how are they presenting? You know, how, how bad are their symptoms? What’s their level of suffering? And you know, the other thing that’s really great about registry development is that it allows, I think for the first time, at least in this space, really great opportunities to sort of. you know, stretch, stretch our muscles, so to speak, in terms of multi-center collaborative research processes and networks, right? And that sort of thing can lead to multi-center research, and some of these prospective projects that can launch out of that. And so I think it’s a real critical piece that probably will make a really big difference for this community.
One thing that became really clear as we looked at the state of the literature is that there has been a real rapid uptick in the number of publications and interest in SIH over the past 10 years, and that was really heartening. The quality of the studies is improving. The number of studies per year is improving. And there are the beginnings of some efforts to solve some of these recognized challenges in the literature and fill these evidence gaps. So I think the future over the next decade or two or three is really extremely bright. And there’s a lot of things that are really excellent and exciting that are coming.
Andi Buchanan
Well, thank you so much for your time. I really appreciate it. And thank you for all you do to accelerate research for SIH and spinal CSF leak.
Dr. Tim Amrhein
Absolutely. Thanks so much for the time.
Further reading: