presentation on behalf of Dr. Mohammed Biden, we will be talking about the feasibility of using computerized adaptive testing to capture patient-reported outcomes in an outpatient setting, a pilot evaluation of PROMIS-CAT in neurosurgery. We have no disclosures. Starting with the definition of a patient-reported outcome, the FDA defines PROS as a measurement based on a report that comes directly from the patient and no one else. PROS helps patients, clinicians, and researchers assess overall patient-reported health status for their social, emotional, mental, and physical well-being, and they have been widely accepted as a valuable way to quantify patient experiences. Now, PROMIS, which stands for Patient-Reported Outcomes Measurement Information System, is the new NIH standard to capture PRO measures. The goal of the NIH was to create highly valid, reliable, and precise measures to capture overall patient-reported health. And PROMIS measures have been found to be more efficient in comparison to other scores in the sense that studies have shown that they require decreased time for completion and the overall results are given as a T-score. Now, why should we choose PROMIS over other legacy measures that are specific to each subspecialty? PROMIS is cross-cutting and specialty agnostic, which allows it to be applied across a wide variety of patient populations, and it also is found to be, like I mentioned, more efficient in administration as compared to other measures, as studies have shown that it requires decreased time for completion, and it gives a wide variety of domains that are available within the PROMIS system that can allow a measurement of patients' physical, mental, and social health. This slide lists all the domains that are currently available within the PROMIS system, and each of these is just another dimension to measure a patient's physical, mental, and social health. And as you can see, this allows a lot of flexibility and a lot of ways, you know, overall patient-reported outcomes can be measured. Now, PROMIS has been compared to other patient-reported outcomes in neurosurgery and especially in the spine population. The PROMIS physical function domain and also the pain interference domain has been shown to correlate well with other pain measures and disability measures, like the ODI, NSF-36, and the NDI, and also it has been more recently been validated in the population of, you know, patients with CNS tumors as well, as shown in the bottom right of the study mentioned on the bottom right. The computerized adaptive testing has more recently been made possible due to advances in item response theory and computing capabilities. Computerized adaptive testing, or CAD, what it does is it allows us to choose the most informative questions from within the item bank, and by tailoring the content of the questions to a patient's answers to a previous question, it allows, you know, increased ease of use and shortens the response time, decreased number of questions, and allows more efficient integration of pros in routine care. Because the current challenges to pro-administration include increased encounter times, increased costs, survey fatigue, and decreased patient satisfaction, as they do not like to fill out many surveys, and providers may also be reluctant to transition to a PROMIS-based registry. Now, the aim of our study was to determine whether or not it's feasible to administer PROMIS-CAD in an outpatient neurosurgery clinic before patients come up for their outpatient appointment, and also to not fill out the questionnaire prior to appointment, whether they can complete it after checking in at the desk while they're waiting, you know, for their appointment within the clinic. And as a secondary goal, we also wanted to look at whether implementing this pilot of PROMIS-CAD would have an effect on outpatient satisfaction as measured by a prescating survey due to increased waiting time or any kind of transit time for patients. This shows the, so we administered eight domains of the PROMIS-CAD. We administered domains for anxiety, depression, physical function, fatigue, pain interference, sleep disturbance, social roles, and upper extremity functions. This shows how we integrated PROMIS-CAD into the electronic health record. We employed a dedicated resource who would call patients three to five days before their appointment to help set up a patient online services account where a patient would be assigned PROMIS-CAD if they have a neurosurgery appointment and they can log in and complete the survey. And for some reason, if they do not fill out prior to appointment, when they would check in at the desk for, you know, when they show up for their appointment, the desk staff would employ an iPad and help patients log in and complete it. Now, once patients have completed, the results would be displayed within the electronic health record and would be available for providers to review and make management decisions based on that. As you can see here, this shows how PROMIS-CAD would be integrated into the electronic health record. This screenshot shows for a given patient how their PROMIS scores would be tracking over time. And as you can see, for this particular patient who is showing an increase in physical function and decreased pain interference. And this would help providers make more efficient management decisions. And so we conducted a two-month pilot from February to March of 2019. During that time, we assigned 3,426 patients the PROMIS-CAD questionnaire who would come for their outpatient neurosurgery appointment and 64.5% or 2,212 completed at least one domain. So our completion rates for each domain were just over 50%. As I mentioned, overall completion rate was over 64%. The highest completion rate was observed in the upper extremity function domain. When we looked at the number of questions the computerized adaptive system would ask each patient for each domain, we found that the upper extremity domain was responsible for asking the most number of questions. And the average number of questions each patient would answer was 45. And with a mean time of completion of 10 minutes. Now we think that despite having asked more questions, still we observed higher completion rates for the upper extremity domain was probably because that would be the first thing a patient would see when they would log in and start doing the survey, which could erroneously account for which domain would have the highest completion rate. As a counterbalance measure, like we mentioned, we wanted to see if implementing PROMIS-CAD pilot in an outpatient neurosurgery setting would have an effect on outpatient satisfaction, which we measured using the PRESS-Guinea survey, specifically the top box scores on the survey. And this slide actually shows that the top box scoring for patient movement and access components of the PRESS-Guinea survey were actually better compared to before the pilot. Now this was not statistically significant, but it does show that we had a relative preservation of outpatient satisfaction, and that did not affect, you know, our outcomes on the PRESS-Guinea survey. So for the next steps for this pilot, in terms of lessons learned, what we're going to do is try to decrease the questionnaire burden for patients. We're going to switch from eight domains that we were using to four domains only, for which we decided would be physical function, pain interference, depression, and social roles. We would also add PROMIS-CAD to something called the Welcome Tablet, which would allow easy integration with all necessary pre-visit questionnaires, which would serve as a one-stop shop, and that would allow for more efficient completion and also possibly increase completion rates. And also what we would like to do is we would like to implement electronic health record alerts, which would signal the provider in case of a significant or noticeable change in their PROMIS scores, which they might need to take into consideration while making management decisions for the patients. Now our experience compared to other ambulatory settings has been a little different, but it can be accounted for. This is a study out of a sports medicine clinic where they administered 581 patients, four domains of the PROMIS-CAD, and they observed a completion rate of 90%, and they found a mean number of questions asked of 15 with a time to completion of 2.6 minutes per patient. Their completion rate was higher, they did administer at the same time fewer domains than we did. And this is another study that examines the implementation of PROMIS-CAD in ambulatory cancer clinic, specifically gynecologic oncology, where they used five domains and their completion rate was just over 35%. And they also performed integration of PROMIS-CAD with the electronic health record, similar to what we did, and for some reason their completion rates were actually lower as compared to our experience. To conclude, computerized adaptive testing is a viable pro-collection mechanism in the outpatient setting. It does not significantly alter outpatient satisfaction, and EHR integration may bypass common patient-provider communication barriers by collecting pre-visit ePROs and delivering results in real-time. So, to conclude, EHR-linked CAD may represent a valid tool to increase pro-collection rates, and selective domain administration may alleviate patient burden. I'd like to thank…these are my acknowledgements. Thank you.

computerized adaptive testing

patient-reported outcomes

PROMIS

electronic health record

outpatient neurosurgery