I want to thank you for taking the time to listen to our presentation of Duke's development and deployment of the Surgical Autonomy Program within our residency and some of the results from its use within our pediatrics group. I represent the larger team below on this project, which was started by Dr. Michael Hagelin. We have no disclosures at this time. Historically, residency training has been based more on an apprenticeship model where it was expected that a resident would learn the techniques and methods of the attending surgeons through observation and practice. Over time, though, there has been additional pressures placed on attending surgeons to increase work volume but still integrate teaching, which is inherently inefficient. At the same time, the number of opportunities in the OR have decreased for residents as work hour restrictions were enacted. This creates the need for more effective teaching in the OR to maximize a resident's limited time with a given procedure and improve case-to-case abilities. At the same time, the majority of formalized feedback is generally collected and presented in a summative manner, whether that is through end-of-rotation evaluations, mid-year reviews, or milestone determinations. These summative evaluations are generally lengthy and resource-intensive with lower compliance. Many times, it is also occurring months after a specific surgery, making it less useful for day-to-day change. There is, of course, typically informal feedback provided during a case, but there exists a gap in more formalized real-time feedback for residents specifically for technical skills. Finally, there is a trend to begin to collect more data to assess the competency for a resident as they move through residency toward certification. We saw the need to develop a platform that met the following four needs. We wanted to increase the real-time resident feedback on procedural skills to enhance learning. We wanted to decrease the time between the event or surgery and the feedback itself. We wanted to increase the amount a resident can learn from a given procedure. And finally, to increase the amount of feedback for faculty as teachers. We therefore developed and implemented an innovative smartphone-based tool that we call the Surgical Autonomy Program in our program at Duke. Here we talk about its implementation within the Pediatric Neurosurgery Group. The SAP applies Vygotsky's social learning theory to the process of acquisition of surgical skills and competence so that we target the portion of the case that a resident is most motivated to learn in. This is the portion or skill right beyond what they last mastered. So, for example, after positioning an opening for epilepsy resection, the next area would be performing the craniotomy. It is this next area that the most knowledge can be efficiently gleaned and the resident is most motivated to learn about. This is applied by the software application, which I'll describe shortly. We examine residents and faculty use for indexed pediatric neurosurgical cases in a 26-month pilot at Duke University Hospital between August 2017 and October 2019. We present data from 171 pediatric neurosurgery cases performed. The following are a few screen captures of the software to give a sense for how it works. Here's the case creation screen where the resident or faculty inputs the initial details of the case. This sends a notification to the other party to fill out their portion. Both the resident and faculty fill out the same evaluation. For the resident, it provides metacognitive feedback to see how their own sense of progress compares to another person's assessment. Everyone notes which zone they were concentrating on and how they felt they performed in each section. The faculty form also includes a final text box to provide a short written comment about areas to improve for next time. Finally, the resident obtains a final screen that compares directly the faculty feedback with their own self-assessment. It then also asks the resident to rate how the teacher performed as an instructor in the OR, which is then anonymized and aggregated for later faculty development. We found that we were able to divide the indexed RRC cases for pediatrics into four zones or parts. These are obtained with the general consensus among pediatric faculty to provide consistency between different attendings, but still allow for individual differences in how they perform a specific portion. So for example, removing a tumor, we divided this up as follows. Zone one included the positioning of the patient and the bony exposure. Zone two, the craniotomy itself and the dural opening. Zone three was the tumor resection. And finally, zone four was the dural closure, flat placement, and closure. Shown also are examples of ventricular peritoneal shunt and the ETV with their four zones and how they were divided amongst the zones. We encouraged the faculty and residents to discuss prior to starting the case, which zone they would be focusing on during the case that day. This allowed for the extra focus on that zone and the focused learning and teaching in that section. The TAG scale was created as a modified version of the ZWISH scale in order to incorporate more details of who is the primary surgeon, who is assisting, and the extent to which the faculty is guiding the procedure. It is the TAG scale that is used to assess the level of work that the resident did within the case. So starting first at teach and demonstrate, this is where the faculty is the primary surgeon and is basically showing the resident how the case is done from start to finish or within that zone. A stands for advise and scaffold, and this is where the resident becomes the primary surgeon and the faculty is kind of walking them through the case. We then move on to guide and monitor, where the resident is again the primary surgeon but they're the ones talking the faculty through what they're about to do with the faculty helping with the finer points of that zone. And finally, solo and observe is where the resident can perform the case independently for the chosen zone or they're at the level that they could train a junior resident to go through that zone. After implementation, we found that the SAP appeared to be both efficient and feasible to integrate into a busy practice. For residents, it took an average of 26 seconds to fill out the form, and for faculty it only took about 36 seconds. So in less than a minute, both parties have a tremendous amount of data to use, and especially for the resident to be able to improve for the next case. The project was able to show progress through a case by specific residents. Here are two examples, and you can see the progress through the tag scale for each of the zones. The graphs also were useful for faculty as well to be able to see where a resident was in a specific case before walking into the OR. Once there is a discussion, this can be undertaken to agree upon what aspects of the case to concentrate on. These are just a small sample of the written feedback given by the faculty to the residents to help them improve from case to case. The feedback is timely and actionable and helps to direct next steps. So even though sometimes small suggestions, they can have a larger impact on learning within the OR. So in summary, we were able to demonstrate the ability of the SAP to easily and clearly measure resident learning and progress in performing pediatric neurosurgery cases. We were able to enhance the efficiency and frequency and timeliness of intraoperative assessment and allowed for focused learning within the OR. And finally, this data will be useful for individual residents to highlight both their strengths and weaknesses and to be able to guide their own training throughout a residency program to both rectify any deficits and to highlight the skills. Over time, this may allow for more fluid program designs that are more based on competencies of the residents as they're moving through it. And bigger picture would be potentially modifying residency training so that we would move away from purely case minimums to more of a competency-based training program. So in summary, the Surgical Autonomy Program makes real-time intraoperative performance assessment feasible for every index pediatric neurosurgery case and can be feasibly integrated into a residency training program. The SAP provided a scalable and efficient approach that divided each surgical case into four zones of proximal development. And this pilot has demonstrated the ability to easily and clearly visualize resident progress for pediatric neurosurgery index cases. Thank you for your time.

Surgical Autonomy Program

residency program

pediatrics group

real-time feedback

procedural skills