This is Kate Drummond. I'm the Director of Neurosurgery at the Royal Melbourne Hospital, Melbourne, Australia. I'd like to speak to you about challenges in the Australasian Neurosurgery training program. In particular, the idea of who should train as a neurosurgeon and where they should train. This is taken from our recent article in Neurosurgical Focus covering these questions. In the recent literature, neurosurgical education has seen an emphasis on innovative educational methods, in particular online education, simulation and virtual reality and the importance of research in neurosurgical training. However, there's been little emphasis on what I feel is arguably a more important aspect of producing excellent neurosurgeons and that is the selection of neurosurgeons to train and accreditation of training units or programs. I'd like to talk about Australia and New Zealand or Australasia and our unique in training environment which brings these aspects of neurosurgical education under close scrutiny. We have a unique situation in Australasia. We're a large country in Australia with a small population and we do like to live near the beach. So the neurosurgical units are located in major coastal cities and are often widely separated and sometimes service relatively small populations. In New Zealand, it's a small country with a small population and thus small units, again, servicing relatively small populations. In the 1980s, the Neurosurgical Society of Australasia or NSA recognized that some of these smaller single neurosurgical units could not really provide adequate breadth of experience and caseload for training and this was especially recognized when the NSA Board Chair, Professor Lee Atkinson, brought in a process of centralized accreditation of training posts which informed the need for rotation through a number of units for a broad training experience. In 1991 to 1994, therefore, a centralized bi-national training program was developed by Professor Peter Riley. Education and training was centralized into a bi-national central panel and trainees moved between units every one or two years with at least one interstate or international allocation, usually moving between three or four training units. This allowed for training and mentorship by 20 or more individual neurosurgeons through the course of an individual's training. Subsequent evolutions have included regular curriculum review, training lengthened from four to six years with competency-based training now used and the introduction of flexible hours and less than full-time training, which is currently in its infancy. Regular in-training assessments have also been added to the long-standing exit examination. Overall, there is still genuine general satisfaction with the standard of training in Australasia. However, there have been some deficiencies raised. There is no rigorous evaluation of neurosurgical or any surgical specialty trainee selection in the literature, so there's no gold standard. So the deficiencies in the Australasian centralized model have been really shown in relation to the underperforming trainee. So although the selection process is arduous and transparent, not all trainees complete training and dismissal proceedings due to poor performance, even commencing late in training still occur, which is really very unsatisfactory. Now these deficiencies may be attributed to a poor selection process, to inadequacies within the accreditation of training units, or most likely due to lack of skin in the game for the training unit who really don't have ownership of the trainee from the time of appointment to the end of training, which is really unlike perhaps North American training units. Therefore, it may be easier to give an excellent reference or pass a trainee to move them to the next unit rather than take the more difficult, adversarial, or litigious path of genuine negative feedback or refusing to give a reference or even fail a trainee in a rotation. So how does this selection process work? Well, it starts with a minimum score on a structured CV, which includes aspects of research, publications, and previous neurosurgical experience. This is weighted to 15% of the total score and to pass this as a satisfactory performance in the CV, 5.5 of 11 possible points must be achieved. The applicant must also undergo a 70 question multiple choice neurosurgery anatomy exam, which is 30% of the weighting and they must achieve a 70% score. Satisfactory performance in these two components will lead to reference checks from three referees. With the scores from these reference checks, which must be 36 points out of a 72 possible, the 24 highest scoring applicants on these three components are subsequently selected for final semi-structured interviews, which are standardized clinical decision-making scenarios with defined marking rubrics. There's been one significant change in practice of this selection process, and that's about references. References have been contentious over the last 10 years. The SET Board and other authors have noted that standard written references really don't discriminate between applicants. And this is either with a pro forma reference or a letter of recommendation. Strategic scores are common or overly positive written references, which do not reflect subsequent performance in training. Therefore, in 2015, the SET Board changed from a pro forma written reference to a detailed semi-structured telephone reference, which is an interview with three neurosurgeons for each applicant with a defined marking rubric. The applicant doesn't get to choose the three neurosurgeons. They identify all recent supervisors and the interviewers can choose any neurosurgeon who's recently worked with the applicant. And the reference interview is undertaken by two neurosurgeon members of the SET Board. The interview has questions related to the competencies of a surgeon that underpin the SET program in neurosurgery as defined by the Royal Australasian College of Surgeons and shown here. So the interview covers the relationship to the applicant, period of contact and in what capacity, aspects of technical skills, including competence in neurosurgical procedures with dexterity, orderly flow, positioning, preparatory study for the procedure and the supervision required. Aspects of collaboration and teamwork, including trustworthiness, recognition of complications, reliability, accuracy of communication and interaction with other departments. Aspects of communication and professionalism, including reliability, workload management and responses to mistakes and negative feedback, as well as aspects of scholarship teaching and readiness for training with satisfactory attendance and contribution to educational sessions. And also a question with regards to whether the applicant is ready for neurosurgical training would benefit from further pre-vocational experience. Unlike many other countries in Australasia, young doctors undertake an internship and a period of pre-vocational training before entering into specialty training and therefore some may even undergo three or four years of working as surgical pre-vocational trainees before commencing neurosurgical training. The accreditation of training posts is another aspect which we have scrutinized and the importance of the trainee experience is increasingly recognized and therefore accreditation of training posts for excellence is critical, particularly if trainees and their family are to be moved long distances. So the minimum characteristics of accreditation are strictly defined, despite the fact of little literature to guide how to do this and regular feedback is sought from the trainees and aggregate five-year data of this feedback is released to each post to maintain anonymity. Standards include a supervisor of training with strict professional and training requirements and considerations of caseload and case mix, hospital facilities, training infrastructure, educational program and recently a strong emphasis on trainee welfare, including robust mechanisms to deal with complaints, discrimination, bullying and harassment. Accreditation occurs every five years but shorter periods may be given if deficiencies are identified and this occurs with site visits and physical inspections, interviews of trainees, neurosurgeons and other staff and review of application documents. So our methodology for analyzing the changes to the referee interviews and trainee post accreditation included a review of all of the scores for components of the application to neurosurgical training between 2014 and 2019 to compare and assess the changes in the process. A review of trainee caseload as assessed as a measure of adherence to accreditation standards looking at 20 trainees who'd completed trainee in the average five or six years of training, looking at their caseload from their logbook numbers and also taking some written evaluations of the trainee experience from trainees at the midpoint of their neurosurgical training and asking them to reflect on selection, training post quality and limitations of the current system. In 2019 these were the numbers for the applicants for the training program just to give you some background. 66 applicants had a CV scored and 66 applied for the anatomy exam. This led to referee reports being undertaken on 54 trainees who had obtained satisfactory scores in both the CV and the anatomy exam with one withdrawing. After referee reports were taken from those 54 trainees, 24 went into the interview process and in 2019 there were nine places available for training with seven male and two female successful applicants. The change in referee process showed a marked difference in the reference scores that were seen. The semi-structured interviews are extremely time consuming and but the initial results are very favourable. In 2014 with a pro forma written reference 96, the average score was 96% with many applicants obtaining a near perfect score. As you can see from this slide from 2015 to 2019, the average score from the reference report with the interviews ranged between 57% to 76% and the range of scores was very wide allowing for a much better spread of scores to differentiate between applicants. There are currently 31 accredited training posts in Australasia with 27 in Australia and four in New Zealand and they're shown on the map here, mostly around the coast as I discussed. There's 56 training posts as some units take more than one trainee. In 2018 and 2019, two new applications were denied due to non-compliance with standards and one unit lost accreditation and five units had shorter periods of accreditation given to rectified deficiencies. So there does seem to be some merit to strong training post accreditation standards with not all units achieving. As you can see from analysis of caseload data, all trainees more than met the requirement for minimum cases, including 100 major cases per six months in adult posts and 75 major cases in a single six month paediatric post. And this confirms the robust nature of unit accreditation for provision of adequate operative experience. The trainee experience as discussed with two trainees here, you can see Heidi McAlpine and Jordan Jones. We asked them to reflect on selection of their training post quality and limitations of the current system. Overall, they appreciated the moving between training posts for diversity of training to produce a well-rounded neurosurgeon with a large professional network and with a diversity of skill acquisition. But they also appreciated the logistical challenge of moving away from support networks, learning new hospital systems, some isolation and financial burden and real relationship challenges with the spouse's career and family. So our conclusions were that the bi-national multi-institutional training programme was very successful in improving diversity, strength and breadth of training for Australasian neurosurgeons, but it has brought challenges. The first challenge is that of uniformity of the training unit accreditation for excellence. And this is essential to balance the challenges for the trainee with moving them between accreditation units. And there's little in the literature to guide us as to what the gold standard is, except for perhaps subspecialty areas such as endovascular, cerebrovascular or spinal surgery. Monitoring metrics such as operative experience, professional behaviour and participation in education within a framework of national accreditation, however, may avoid rogue surgeons and recent appearances in the media of underperforming and dangerous surgeons and often show in retrospect, poor training and chronic underperformance that has not been noted due to lack of oversight. And so therefore, accreditation of units and oversight of trainees would seem to be exceptionally important. The second challenge is the best method, of course, for trainee selection, and there is no agreed standard in the literature. Two recent articles looking at various resident applications and the electronic residency application service basically showed that objective standardised test scores do well correlate with in-training and board examinations, but correlate poorly with faculty evaluations. And factors such as letters of recommendation or references, although highly valued, do not lead to correlations with faculty evaluations or a high level of satisfaction with selected residents. So I think efforts to improve the referee process have addressed a recognised deficiency in Australia with an emphasis on proficiency in non-technical skills, which is increasingly seen as critical to neurosurgery. And the success of our in-depth referee interviews will only, of course, become evidence with future analysis of trainee completion and performance. And the most serious problem of our centralised selection process is a recognised lack of ownership of some centrally selected trainees allocated to a training unit with which they have no ongoing mentoring and supportive relationship. And you can see here, my training unit and many of the trainees shown here will soon rotate into other units, showing the difficulties of this system. There's an implication, of course, for future employment of orphan graduates who may not have a home unit which might take them on for employment. Future challenges will be the ever-increasing number of applicants from whom we must choose the very best, who are older and more likely to have professional partners and children, and who will demand flexible, rigorous and transparent and excellent selection processes and training standards. Solutions are not clear for us, but creation of smaller training networks or home units has been suggested. The referee interview process certainly needs to be analysed in future years in relation to trainee retention and progression. These are some references that you might want to see, you might want to read, and many thanks for your attention.

Neurosurgery

Australasian Neurosurgery training program

training neurosurgeons

accrediting training units

challenges in moving between training units