Measuring the Impact of Training the Trainers: Lessons From Pilots and Plastic Surgeons

How can organizations quantify the impact of the train-the-trainers model? A pioneering new study from a health-care nonprofit offers a template.

The train-the-trainers model is an educational framework designed to turn trainees into expert trainers themselves, equipped to pass on knowledge or skills they’ve acquired to subsequent learners in their communities. The World War II era has been credited with institutionalizing and popularizing the approach when it was used to quickly mobilize a mostly new industrial workforce. The exponential nature of the model’s reach had an obvious appeal to the social sector. When the problems at hand are often so great, direct service delivery can never catch up to ever-growing needs. In those cases, local capacity building and a train-the-trainers model offers a promising path to close gaps in access to health, education, and other necessary social benefits. For instance, in the field of global surgery, where we work, the gap in access to surgical care is astronomical, with five billion people lacking access to safe, affordable, and timely surgery. The problem is simply too large to ever solve one surgery at a time.

Despite train-the-trainers models becoming more and more common, a pressing issue remains: How do we measure the impact of training? This question becomes even more critical across sectors where the training requires education beyond online learning to specialized training that necessitates hands-on learning from an expert. For example, for surgeons, astronauts, pilots, chefs, tailors, and many other fields, theoretical lectures are critical to their learning, yet hands-on time is essential to the mastery of their craft. Just as we may never forget who taught us to tie our shoes or ride a bike for the first time, many surgeons will always remember the trainer who taught them how to gracefully tie her first suture. How can we begin to quantify that type of impact? And, by extension, how do we demonstrate to funders that these programs are worth the investment?

Embracing the Train-the-Trainers Model

At our organization, ReSurge International, a global health nonprofit with a mission to build reconstructive surgical capacity in low- and middle-income countries (LMICs), our evolution has taught us about the transformative impact of training. During the 1990s, we initiated a shift from the then-standard medical-mission fly-in approach to a more sustainable model centered on training local health care professionals and capacity building. Many across the nonprofit sector have made similar investments in the train-the-trainers approach, intuitively knowing that this model will have a more profound impact than one-off service delivery models. The impact of that shift has been profound at ReSurge. For instance, in Nepal, we trained the first-ever reconstructive surgeon in the country who has now gone on to establish a hospital dedicated to burns and reconstructive surgery. Kirtipur Hospital in Kathmandu now boasts fourteen full-time specialist plastic and reconstructive surgeons with six operating rooms, and a full-time residency and specialty fellowship programs, also hosting multiple international trainees. Just last year, the team performed over 1,200 plastic and reconstructive surgery procedures, admitted close to 800 acute burn patients, and treated patients from 75 out of the 77 periphery districts across Nepal. The reach of training a leading local surgeon who can go on to train others far outweighs the impact of paying for only one surgery at a time.

Are you enjoying this article? Read more like this, plus SSIR's full archive of content, when you subscribe.

These numbers make the investment in training seem obvious. We went from one-off episodic trips where medical volunteers could treat 50-80 patients in a few weeks to a long-term investment in training a local surgeon who will treat 300-500 patients per year over a lifetime and create a multiplier effect of impact. As ReSurge’s consulting medical officer and chief of plastic and reconstructive surgery at Stanford University, Dr. James Chang, likes to say, “We aren’t just teaching a person to fish, we are building fishing academies.”

While there was consensus that training was the most effective path forward, our funders still asked for tangible evidence that this new model was having a greater impact. They wanted to quantify this “multiplier effect.” What exactly was the impact we were having in the world by training one surgeon who would go on to teach others? What outcomes can we legitimately claim and when do they become diluted?

The Data Dilemma

Determined to find a way to quantify the impact of our training efforts, we did a literature review that assessed 48 studies that evaluated the impact of surgical training. However, these papers were all focused on a specific technique, training component, or patient experience, leaving the question of what was the general impact on patients by deploying a train-the-trainers model still unanswered.

Convinced that insights could be gleaned from other sectors that required both didactic and hands-on teaching as opposed to online-only or lecture-based learning, we explored various other industries for potential models or methodologies to quantify the impact of training. We consulted tailors, criminal law solicitors, musicians, and even military pilots to see if they had established methods. What we uncovered was a common gap across disciplines: a lack of existing frameworks to measure the long-term, multigenerational impact of training. One survey respondent in the US Navy in charge of aviation training said, “We have a formalized channel, training methodology, and set of venues for passing those important emergent new skills to subsequent generations of pilots, but we do not have a way to measure the attributable impact of those channels.” This research would be the very first of its kind.

A First-of Its Kind Model

This led us to develop a model to estimate the influence of surgical training on the number of surgeries performed across two generations of trainees. We defined the impact of a surgeon's training as the number of lifetime cases associated with that surgeon, extending over two successive "generations" of trainees. We decided to limit the impact to only two generations, choosing to err on the conservative side, as each generation further away from the expert trainer would have a diluted impact. In surgical training, as Surgeon A trains Surgeon B, and Surgeon B continues on to become a trainer, she will certainly have attributes of her practice taken from Surgeon A that she carries onward to her trainees. However, Surgeon B will be taught and influenced by multiple surgeons along the way, gaining her own teaching style. As such, we felt we could not continue to count Surgeon A’s impact beyond the second generation.

We then conducted quantitative surveys of surgeons in high-, middle- and low-income countries. Surgeons provided estimates of the annual number of cases they performed, the number of trainers who taught them, and the number of trainees they expected to train in the future.

A chart showing calculations for attributable impact.
(Click to enlarge)

Our findings revealed a sizable impact in the first generation of training, amplifying several-fold in the second generation of training—the "multiplier effect" that we so often referred to. This was particularly high for trainees in LMICs, where there may only be one plastic and reconstructive surgeon serving the entire country. In contrast, during medical training in high-income countries, residents might benefit from the knowledge and techniques of over 50 trainers. In 2014, the entire country of Uganda had only three plastic surgeons for a population of 36 million people, and Zambia had just one for 20 million people. Given this shortage, training a single surgeon-trainer in an LMIC can have a tremendous potential for impact—a potential that we have (now) successfully quantified.

Bar graph showing lifetime attributable impact of surgical training
(Click to enlarge)

Through this model (read our full study here), we estimate that a reconstructive surgeon in an LMIC is projected to directly treat 10,000 patients over their career. More impressively, if this surgeon evolves into a surgeon-trainer, their influence could extend to impacting over 400,000 patients across two generations.

Caveats to the Model

It is critical to acknowledge that while our model predicts potential impact, it does not directly correspond to actual surgeries performed. By building out the local capacity of the surgical workforce, we are addressing one piece of the global surgery gap. However, numerous other barriers to accessing care, particularly in low- and middle-income countries (LMICs), remain. These include financial barriers like the cost of procedures; geographical access to a facility with a trained specialist; gender equity issues involving awareness, stigma, and cultural norms; and hospital infrastructural issues including the availability of adequate supplies, medications, and equipment.

Another interesting insight from the research is that the train-the-trainers model has a higher impact in an LMIC setting when compared to a high-income setting. This is due to the availability of expert trainers to learn from. In Sub-Saharan Africa there is only one plastic and reconstructive surgeon per 10 million people, and in some countries, such as Burundi, there are still none. Therefore there are no expert trainers to learn from, meaning the first trainer in that country will have an outsized impact. Our goal is to expand the specialty workforce so that the profound influence of an expert trainer in these regions begins to mirror the impact seen with trainers in the United States.

It's also important to note that not all surgeons will reach this level of impact; our model specifically considers those who advance to become expert trainers. Additionally, as the number of trainers increases in these countries, their individual impact may decrease, which is a positive development as it indicates broader access to care and overall success.

The Future of Our Data

Since publishing this study, we have seen a groundswell of interest from other major global health organizations. We also have concerns regarding how our model is used, particularly if the large impact numbers are misused to represent actual impact rather than potential impact. However, since this is the first quantification of its kind, we are excited to share across the social sector that what we’ve found validates what we’ve all intuited to be true–that training and investing in human capital and education within local ecosystems drives social change.

It remains true that many funders want to see the annual return on their investment. However, we urge funders to recognize the value of deep, long-term investments in these expert trainers as crucial to building local capacity. Measuring the impact of this work is more complex than merely counting the number of beneficiaries, as true impact lies in the depth of the training provided. It's important to understand that building capacity through a train-the-trainers model is a generational effort, essential for closing the significant gaps in services and strengthening systems.

Given the paucity of other models that quantify the impact of training, the implications of this new model are significant for organizations wanting to better understand the impact their training model is having in the world. While surgical and medical nonprofits can directly input their variables into our model, the general concept can be used across any sector to input their own specific variables. We have already consulted with other foundations and surgical specialty organizations who are looking to apply this model to their own work. So far, as we work to directly apply it within other surgical specialties, we have found that the variables are indeed quite easy to interchange. For example, with cataract surgery, a life-changing specialty procedure with many nonprofits working on this in LMICs, we found we could even simplify some of the variables when applying our model to their work. Whereas our model looked at the diverse field of plastic and reconstructive surgery, we used eight common index cases as an average, cataract surgery organizations need only input data from one procedure type. Another common variable that will certainly change across other surgical specialties or non-medical fields, and certainly within different contexts of HIC versus LMIC settings, is higher or lower attrition rates for the trainees who go on to become master trainers. This is also a variable that can easily be updated within the model, but it will require consideration.

For us at ReSurge International, this study reinforces the foundation of our theory of change, that building surgical capacity through local training programs is essential for any long-term global health initiative. We will leverage this data to fuel our work and continue to adapt and innovate while demonstrating to donors and institutions that we can close the unmet burden of surgical disease through investment in long-term training to increase surgical capacity.

We hope other organizations, across disciplines and around the world, will also use this model to advocate for the teachers and trainers who are pioneering legacies for good, whether that is in the community or on the operating table. We are profoundly grateful for our teachers and trainers, for both the measurable impacts we can quantify and the intangible influences that we cannot.

Support SSIR’s coverage of cross-sector solutions to global challenges. 
Help us further the reach of innovative ideas. Donate today.

Read more stories by Natalie Meyers & Anna Santos.

 

Measurement & Evaluation

Natalie Meyers
Natalie Meyers is the chief program officer at ReSurge International.

Anna Santos
Anna Santos is the chief marketing officer at ReSurge International.