Journal of Dermatology for Physician Assistants

The official journal of the Society of Dermatology Physician Assistants

Evaluating the Use of Supplemental Training Technologies in Dermatology Education

By Mallory M. Aycock, MPA, PA-C; Craig D. Marker, PhD; and Philip J. Kellman, PhD

ABSTRACT
Physician assistants (PAs) are licensed to evaluate, diagnose, and treat dermatologic conditions. Data show that medical students have less than optimal dermatology diagnostic abilities. Although no known data exists for PA students, similar medical school and PA school training methods highlight a need for improved dermatology education in medical and PA programs. This project explored the use of perceptual and adaptive learning modules (PALMs) that target pattern recognition skills with PA students to hopefully improve PA knowledge of dermatology concepts.

The Journal of Dermatology Physician Assistants is proud to present “Evaluating the Use of Supplemental Training Technologies in Dermatology Education” by Mallory M. Aycock, MPA, PA-C; Craig D. Marker, PhD; and Philip J. Kellman, PhD. Although it focuses on areas outside the scope of the practicing dermatology PA, the original research discussed within explores education methods that have potential to improve PA knowledge of dermatology concepts and is therefore likely of high interest to our readership.

BACKGROUND
There are no known studies on dermatology training in PA programs. The data on medical school training suggest that about 50 percent of medical schools in the United States provide 10 or less hours of dermatology training, while about eight percent require no instruction in dermatology.1,2 One study by Ulman et al demonstrates medical students are not proficient in diagnosis and treatment of dermatologic disease despite about 18 hours of training, and the authors of this study suggest evaluation of dermatology curriculum nationwide.1 While our PA program provides roughly 29 hours of didactic education in dermatology, mastery of dermatology concepts cannot be achieved without practice.3 There is a discrepancy between learned knowledge in medical education and application of that knowledge during clinical practice, notably in medical skills that require recognition of clinical patterns, such as dermatology.3 Traditional teaching methods are thought to lack in the training of perceptual learning, defined as “experience-induced changes in the way perceivers extract information.”4 Whereas most instruction emphasizes explicit declarative and procedural learning, other crucial components of expertise, including pattern recognition, fluency, and clinical intuition involve different learning systems and advance through more implicit and interactive learning experiences.3,4 Diagnostic expertise in medical learners advances through classification episodes that incorporate a range of instances that encompass normal and pathological variations across relevant categories. In traditional medical education in dermatology, as in other domains, these aspects of learning occur somewhat unsystematically through recurrent clinical experiences or exposures to patients with dermatology complaints.5,6 Gaining this experience may take large amounts of time that is not afforded in conventional didactic dermatology education models.2

One resource proven to bridge this gap in medical education is an online supplemental technology resource developed by Insight Learning Technology, Inc. called Perceptual and Adaptive Learning Modules (PALMs).3,7 PALMs were created as a technology resource to help students increase mastery of medical skills through pattern recognition. They are intended to supplement didactic training by providing practice examples that enable optimization of rate and retention of performance in specific medical skill areas.3,7 PALMs have provided significant improvement in medical school education and medical resident training in the areas of echocardiogram interpretation, histopathology, electrocardiogram interpretation, and dermatology.3,6,8,9 This supplemental platform, however, has never been studied in dermatology training specifically in PA education.

PALMs target perceptual learning—changes in the pickup of information that occur in a given domain as a result of practice or experience.4 They incorporate a number of principles of learning, including spacing and interleaving, and systematic variation of exemplars, to accelerate expert pattern recognition skills. The adaptive components of PALMs pair advanced adaptive algorithms with perceptual learning, such that the spacing and recurrence of each learning category is based on each learner’s accuracy and speed with exemplars of that category in ways that tend to optimize the efficiency and durability of learning. The adaptive elements in PALMs also track learning of each category to mastery criteria that include both accuracy and fluency.3,10 Currently, there are five dermatology PALMS allowing students to practice lesion morphology, lesion distribution, lesion configuration, lesion identification (skin cancer), and lesion diagnosis (Figure 1).

In four out of five dermatology PALMs, students are presented with an image that requires them to choose a diagnosis or description (Figure 2, left).

The exception is the “Lesion Identification: Skin Cancer” PALM, where the user is presented with two images, a regular image and a dermoscopic image, and the software then requires them to choose a diagnosis using both pictures (Figure 2, right). For all modules, after the student selects an answer choice, the software displays the correct response. The PALMs software adapts to the student user by adjusting presented cases based on the learner’s performance, thus making it more customized for each individual student learner. This adaption also allows the student to improve on areas of weak understanding and improve proficiency of the dermatology concepts listed in Figure 1.3,6,8,9 This supplemental platform, however, has never been studied in dermatology training, specifically in PA education. Therefore, the aim of this project was to evaluate the use of PALMs in PA education.

RESEARCH QUESTIONS
1. Will use of supplemental training technologies in dermatology education improve PA student knowledge and diagnostic accuracy of dermatologic conditions?

2. Will use of supplemental training technologies in dermatology education improve PA student perceptions of their dermatology knowledge and skill on clinical rotations?

RESEARCH METHODS
Group and Sample Size
This study was completed at a PA program from a single university. All students from the Class of 2020 (68 students), Class of 2021 (69 students), and Class of 2022 (70 students) used the PALMs educational intervention during the study period, though each class was in a different stage of their education (Figure 3).

Methods
Students are typically trained in dermatology at different times in the curriculum (Figure 3). The traditional lecture-based and lab-based education was maintained in this study, totaling almost 29 hours of instruction. This instruction includes training in Physical Diagnosis of Skin, Hair, and Nails (3 hours) along with a Clinical Medicine Dermatology module (21.5 hours) covering dermatologic disease states. Students also participate in a hands-on skills lab where they practice excisions, punch biopsies, shave biopsies, and dermoscopy. Lastly, all students attend dermatology review lectures during their clinical education year at an “End of Rotation” (EOR) training session (Figure 3).

For the educational intervention, each cohort was required to complete dermatology PALMs specific to their level of education within 2 to 4 weeks of completing their lecture series or lecture-based module (Figure 4). For each dermatology PALM, the technology platform was set. Students completed a pretest, the PALMs training module, then a posttest. Students were then asked to voluntarily complete a survey regarding their experiences using PALMs in dermatology training.

Data Collection and Statistical Analysis
Institutional Review Board (IRB) approval was obtained. IBM SPSS statistical software was used for all analyses. Perception survey responses and dermatology PALMs pretest and posttest scores were collected (Figure 4). Data were de-identified and aggregated by the following groups: Class of 2020, Class of 2021, and Class of 2022. Paired-sample t-tests were used to compare pretest and posttest scores for each group to look for changes before and after dermatology PALMs use, specifically looking at accuracy (percentage of images correctly identified) and fluency (percentage of images correctly interpreted within a response time of 15 seconds or less). Statistical significance was set at p<0.05. In addition to evaluating PA student knowledge, we assessed PA student perceptions of PALMs using a 5-point Likert scale perception survey. Responses ranged from “strongly disagree” to “strongly agree.” For the purposes of this project, responses of “agree” and “strongly agree” were combined and noted as “respondents in agreement.” Overall survey response rates for Classes 2020, 2021, and 2022, were 59.94, 92.75, and 88.57 percent, respectively.

RESULTS
Student Performance
Class of 2020
The Class of 2020 was in their seventh semester of PA education at the time of the study (“PA Year 3 Spring,” Figure 3). This cohort was required to complete the “Lesion Diagnosis” and “Lesion Identification: Skin Cancer” modules. Students performed an average of 195 and 291 practice cases, respectively, for each module and it took students an average of 21.9 minutes and 44.2 minutes, respectively, to complete each module. For both the “Lesion Diagnosis” and “Lesion Identification: Skin Cancer” modules, there was statistically significant improvement (p<0.05) of pretest to posttest scores for both accuracy and fluency (Figure 5).

Class of 2021
The Class of 2021 was in their third semester of PA education at the time of the study (“PA Year 1 Fall,” Figure 3). This cohort was required to complete the “Lesion Morphology,” “Lesion Configuration,” and “Lesion Diagnosis” modules. Students performed an average of 102, 75, and 202 practice cases, respectively, for each module and it took students an average of 11.9 minutes, 8.9 minutes, and 26.6 minutes, respectively, to complete each module. For the “Lesion Morphology,” “Lesion Configuration,” and “Lesion Diagnosis” modules, there was statistically significant improvement (p<0.05) of pretest to posttest scores for both accuracy and fluency (Figure 6).

Class of 2022
The Class of 2022 was in their first semester of PA education at the time of the study (“PA Year 1 Spring,” Figure 3). This cohort was required to complete the “Lesion Morphology” and “Lesion Configuration” modules. Students performed an average of 105 and 85 practice cases, respectively, for each module; it took students an average of 11.5 minutes and 11.1 minutes, respectively, to complete each module. For both the “Lesion Morphology” and “Lesion Configuration” modules, there was statistically significant improvement (p<0.05) of pretest to posttest scores for both accuracy and fluency (Figure 7).

Student Perceptions
More than 80% of respondents from all three cohorts agreed that utilizing the PALMs improved their overall understanding of dermatology. More than 60 percent of respondents from all three cohorts agreed that utilizing PALMs improved their ability to interpret skin lesions and their accuracy in diagnosing dermatology conditions. Lastly, more than 70 percent of respondents from all three cohorts agreed that “online dermatology modules (PALMs) should be added to the curriculum of medical education programs for PAs.” (Table 1)

DISCUSSION
Research Question #1: Will use of supplemental training technologies in dermatology education improve PA student knowledge and diagnostic accuracy of dermatology concepts?

For all three cohorts, the use of supplemental PALMs improved student knowledge in dermatology conditions and diagnoses. These findings suggest that PALMs supplemental modules are a method to provide students with multiple practice examples that enable students to improve their knowledge base in dermatology. Because PALMs are housed in online modules, this supplemental training is an efficient way for students to practice concepts before seeing patients. This PA student performance improvement aligns with research using PALMs for training of medical students and medical residents.6,8,9

The gains seen in these data came from relatively brief learning interventions. With the mastery criteria used in the PALMs tested here, completion times averaged under 20 minutes, with some being completed in an average of about 11 minutes. These results are consistent with earlier work in indicating that PALMs address missing components of learning, such as pattern recognition, that are not much advanced by traditional instructional methods.3,6-9 Traditional learning methods in medical education (e.g., lectures and textbook) allow students to study and learn but often lack opportunity to practice the concepts. With PALMs, students are actively practicing the material they are learning (e.g., diagnosing skin lesions). PALMs systematic interventions targeting perceptual learning can begin to exert their effects in a short time, given appropriate use of the software. Lastly, the pretest levels did not vary greatly by cohort. This result indicates that PALMs have a role in addressing aspects of learning that might not be covered by conventional instruction.

Research Question #2: Will use of supplemental training technologies in dermatology education improve PA student perceptions of their dermatology knowledge and skill on clinical rotations?

Results of the perception surveys note overwhelmingly positive feedback from use of dermatology PALMs in all three cohorts for all questions asked (Table 1). It is important to highlight that respondents agreed that use of dermatology PALMs improved their overall understanding of dermatology, improved their ability to interpret skin lesions, and improved their ability to diagnose dermatology. These results support that dermatology PALMs improved PA student perceptions of their dermatology knowledge and skills. Lastly, a majority of student respondents from all three cohorts felt these dermatology PALMs should be added to PA curriculum, which captures an overall level of satisfaction with the modules and emphasizes student willingness to continue using these modules in PA education.

Instructor Implementation Perspective
Combining e-learning modules with traditional medical education methods has been shown to produce a more time-efficient and effective learning system than traditional learning methods alone.11 As dermatology educators, this supplemental online training platform was simple to implement into our PA program curriculum. The average completion time for each module was about 19 minutes (range: 8-44 minutes), which made adding these modules achievable in a challenging and full curriculum. The adaptive nature of these modules allow individualized learning for all students and real-time feedback as they practiced, which was noted to be helpful for all students no matter their areas of strengths or weaknesses.

It should also be highlighted that these modules have images of patients with all six Fitzpatrick skin phototypes, allowing students to practice dermatology concepts on patients with variable skin pigmentation. Although we observed benefits of perceptual learning with PALMs ability to address aspects of learning that are often difficult to address with traditional instruction, we do not feel these interventions replace the important aspects of conventional instruction used in dermatology education. In fact, the synergy of PALMs as additions to learning with more conventional elements may be particularly powerful in their long-term consequences for medical practice.

Limitations and Future Research
The dermatology PALMs modules were not inclusive of every dermatology lesion or abnormality students may encounter, therefore limiting students’ ability to practice all types of dermatology cases. The findings presented here might not be generalizable since this was a single-site study; conducting this research at multiple PA programs would improve the validity. Future research should be considered to note the impact of dermatology PALMs on Physician Assistant National Certifying Examination (PANCE) board performance and/or clinical rotation performance. It would also be beneficial to study the use of dermatology PALMs in practicing PAs.

CONCLUSION
The combination of improved student performance data and positive student perception study data supports the continued use of dermatology PALMs in the dermatology training of PA students at our PA program.

REFERENCES:

1. Ulman CA, Binder SB, Borges NJ. Assessment of medical students’ proficiency in dermatology: Are medical students adequately prepared to diagnose and treat common dermatologic conditions in the United States? J Educ Eval Health Prof. 2015;12:18. doi:10.3352/jeehp.2015.12.18
2. McCleskey, PE, Gilson, RT, DeVillez RL. Medical student core curriculum in dermatology Survey. J Am Acad Dermatol. 2009;61(1):30-35. Doi: 10.1016/j. jaad.1008.10.066
3 Kellman PJ, Krasne S. Accelerating expertise: Perceptual and adaptive learning technology in medical learning. Med Teach. 2018;40(8):797-802. doi:10.108 0/0142159X.2018.1484897
4. Kellman PJ, Garrigan P. Perceptual learning and human expertise. Physics of Life Reviews. 2009; 6(2):53-84. Doi:10.1016/j.plrev.2008.12.001
5. Loh T, Hsiao J, Shi V. COVID-19 and its effect on medical student education in dermatology. J Am Acad Dermatol. 2020; 83(2), e163-e164. doi:10.1016/j. jaad.2020.05.026
6. Rimoin L, Altieri L, Craft N, Krasne S, Kellman PJ. Training pattern recognition of skin lesion morphology, configuration, and distribution. J Am Acad Dermatol. 2015;72(3):489-495. doi:10.1016/j.jaad.2014.11.016
7. Kellman PJ. Adaptive and perceptual learning technologies in medical education and training. Mil Med. 2013;178(10S):98-106. doi:10.7205/MILMED-D-13-00218
8. Drake T, Krasne S, Hillman J, Kellman P. Applying perceptual and adaptive learning techniques for teaching introductory histopathology. J Pathol Inform. 2013;4(1):34. doi:10.4103/2153-3539.123991
9. Romito BT, Krasne S, Kellman PJ, Dhillon A. The impact of a perceptual and adaptive learning module on transoesophageal echocardiography interpretation by anaesthesiology residents. Br J Anaesth. 2016;117(4):477-481. doi:10.1093/bja/aew295
10. Mettler, E & Kellman P. Adaptive response-time-based category sequencing in perceptual learning. Vision Research. 2014;99:11-123.
11. Scaperotti et al. Development and evaluation of a web-based dermatology teaching tool for preclinical medical students. MedEdPORTAL. 2017;13:10619.

Acknowledgements: This project was funded by the Dermatology PA Foundation (DPAF) Physician Assistant Research Grant Program. We would like to acknowledge and show appreciation to the Society of Dermatology Physician Assistants (SDPA), DPAF, and Sun Pharmaceuticals for supporting this project through the 2019 DPAF Physician Assistant Research Grant Award. We also gratefully acknowledge support from NIH/NCI R01CA236791

Mallory M. Aycock, MPA, PA-C served as the Principal Investigator on this grant project. She is a graduate of Medical College of Georgia in Augusta, GA. She has practiced both in Dermatology and Surgery. Mallory is currently an Adjunct Clinical Assistant Professor at Mercer University in the Department of Physician Assistant Studies.

Craig D. Marker, PhD served as the Statistician on this grant project. He received his PhD and MS in clinical psychology from Rosalind Franklin University and a postdoctoral respecialization in quantitative methods from the University of Virginia. Dr. Marker is Professor, Chair, and Program Director of Clinical Psychology at Mercer University. 

Philip J. Kellman, PhD did not participate in the grant project. He contributed to the manuscript in the area of perceptual and adaptive learning, as this is his area of expertise. Dr. Kellman received his PhD from the University of Pennsylvania. He is Distinguished Professor of Psychology, Adjunct Professor of Surgery, and the current Cognitive Area Chair in the Department of Psychology at the University of California, Los Angeles.

Disclosure: Mallory M. Aycock, MPA, PA-C, and Craig D. Marker, PhD, have disclosed no potential conflicts of interest, financial or otherwise, relating to the content of this article. Philip J. Kellman, PhD, wishes to disclose he is the author of patents in perceptual learning and adaptive learning and the founder of Insight Learning Technology, Inc., which develops learning technology to accelerate pattern recognition and expertise in medical learning and other domains.

Like this article?

Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on Linkdin