Korean J Fam Med Search

CLOSE


Korean J Fam Med > Volume 39(1); 2018 > Article
Park, Park, Kim, Seo, Park, and Hwang: Effects of Module Development and Role Play Course on Clinical Practice Examination Scores during a 4th Year Clerkship

Abstract

Background

After introduction of clinical skills assessment in the Korean Medical Licensing Examination, medical schools have reinforced both experiential learning with real patients and preparatory programs. This study was conducted to investigate whether a clinical practice examination (CPX) preparation program improves students' CPX score in terms of case specificity.

Methods

One hundred and thirteen senior students in a medical school participated in this study. During the fourth-year clerkship, 28 students (24.8%) from three rotation groups took a 3-day CPX preparation course consisting of module development, role play, and comprehensive physical exam skills training. Eleven rotation groups (n=85) were compared as control. Both the intervention and control group took two comprehensive CPXs before and after the clerkship was completed.

Results

There was no significant difference in age, sex, and school type between the two groups. On pre-test CPX, there was no significant difference in total and sectional scores between the two groups. On post-test CPX, total scores of the intervention group were higher than those of the control groups (69.5±4.3 vs. 67.5±4.4, P<0.05). History taking scores were higher in intervention groups (70.0±6.0 vs. 66.0±6.6, P=0.01). The station scores of vaginal discharge with case similarity were higher in the intervention groups (73.0±6.3 vs. 68.9±9.3, P=0.03).

Conclusion

A short CPX preparation course improved history taking ability, but its effect was greater only in a specific case, similar to the pre-course case. Whether this effect was due to the test experience or true improvement in competency requires further investigation.

INTRODUCTION

Most medical schools in the world have recently begun to emphasize competency-based curricula to train highly skilled primary care physicians. The clinical practice examination (CPX) is being used as a means of assessing and training clinical competency. CPX brings several benefits in improving competent primary care physicians. Teachers are able to assess clinical skills of reasoning and communication while students demonstrate their ability to apply both practical knowledge and skills in an authentic clinical setting.1)
In Korea, both the CPX using standardized patients, and the objective structured clinical examination (OSCE) was implemented in the Korean Medical Licensing Examination (KMLE), henceforth assessing the clinical competence of applicants as primary care physicians. It is ideal for students in clinical clerkship to enhance their clinical competence by having a good number of opportunities to experience direct interviews with actual patients. Practically, however, medical schools in Korea would find it difficult to provide an adequate number of opportunities of balanced clinical training with real patients on major clinical presentations because the number of actual patients that students may encounter is limited and the patients are not standardized. This lack of opportunity is then further exacerbated since protecting patients' rights is an essential part of health services. Difficulty of acquiring patient permission about a student practice further limits the range of clinical trainings that can be implemented. Current undergraduate medical education tends to be theoretical and textbook-oriented while classes would hardly provide either simulated or actual patients. Regarding clinical clerkship involving inpatients or outpatients, the diagnosis is usually given by other experts prior to a student-patient encounter, thus restricting students' clinical experience in a disease-oriented manner. This renders the clinical experience partly inadequate to provide authentic training in clinical reasoning skills and competency.
Many medical schools have made attempts using diverse methods of clerkship education to enhance the clinical competency of students and to help them perform well on CPX and OSCE. These attempts to facilitate active learning include running CPX multiple times so as to hone clinical competence while producing feedback, educating students with standardized patients, preparing for CPX by designing a CPX scenario firsthand, and performing role play in small groups. These techniques also facilitate students' learning of medical and interpersonal skills through the reinforcement of bedside teaching.2,3,4,5) Despite these many methods, there has yet to be a verified assessment on the effectiveness of the aforementioned preparations for CPX.
Medical schools in Korea, in an effort to ensure an increase in their KMLE pass rates, have reinforced experiential learning within their clinical clerkship education or programs that are used to prepare students for clinical skills test. These are taken prior to graduation with case studies in the form of CPX or role play. There is, however, a lack of studies regarding the effect of CPX preparation programs on the clinical competency of students. The purpose of this study was to investigate whether CPX preparation programs during the fourth-year clerkship improve the clinical competency of students in terms of case specificity.

METHODS

1. Subjects of Study and Education Method

One hundred and thirteen students in their fourth-year of a medical school located in Seoul participated in this study. Seventy-one (62.8%) of the subjects were male students, and 57 (50.4%) of the subjects were medical college students. The average age was 26.2±2.5 years. During the fourth-year clinical clerkship at the Hanyang University Medical College, a total of 28 students (24.8%), comprising three of 14 clerkship rotation groups, could not participate in the community medicine clerkships, which occurs over the course of a week between June and July in 2015 at a community health center, due to a MERS (Middle East Respiratory Syndrome) outbreak. These three rotation groups then completed a CPX preparation program, which was developed under the supervision of the department of medical education, as an alternative independent curriculum. The details of this preparation program are shown in Table 1. The other 11 rotation groups had completed their respective community medicine training were not subject to any organized CPX-related training program. Effects of the three-day preparation program could thus be compared through a common subsequent comprehensive CPX between the intervention and control groups.
The CPX preparation program was operated by a professor, a clinical instructor, and a medical resident in the department of family medicine. Guidelines for CPX case development was taught as the core lecture on the first day, and each group was then divided into three subgroups consisting of three students each to develop cases on three CPX clinical problems (cough, vaginal discharge, and hand tremor). For case selection, students referred to the ‘2015 basic clinical practice guidelines,’ published by Korea Association of Medical Colleges, using their experience with the actual patients they came across during their clinical clerkship. The three groups were given the same CPX clinical problems, but were asked to develop modules based on different diseases. As for the development of the CPX scenario, a scoring rubric and scoring checklists, which are evaluation tools for reviewing the quality of a clinical performance, the medical resident operated as a facilitator and made amendments as deemed appropriate. Each subgroup was instructed to film a video clip of a typical medical encounter with a standardized patient, in which the checklist items were sufficiently reflected, so that it may be used in the training of simulated patients for role play. All members of each group were gathered to watch the video clips and complete the modules through cross-checking and feedback.
On the second day, each subgroup had a person prepare a schema on cough, vaginal discharge, and hand tremor. This individual summarized the details of learning with a focus on differential diagnosis and presented the summary to colleagues. Afterwards, members of each subgroup participated in role play by changing roles between physician, evaluator, and patient who exhibited cough, vaginal discharge, or hand tremor. The student acting as the physician encountered the student in the patient role for about 10 minutes as if to carry out CPX. Once the role play was completed, the students who were the patient and the evaluator provided 5-minutes of feedback about clinical performance according to checklists. The facilitator provided feedback based on the checklist on strengths and weaknesses of each subgroup after watching all role play practices through a one-way mirror. Then, students who experienced different cases were reorganized into different subgroups to experience different role play.
On the third day, an intensive training on physical examination was held, in which each student watched a standard video clip on basic physical examination (heart, lung, abdomen, neck and oral cavity, neurological examination) prior to having a role play session. Afterwards, the facilitators provided corrections through feedback.

2. Items of Investigation

All students took a CPX on 13 stations in February 2015. CPX on 13 stations was conducted again in July 2015 after the fourth-year clerkship. Both examinations involved a 10-minute one-on-one interview with standardized patient and evaluation of standardized patient on clinical competency of students based on a checklist that consists of six sections (patient satisfaction, history taking, physical exam, clinical courtesy, patient education, and patient-physician interaction). This study sought to analyze the difference between the score marked by the students who underwent these programs and the score marked by the student who did not undergo these programs. Two of the taught CPX clinical problems (vaginal discharge, cough) were comprised some of the problems in the CPX taken in July.

3. Data Analysis

The scores were determined by standardized patients who acted as raters. A chi square test and Student t-test were used to analyze basic characteristics of students. Student t-test and paired t-test were used to compare CPX results between February and July. Statistical significance was assumed when P<0.05, and the analysis of all data was done using IBM SPSS ver. 21.0 (IBM Corp., Armonk, NY, USA).

RESULTS

There was no significant difference in the distribution of age, sex, and school type between the intervention and control group (Table 2). There was no significant difference between two groups in total score and sectional scores (patient satisfaction, history taking, physical exam, clinical courtesy, patient education, and patient-physician interaction) on the CPX conducted in February 2015 (Table 3).
All scores, except for physical exam, increased in both the intervention group and control group on the CPX conducted in July. In particular, the average total score of the CPX intervention group was 69.45±4.34 points, which was significantly higher than 67.53±4.43 points of the control group (P<0.05) (Figure 1). History taking was the only section in which the intervention group exhibited a markedly higher sectional score than did the control group, with a score of 70.0±6.0 points compared to 66.0±6.6 points (P<0.05) (Table 3). There was no notable difference in sections, such as physical examination, education, and patient-physician interaction (Table 3). For each station score, the score on vaginal discharge on the educated case was 73.05±6.31 points for the intervention group, which was significantly higher than 68.87±9.25 points for the control group (P=0.03) (Table 4). On the other hand, there was no difference in scores on cough, which had similar clinical problems as the educated case, but with a different context, between the groups (72.41±8.98 versus 69.14±8.91, P=0.10). Scores of the intervention group were higher than in the control group for clinical problems, such as foamy urine, palpitation, sexual violence, knee pain, and vaginal discharge. A significant score difference was only observed for the history taking section. There was no meaningful difference in CPX scores in February among three intervention groups. There also was no notable difference in CPX scores among groups in July. No significant difference in scores was found across sections and cases.

DISCUSSION

This study investigated the effects of an organized CPX preparation education, which comprises module development, role play, and physical examination, on the outcome of a CPX involving standardized patients. An organized CPX preparation education improved CPX scores in the short term. Previous studies have struggled to provide clear evidence on the effectiveness of CPX preparation education, but this study proved that there is an improvement in CPX scores, while the effect size remains insignificant.
Role play is an education method in which students experience emotional and intellectual responses of another person to explore the motivation for and result of actions, thereby helping them form a desirable behavior or value.6) Full-time fourth-year students at medical schools prepared for CPX based on such role play. Medical education using standardized patients ensures high receptiveness, not only for educators, but also medical students, while culminating in higher effectiveness compared to that using role play.7) However, considerable amount of time that the standardized patients require in their training, coupled with a high cost, means that the clinical skills test within the KMLE is prepared mainly with an attempt to enhance clinical competency using role play. This study was conducted to examine the effect of role play practice on CPX ability of students and to determine which sections of CPX were improved by role play. The results showed that role play practice of students before CPX effectively improved score, specifically for some sections of history taking. Scores on other sections, such as patient-physician interaction and physical examination, were not significantly different compared to the control group. CPX scores increased with repeated usage of cases in a previous study, which improvement may be explained by an increase in test awareness of examinees or a maturation effect during an additional period of educational experience.8) In this study, students' sectional CPX scores improved in all areas within each group over 5 months of clinical clerkship. However, only competency of history taking improved more among the intervention group compared with the control group. We may interpret it as the educational module being the focus on history taking with case specificity. Other educational interventions, such as a general hands-on skill course for comprehensive physical exam skills, were ineffective enough to affect the area score.
There was no significant correlation between school achievement, history taking, and physical exam scores of students;9) history taking and physical exam, sections that require frequent contact with patients, were regarded by students to be the most difficult on the first CPX held in Korea.10) This means that it is difficult to significantly increase scores on history taking and physical exam through ordinary curriculum. In this study, activities of students to prepare checklist for role play, feedback weak points during role play using the checklist, and repeatedly learn about the checklists of history taking section are believed to have directly and indirectly increased scores on history taking section.
Physical exam was practiced by students after watching a video clip about standard physical exam based on demonstration by a medical resident of family medicine. Scores on physical exam section were not increased in this study, possibly for several reasons. First, there is a lack of feedback from colleagues. Studies have revealed a lack of dependability from the result of the colleague-evaluations that had been taken domestically, which resulted in the following: the lack of differentiation between the students as they gave the same scores because of mental pressure; evaluating colleagues based on their general impressions rather than objective criteria, etc.11,12) It is plausible that students found it difficult to study all the necessary physical examinations correctly in those role play situations with their colleagues from the same departments. Female students in particular may be inevitably forced and restricted to playing their roles in a passive manner when acting as patients during a physical exam, which could work as a hindrance in effective experiential training. Moreover, the feedback from patientdoctor relations or physical exam may have lacked enough accuracy as it is more vulnerable to inaccurate observations from a less skilled observer; history taking shows the contrary, as its evaluations are rather simple and the stipulated checklist is written precisely. It may also be interpreted as the actual effectiveness of education was limited because the role play in the physical exam was not operated in the context of the cases, which includes a general physical exam. Second, there is a lack of self-evaluation during physical exam. Students lack experience in receiving appropriate feedback and performing selfevaluation on their clinical competency.13) Failure to recognize weaknesses in the physical exam section is probably one of the reasons why scores on physical exam were not improved.
In a study by Han et al.,14) physician-patient interaction (PPI) score had high correlation with history taking. Smith15) defined history taking as “a communication in which a patient can speak freely and a physician plays the role of an attentive listener to understand and appropriately respond to true wants and needs of the patient on the same level.” This means that PPI can be improved by naturally listening to patients during history taking. While the increase in PPI scores through role play was only marginal in this study despite a significant increase in history taking scores, is possibly that students were unable to fully concentrate on interviewing the patients as they were focused on following the checklist, while the checklist-oriented learning ensured a desirable outcome in history taking. Lee et al.16) explained that since CPX is an examination, students have difficulty in empathizing with their patients because they only concentrate on achieving a correct diagnosis and perceive that patients are actors, thereby reducing empathy expression by students during interviews when compared with actual patients and the CPX interview. During patient-physician interactions, role play alone does not provide adequate training, while clinical practice involving standardized patients may prove to be rather more effective.
Many studies have already discovered that clinical competency of an organ system cannot improve clinical competency of other systems; this is referred to as case specificity.17) The case of vaginal discharge in this study appeared as case of candida vaginitis on the actual examination, and cases practiced by students were trichomonas vaginitis, bacterial vaginitis, and atrophic vaginitis. Scores during history taking were noticeably improved for this item because the practiced cases were of the same clinical presentation as the case used during actual examination. Students practiced cases of gastroesophageal reflux disease (GERD), chronic obstructive pulmonary disease, and asthma for cough symptoms, while a case of tuberculosis appeared on the actual examination. Diverse diseases can be developed into CPX modules in the case of cough, and GERD is a key diagnosis to implicate the digestive system, unlike other cough problems (respiratory system). Therefore, this result suggests that scores may not increase unless role play practice is done on a wide variety of diseases for each item. However, this case specificity alone would not fully explain how the CPX results of the group, which underwent a preparation course, were higher than those of the group that did not undergo a preparation course. Training experiences would have led to acquiring relevant working knowledge and subsequent improvements in clinical competency in general.
The limitations of this study are as follows. First, the scope of this study is limited to activities conducted within a single medical school, and thus carries inevitable restrictions when its findings are presented to a broader scope. Second, the long-term effects of CPX preparation education on the performance at the national examination could not be examined, as only the short-term effects were observed and examined. Research in the future must examine the effect of CPX preparation education on improving the result of the practice examination within the KMLE. Third, it was not feasible to determine which method of CPX education is the most effective among module development, role play, and physical examination. Future studies are therefore required and should include several medical schools with additional examinations of the effects of standardized education modules.
In conclusion, the effects of the CPX preparation program, whose case development and role play among colleagues improved the ability of history taking, was shown to be greater in a station with a high case of similarity. As test experience with repeated exposure resulted in extensive changes in the general clinical competencies in the CPX, we concluded that a short preparatory educational module produced little effect in a clinical sense during clerkship for CPX. Further studies need to be conducted to determine the long-term effect of CPX preparation and its impact on the KMLE.

Notes

CONFLICTS OF INTEREST: No potential conflict of interest relevant to this article was reported.

References

1. Chang A, Boscardin C, Chou CL, Loeser H, Hauer KE. Predicting failing performance on a standardized patient clinical performance examination: the importance of communication and professionalism skills deficits. Acad Med 2009;84(10 Suppl): S101–S104. PMID: 19907367.
crossref pmid
2. Yang E, Suh DJ, Lee Y, Lee S, Kim S, Lee E, et al. Status of clerkship education and its evaluation in Korean medical schools. Korean J Med Educ 2007;19:111–121.
crossref pdf
3. Park WB, Lee SA, Kim EA, Gwack J, Yoo KY, Shin JS, et al. Evaluation using simulated students for reliability of multiple standardized-patients scoring in clinical performance examinations. Med Teach 2007;29:978–980. PMID: 18158676.
crossref pmid
4. Park KH, Kim WJ. Teaching clinical performance examination using action learning techniques. Korean J Med Educ 2012;24:23–30. PMID: 25812788.
crossref pmid pdf
5. Gierk B, Harendza S. Patient selection for bedside teaching: inclusion and exclusion criteria used by teachers. Med Educ 2012;46:228–233. PMID: 22239336.
crossref pmid
6. Kim S. How to apply small group teaching method. Korean J Med Educ 2014;26:83–86. PMID: 25805194.
crossref pmid pdf
7. Bokken L, Rethans JJ, van Heurn L, Duvivier R, Scherpbier A, van der Vleuten C. Students’ views on the use of real patients and simulated patients in undergraduate medical education. Acad Med 2009;84:958–963. PMID: 19550197.
crossref pmid
8. Lee YH, Park JH, Ko JK, Yoo HB. The change of CPX scores according to repeated CPXs. Korean J Med Educ 2011;23:193–202. PMID: 25812612.
crossref pmid pdf
9. Kim JJ, Lee KJ, Choi KY, Lee DW. Analysis of the evaluation for clinical performance examination using standardized patients in one medical school. Korean J Med Educ 2004;16:51–61.
crossref pdf
10. Jun KA, Shin SY. Analysis of first clinical skills examination in the Korean medical licensing examination: focus on examinees' experience in a medical school. Korean J Med Educ 2011;23:203–207. PMID: 25812613.
crossref pmid pdf
11. Kim JY, Son HJ, Lee JH, Kim JH, Hong KP. Educational implication of peer evaluation in problem-based learning. Korean J Med Educ 2005;17:275–284.
crossref pdf
12. Huh S. How to administer the peer evaluation in team-based learning. Korean J Med Educ 2012;24:359–361. PMID: 25813333.
crossref pmid pdf
13. Park WB, Lee SA, Kim EA, Kim YS, Kim SW, Shin JS, et al. Correlation of CPX scores with the scores of the clinical clerkship assessments and written examinations. Korean J Med Educ 2005;17:297–303.
crossref pdf
14. Han JJ, Lee MJ, Im HJ. Comprehension of patient-physician interaction through analysis of relationships between domains in clinical performance examination. Korean J Med Educ 2010;22:177–184. PMID: 25813942.
crossref pmid pdf
15. Smith RC. Patient centered interviewing: an evidence-based method. 2nd ed. Philadelphia (PA): Lippincott Williams & Wilkins. 2002.

16. Lee HM, Park HK, Hwang HS, Chun MY. Patient-centeredness of medical students during a real patient encounter and a standardized patient encounter on the clinical performance examination. Korean J Med Educ 2013;25:139–147. PMID: 25804694.
crossref pmid pdf
17. Norman G, Bordage G, Page G, Keane D. How specific is case specificity? Med Educ 2006;40:618–623. PMID: 16836533.
crossref pmid
Figure 1

Change in the average total score of the clinical practice examination with preparatory intervention.

kjfm-39-23-g001
Table 1

Schedule template for preparatory program for CPX

kjfm-39-23-i001

CPX, clinical practice examination.

Table 2

Baseline characteristics of study subjects

kjfm-39-23-i002

Values are presented as mean±standard error or number (%).

*Obtained using a Student t-test for age; the χ2 test was used for sex and type of school.

Table 3

Comparison of clinical performance examination section scores between intervention and control groups

kjfm-39-23-i003

Values are presented as mean±standard error.

*Obtained by paired t-test within a group. Obtained by Student t-test between groups.

Table 4

Comparison of clinical performance examination station scores according to groups

kjfm-39-23-i004

Values are presented as mean±standard error.

*By Student t-test.



ABOUT
ARTICLE CATEGORY

Browse all articles >

BROWSE ARTICLES
AUTHOR INFORMATION
Editorial Office
Room 2003, Gwanghwamun Officia, 92 Saemunan-ro, Jongno-gu, Seoul 03186, Korea
Tel: +82-2-3210-1537    Tax: +82-2-3210-1538    E-mail: kjfm@kafm.or.kr                

Copyright © 2018 by Korean Academy of Family Medicine. All rights reserved.

Developed in M2community

Close layer
prev next