The Effect of Applying DOPS on BDS Students’ Orthodontic Skills: A Quasi-Experimental Study

Muhammad Azeem¹                                                                            BDS, FCPS                                            

Javed Iqbal²                                                                                                      BDS, M.Phil

Arfan Ul Haq³                                                                                               MDS, FCPS, MCPS, BDS

INTRODUCTION

Directly Observed Procedural Skills (DOPS) is effective teaching tool for teaching of essential procedural skills at undergraduate and postgraduate

level.1 It usually consists of 5 minutes task followed by 5 minutes feedback to students. Therefore it improves performance of students and increases interaction between teachers and students. In comparison, traditional methods of teaching essential procedural skills are devoid of interaction between teachers and students as no feedback session is involved in it.2,3

Studies have been conducted on evaluation of effectiveness of DOPS in different specialties of medical and dentistry. Shahgheibi et al. conducted a study on evaluation of effectiveness of DOPS in maternity units.4

Haq et al. conducted a study on evaluation of effectiveness of DOPS in Opthalmology at undergraduate levels.5 This method of skill assessment is not limited to postgraduate residents. Bazrafkan et al. did comparison of the assessment of dental students’ laboratory performance.6

Very few studies have been conducted on evaluation of effectiveness of DOPS in Orthodontics at undergraduate levels.6 Most of the studies didn’t compare DOPS with other existing traditional teaching methods.

Following this rationale, the objective of present study was to compare the effectiveness of two teaching methods i.e. Directly Observed Procedural Skills (DOPS) and traditional teaching of procedural skills in Orthodontics to dental students. DOPS may enhance learning capacity of procedural skills in orthodontics at the undergraduate level. DOPS ensures that students are given feedback based on direct observation so as to improve the orthodontic skills of students at undergraduate level.

METHODOLOGY

This Quasi-Experimental Study, was designed and conducted at Dental Section-Faisalabad Medical University, Faisalabad. On 30 BDS final year students after taking informed consent and institutional ethical board approval (010/D- FMU) of Dental Section-Faisalabad Medical University. The sample size of 30 was based on the number of BDS final year students that gave consent ot take part in the present study.

Dental students were divided into two groups i.e. 15 students in the DOPS group and 15 students in the traditional teaching group. The distribution was made with assurance to distribute students of similar knowledge in both the groups based on the marks in the third professional BDS exam. So students were statistically same in both the groups at baseline. Obtaining informed consent from the orthodontic patients  and doing wire work (Labial Bow) on plaster models were the skills selected for the study. The faculty of the orthodontic department was given orientation to the DOPS and also in the skills regarding giving good feedback by medical educationists and subject experts. A brief orientation was also provided to the students  regarding  DOPS. The Multiple choice questions (MCQs) tests were taken before and after the procedural skills teaching from both the groups. Based on the expert opinion, DOPS structured teaching scheme was devised and grading/marking form for each procedure was created. On average it took 7 minutes to complete the procedure on each student i.e. 5 minutes for students to complete the task followed by 2 minutes feedback to the students. The cumulative scores were calculated before and after the teaching of procedural skills in both the groups.

DATA ANALYSIS

The mean scores and standard deviations were calculated before and after the teaching of procedural skills in both the groups. Student paired t test was used for comparison of data between the two groups. The level of significance was determined at p < 0.05.

RESULTS

Present study was conducted on 30 BDS final year students (N=30). There were 12 female and 3 male students in each group.

Results showed that at the baseline, for all the taught orthodontic procedural skills, pre-test scores were found to be significantly similar in DOPS and in traditional teaching group (p > 0.05) (Table 1).

Table 1: Comparison (N=30)

Results showed that for all the taught orthodontic procedural skills, post-test scores were found to be significantly higher in DOPS group in comparison with the post-test scores in traditional teaching group (p < 0.05) (Table 1).

DISCUSSION

Workplace based assessment is of the effective way of enhancing educational impact and to improve learning and teaching.7-9 DOPS focuses on technical, operative and procedural skills for basic procedures.10 The objective of present study was to compare the effectiveness of two teaching methods i.e. Directly Observed Procedural Skills (DOPS) and traditional teaching of procedural skills in Orthodontics to dental students.

Obtaining informed consent from orthodontic patients, and doing wire (Labial bow)11 on plaster models were the skills selected for the study. These procedures were selected because of importance of procedures at undergraduate level and observer could mark student’s performance stepwise. There were 12 female and 3 male students in each group in the present study. The equal distribution of male and female students in each group was not possible because of induction of maximum female students in BDS while entrance in dental colleges on merit.

Results in the present study showed that at the baseline, for all the taught orthodontic procedural skills, pre-test scores were found to be significantly similar (p >0.05) in DOPS and in traditional teaching group, this may be due to the fact that the distribution was made with an assurance to distribute students of equal knowledge in both the groups based on the marks in third professional BDS exam. So students were statistically same in both the groups at baseline. The findings of present study are in agreement with the findings of study by Inam et al., where difference among undergraduate students in DOPS group and traditional group was insignificant.5

Results showed that for all the taught orthodontic procedural skills, post-test scores were found to be significantly higher (p <0.05) in DOPS group in comparison with the post-test scores in traditional teaching group. Thus it was found that the teaching of orthodontic procedural skills to BDS students is more effective using DOPS than the traditional teaching of procedural skills. The findings of present study are in agreement with the findings of study by Shahgheibi et al., where DOPS were found to be very useful in increasing students’ skill learning.4 The findings of present study are also in agreement with the findings of study by Inam et al., where undergraduate students in DOPS group performed significantly better than their counterparts in

traditional group.5 The findings are also in accordance with the findings of study by Bazrafkan et al., where findings revealed that DOPS is a useful tool for assessing dental students’ dental laboratory skills.6

In general, current evidence showed that workplace based assessment is not yet sufficient in terms of reliability but highly valid.12-16 DOPS is not widely used in undergraduate dental education in Pakistan, but in light of findings of current study DOPs may enhance learning capacity of procedural skills in orthodontics at undergraduate level. However there are certain disadvantages of DOPS, such as, financial cost, time cost, and training of trainers and trainees and feasibility issues.17-19

Limitation of present study is small sample size, for which future large scale studies are suggested to validate the results of current study.

CONCLUSION

The teaching of orthodontic procedural skills to BDS students is more effective using DOPS than the traditional teaching of procedural skills.

CONFLICT OF INTEREST

None to declare

REFERENCES

1. Hatala R, Cook DA, Zendejas B, Hamstra SJ, Brydges Feedback for simulation-based procedural skills training: a meta-analysis and critical narrative synthesis. Adv Health Sci Educ. 2014;19:251-72. https://doi.org/10.1007/s10459-013-9462-8
2. LaDonna KA, Hatala R, Lingard L, Voyer S, Watling C. Staging a performance: learners’ perceptions about direct observation during residency. Med Educ. 2017;51:498-10. https://doi.org/10.1111/medu.13232 
3. Modi JN, Gupta P, Singh Competency-based medical education, entrustment and assessment. Indian Pediatr. 2015;52:413-20. https://doi.org/10.1007/s13312-015-0647-5 
4. Shahgheibi SH, Pooladi A, BahramRezaie M, Farhadifar F, Khatibi
5. Evaluation of the Effects of Direct Observation of Procedural Skills (DOPS) on clinical externship students’ learning level in obstetrics ward of kurdistan university of medical sciences. J Med Educ. 2009 25;29-33.
6. Inam UH, Jamil B, Durrani Procedural skills; learning of procedural skills in ophthalmology through directly observed procedural skills (DOPS) or traditional methods; a randomized controlled trial. Professional Med J 2018; 25:1207-12.
7. Bazrafkan L, Shokrpour N, Torabi Comparison of the Assessment of Dental Students’ Laboratory Performance through MCQ and DOPS Methods. J Med Educ. 2009;13:3-8
8. Kogan JR, Conforti LN, Bernabeo E, Iobst W, Holmboe E. How faculty members experience workplace-based assessment rater training: a qualitative Med Educ. 2015;49:692-08. https://doi.org/10.1111/medu.12733 
9. Massie J, Ali JM. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. . Adv Health Sci Educ. 2016;21:455-73. https://doi.org/10.1007/s10459-015-9614-0 
10. Jain MD, Tomlinson GA, Lam D, Liu J, Damaraju D, Detsky AS, Devine Workplace-based assessment of internal medicine resident diagnostic accuracy. J Grad Med Educ.2014;6(3):532-35. https://doi.org/10.4300/JGME-D-13-00431.1 
11. Augustine K, McCoubrie P, Wilkinson JR, McKnight Workplace- based assessment in radiology – where to now? Clin Radiol 2010; 65: 325-32.https://doi.org/10.1016/j.crad.2009.12.004 
12. Mahmood A, Nazir A Structured Programme for Post Graduate Training in Orthodontics. Pak Orthod J. 2014;6:87-108.
13. Sia JY, Harrison Trainers’ and trainees’ perceptions of workplace- based assessment in orthodontics. J Orthod. 2015;42:284-300. https://doi.org/10.1179/1465313315Y.0000000015 
14. McKenzie S, Burgess A, Mellis C. Interns reflect: the effect of formative assessment with feedback during pre-internship. Adv Health
15. Sci Educ Theory Pract. 2017;8:51-56.https://doi.org/10.2147/AMEP.S114480 
16. General Medical Council. Workplace-Based Assessment: A Guideline for Implementation. London: GMC,
17. Siau K, Levi R, Howarth L, Broughton R, Feeney M, Beales IL, Tzivinikos C, Narula Validity Evidence for Direct Observation of Procedural Skills in Paediatric Gastroscopy. J Pediatr Gastroenterol Nutr. 2018;67:111-16 https://doi.org/10.1097/MPG.0000000000002089
18. Siau K, Dunckley P, Valori R, Feeney M, Hawkes ND, Anderson JT, Beales IL, Wells C, Thomas-Gibson S, Johnson Changes in scoring of Direct Observation of Procedural Skills (DOPS) forms and the impact on competence assessment. Endoscopy. 2018;50:770-78. https://doi.org/10.1055/a-0576-6667 
19. Erfani Khanghahi M, Ebadi Fard Azar Direct observation of procedural skills (DOPS) evaluation method: Systematic review of evidence. Med J Islam Repub Iran. 2018;32:254-61. https://doi.org/10.14196/mjiri.32.45 
20. Amini A, Shirzad F, Mohseni MA, Sadeghpour A, Elmi Designing Direct Observation of Procedural Skills (DOPS) Test for Selective Skills of Orthopedic Residents and Evaluating Its Effects from Their Points of View. Res Dev Med Educ. 2015;4:147-52. https://doi.org/10.15171/rdme.2015.026 
21. Singh T, Kundra S, Gupta Direct observation and focused feedback for clinical skills training. Indian Pediatr. 2014;51:713-17. https://doi.org/10.1007/s13312-014-0487-8