Dental Students Perceptions About Assessment Methods

Nabiha Farasat Khan¹                                            BDS, M. Phil, MHPE

Muhammad Saeed²                                                           BDS

Attia Bari³                                                                  MBBS, FCPS, MHPE

Arshad Kamal Butt⁴                                              MBBS, FCPS, MHPE

INTRODUCTION

Organized technique to gather evidence about student’s performance is called assessment.¹ To meet the demand of universal change in medical education assessment tools have evolved over the years,² and currently the concept of examination is to assess for learning.^[1] In Western countries a number of different assessment techniques were utilized since 1950’s.^[2] In Pakistan higher authorities in medical education like Pakistan Medical

& Dental council (PM&DC) and Higher Education

Commission (HEC) have also encouraged the utilization of new learning and teaching strategies; and hence take steps to reform curricula according to the need of Global advancement in Health Profession Education.^5,6

Traditional assessment procedures are definitely essential to assess cognizance but are unsuitable and incomplete tools to measure higher cognitive levels.⁷ There have been issues regarding its reliability and validity from students and faculty about assessment. Continuous efforts were accomplished to assemble assessment more objective and reliable than subjective to confirm adequate knowledge of students in the concern subject with significant potential of scrutiny, recognition and acquisition of interpretation abilities.^8,9 Reported requirement of students’ knowledge according to Miller pyramid (1990) comprised knows the fact, show how accomplishment of clinical techniques and performance.¹⁰ OSCE/OSPE, SEQ and structured oral viva are the assessment formats that authorize the instructors to assess students all cognitive domains of Bloom’s taxonomy including cognition, application, interpretation, synthesis and judgment abilities.¹¹ OSCE is an established assessment tool; which helps to assess student’s clinical skills, its reliability and validity is also confirmed, whereas for the assessment of preclinical and para-clinical subjects a modified form of OSCE that is known as OSPE is used.^8,12,13 OSPE like OSCE is also a valid tool for assessment purposes.¹⁴ A structured oral viva reduces the bias, anxiety and luck factors. The validity and reliability of structured viva can be increased by making multiple checklists.¹¹ On the other hand SEQ’s are easier, have good reliability & validity.¹⁵

Students most likely wish to assess their cognizance through SEQ, whereas MCQ as an assessment tool is also appreciated by them.⁶ Consequences of Amin TT from Saudi-Arabia detected MCQ as the most approving assessment tool,¹⁵ however study participants of Anwar M from Riphah International University, Pakistan faced difficulties in MCQ pattern.¹⁶

Conventional assessment methods namely SAQ, MCQ and viva were applied in this department (Oral Pathology) previously. This study was conducted to check the student’s perception about new assessment tools that were applied for the first time in Oral Pathology Department of a public sector medical college.

METHODOLOGY

The study was conducted at The Dental section, Bolan Medical College (BMC), Quetta in December 2017. This was a qualitative exploratory research, conducted through interviews. Approval for the study was taken from the Institutional Review Board of BMC Letter No. AdminI/BMC/2018/2848/50. Universal sampling was done after informed consent with inclusion of all 25 students of both genders in 3rd year dentistry in BMC who had gone through all assessments methods including MCQ, SEQ, SAQ, Viva and OSPE. Interview was conducted by senior demonstrator by unstructured questions. The questions were about their liking of a particular method of examination among all the methods, reason for liking or disliking that method of assessment. Particular questions asked from the students were;

1. Why did you prefer OSCE as an assessment tool?
2. What are the causes of interest in SEQ?
3. What is the source of appeal in viva?

Our qualitative analysis was a thematic description which aided in organizing the content and arriving at a narrative description of the student’s liking about a particular method of assessment. We identified codes through open coding process.

RESULTS

More than half of the study population (60%) was female. Seventeen out of 25 students were 21-22 years of age with the mean age of 19±2years. Majority 13/25 (52%) liked OSPE followed by viva 6 (24%) shown in (Table 1).

Initially open coding based on important categories of interest generated a total of 26 codes and then through axial and selective coding these was condensed to 13 subthemes.

Table 1: Demographics of the study participants

Fig.1: Hierarchy of themes and subthemes

Finally, by merging of open codes and axial codes subcategories were arranged under three major themes. These themes seem to be discrete, but there is considerable overlap among them. The hierarchy of themes and subthemes are given in Fig.1.

Justice: Every examination method tests the knowledge, insight and skills of the student. The dentistry students liked the OSCE due to multiple reasons and the major reason was marking equality and justice. Majority of the student’s reply was the same narrations when researcher interrogated about the interest in assessment methods and the reason behind their interest in OSCE. They said that:

Student 2: “OSCE allow students to be judged on same parameters.”

In viva, some students are evaluated for longer time as compared to others depending upon the examiners will. Students perceived that this bias is removed in OSCE and every student gets equal time. Student 4, 9, 24: “I prefer OSCE as an assessment tool and the reason behind it is that this new test system where every student gets equal time to express their views which reduces examiners bias.” Students trusted OSCE because they have the trust that they will be given marks according to what they have performed. Student 7, 15: “Before introducing OSCE we were unsure about equality of marking but now as it tests the skills of students with equal marking, we are confident about our assessment, they claim that OSCE reduces the chance of inequality”. Student 10, 12: “I can do it comfortably, I prefer OSCE due to its equality in time and marks.”

Some students preferred OSCE as best method of assessment due to better time management.

Student 13, 14:  Some students state that “OSCE is very fair assessment method, it is easy to understand and solve. Applicable for time management”

Participants of current study felt happy with the introduction of OSCE as an assessment method and were of the view that they would prefer and recommend this exam format for use in future.

Students thinks OSPE is a true reflection of justice as the scores obtained in the orals are affected by student’s self-confidence, anxiety and examiner’s impression they do not truly reflect the student’s competence.

Time Management: Students were asked about the type of assessment method they feel to be most suitable for? What was the cause of interest in SEQ?

SEQ are often used as an assessment tool by examiners as it is considered to be a good method of assessing higher order thinking of students in a shorter time.

Student 1, 18: “I choose it because I think that SEQ is less time consuming, indicates precise notes easy to understand, and is more precise.”

As the SAQ require longer time for students to think, organize and compose their answers they can assess higher order thinking. The students who have studied thoroughly find this method of assessment a better method as they think they can answer the question properly in a short time. Student 16: “I like SEQ as an assessment instrument due to its time management quality and it is precise.” Students

22: “I think that SEQ is convenient to understand and solve”

CONFIDENCE

Viva or oral examination is an integral part of assessment in medical school examination system. The student’s response to questions is verbal instead of writing. The ability to express views, knowledge and confidence of facing the examiner is evaluated in viva examination. Similar is the perception of students who liked viva as best method of examination and mainly due to the reason that viva is helpful in building confidence. When students were inquired about the causes of their interest in viva they answer; Student 3, 23: “My choice for assessment method is viva as it builds up confidence and polishes knowledge.” they revise all learned material during viva,” and It covers all aspects of curriculum. Student 6: “I like viva as an assessment method. It flourishes our confidence level.”

Student 8, 25: Students declare that “Viva is the best method for assessment, it polishes practical skills.” It assesses the deep knowledge of students.” it covers the syllabus when it is taken along with SAQ.

Students 11: “Viva along with SAQ boosts belief. Students study deep and thorough for preparing viva.” Students 17: I believe that that “Viva has ability to explore the level of student’s cognition; moreover, it presents students ability to convey their knowledge. Viva is capable to increase confidence, students study in detail to prepare viva.”

DISCUSSION

This is the first study conducted to check the perception of medical students about SAQ, SEQ, MCQ, OSCE/OSPE in a public sector dental college in Balochistan and has provided important data to reorganize teaching/learning and examination format in dentistry to the faculty and examiners. Assessment tools assists in classification, prediction and accountability purposes.¹  Modification in assessment methods bring changes in students learning approach so instead of assessment of students learning (testing culture) it should be assessment for students learning (assessment culture).¹⁶ The aim of assessment is to balance between student’s knowledge and skills after completing their learning and to connect the discontinuity between their learning and working.¹⁶

Negligible number of teachers utilizes formative assessment procedures which provide not only feedback but heighten students’ self-esteem but it flourishes skills also.¹⁷ Literature reports of several studies have concluded that students approve those methods of assessment that enhance competencies pertinent for their future professional life and making a bridge between learning and working.¹⁶

New assessment techniques help in evaluating the student’s ability to solve the real life difficult problems. OSCE is a précised, objective, reproducible, valid and reliable assessment tool.¹⁸ In our study majority (52%) of 3rd year BDS students favor OSCE, they said that it was the best pattern of assessment as it was more convenient and it would not drag students into unwanted things. Students are allowed to be judged in a same para-meter. Our results were similar to Zayyan M and Abidullah et al.¹⁹ Zayyan M concluded that OSCE presents flexible and wide range of fairness, repeatable and is simple to remember.¹⁹ On the other hand 67 participants of Shitu B study recognize OSCE as a fair assessment tool in evaluation of their cognition and acquisition.²⁰ Our results indicated student’s satisfaction towards OSCE due to its equality in task and marks distribution. Moreover, analyzing other studies, we found that OSCE has potential to test all level of clinical skills including technical and interpersonal skills.²¹Outcomes of current research demonstrated student’s preference towards scenario based OSCE the reason behind it was its clarity.¹³, 10 Respondents of Shitu’s study confirm this statement as 87/122 samples were satisfied about the clarity and adequacy of this exam pattern.²⁰

An assessment of a student’s knowledge can be gathered through viva along with SAQ. If properly supervised viva assists in checking the communication skills and is also helpful in assessment of student’s professional attitude.⁹ In our study students show positive perceptions of oral structured viva and it was the second most popular method of assessment. According to Joughin it is the type of assessment where students transfer answer orally instead of written form.²² Results of research conducted by Anastakis et al present the perception of postgraduate residents; they assume that structured oral viva is a useful assessment method.²³ Same was true in our study where study samples were satisfied about the clear and understandable oral structured questions. They reported that viva questions were precise, were in a proper sequence and heighten their confidence. Hashim et al also confirmed these findings as majority of their study population (98%) were satisfied with structured viva.²⁴ Likewise 68% study participants of Gargi Dangre-Mudey also support our findings that structured viva is a successful assessment tool.²⁵

SEQ assist in evaluation of the student’s capacity to recognize the principal constitute of issue and its analytical solution. SEQ approach focuses on competencies that do not involve clinical demonstrations.²⁶ In our study 16% dental students of 3rd year liked SEQ, as it was found to be precise, was less time consuming and easy to understand. Same conclusion was identified in Shilpa GS where 35% (n=/150) students declared SEQ as less time consuming tool and 29% reported that SEQ was structured and precise.²⁷ Strengths, Limitation &Generalizability

It is the first qualitative study in Balochistan as well as in Pakistan which assesses the perceptions of dental students about assessment methods. Students of this study were keen to learn and willing to adopt innovative assessment strategies. This research work comprises students of 3rd year BDS of only one medical college therefore for more generalize results, investigations to check the perceptions of both medical as well as dental students should include students from different professional years and other medical colleges as well with an even larger sample size preferably comparing private and public sector institutions.

CONCLUSION

OSCE is the most accepted assessment method among 3rd year dental students of public sector Bolan Medical College, Quetta because the timing and markings are just and this gives OSCE its main strength. Second highly rated assessment tool is oral structured viva due to the reason that viva is helpful in building confidence.

ACKNOWLEDGEMENT

Authors acknowledge students of 3rd BDS for their voluntarily participation in this research.

Conflict of interest: None

Source of funding: Nothing to declare

REFERENCES

1. hmad S, Mussawy J. Assessment Practice: Student ‘ s and Teachers’ Perceptions of Classroom Assessment University of Massachusetts School of Education. Master Capstone Project. 2009.
2. Ferris HA, O’ Flynn D. Assessment in Medical Education; WhatAre We Trying to Achieve? Int J High Educ [Internet]. 2015;4(2): 139-44. Available from: http://www.sciedu.ca/journal/index.php/ijhe/article/view/6662
3. Birenbaum M, Feldman R a. Relationships between learning patterns and attitudes towards two assessment formats. Educ Res. 1998;40(1): 90-8. https://doi.org/10.1080/0013188980400109
4. Norcini JJ, McKinley DW. Assessment methods in medical education.Teach Teach Educ. 2007;23(3):239-50. https://doi.org/10.1016/j.tate.2006.12.021
5. Rauf A, Shamim MS, Aly SM, Chundrigar T, Alam SN. Formativeassessment in undergraduate medical education?: concept, implementation and hurdles. J Pak Med Assoc. 2014;1:72-5.
6. Batool H, Mumtaz A, Chughtai AS. Perception of UndergraduateMedical Students about different formats of assessment in subject of Pathology. JUMDS. 2017;8(2):40-5.
7. Miller GE. The assessment of clinical skills/competence/performance.Acad Med [Internet]. 1990;65(9):S63-7. Available from: http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landin gpage&an=00001888-199009000-00045 https://doi.org/10.1097/00001888-199009000-00045
8. Malhotra S, Shah K, Patel V. Objective structured practicalexamination as a tool for the formative assessment of practical skills of undergraduate students in pharmacology. J Educ Health Promot[Internet]. https://doi.org/10.4103/2277-9531.119040
9. Zafar M, Yaqinuddin A, Ikram F, Ganguly P. Practical Examinations- OSPE, OSCE and Spot. J Uni Med Dent Coll. 2013:1-16.
10. Puryer J. Dental Undergraduate Views of Objective StructuredClinical Examinations (OSCEs): A Literature Review. Dent J 2016;4(4):6. https://doi.org/10.3390/dj4010006
11. Shenwai MR, Patil KB. Introduction of structured oral examinationas a novel assessment tool to first year medical students in physiology. J Clin Diagnostic Res. 2013;7(11):2544-7. https://doi.org/10.7860/JCDR/2013/7350.3606
12. Tabish SA. Assessment methods in medical education. Int J HealthSci (Qassim). 2008;2(2):3-7.
13. Sandila MP, Ahad a., Khani ZK. An objective structured practicalexamination to test students in experimental physiology. J Pak Med Assoc. 2001;51(6):207-10.
14. Kurdi MS. Essay type questions & their improvements & shortanswer questions. Conference paper. 2015.
15. Amin TT, Kaliyadan F, Al Qattan EA, Al Majed MH, Al KhanjafHS, Mirza M. Knowledge, attitudes and barriers related to participation of medical students in research in three Arab Universities. Educ Med J. 2012;4(1):43-56.
16. Masood A, FMH Hameed. Students’ Perceptions towards Formativeand Summative Assessment: A Single Institution Study. J Islam Int Med Coll. 2016;11(1):35-40.
17. Gulikers JTM, Bastiaens TJ, Kirschner P a., Kester L. RelationsBetween Student Perceptions of Assessment Authenticity, Study Approaches and Learning Outcome. Stud Educ Eval. 2006;32(4):381-400. https://doi.org/10.1016/j.stueduc.2006.10.003
18. O’Shaughnessy SM, Joyce P. Summative and Formative Assessmentin Medicine: The Experience of an Anaesthesia Trainee. Int J High Educ. 2015;4(2):198-206.
19. Zayyan M. Objective structured clinical examination: Theassessment of choice. Oman Med J. 2011;26(4):219-22. https://doi.org/10.5001/omj.2011.55
20. Khan A, Ayub M, Shah Z. An Audit of the Medical Students ‘Perceptions regarding Objective Structured Clinical Examination. Educ Res Int. 2016;2016:1-4. https://doi.org/10.1155/2016/4806398.
21. Shitu B, Girma T. Objective Structured Clinical (Osce) : Examinee’ S Perception At Department of Pediatrics and Child Health, Jimma University. Ethiop J Heal Sci. 2008;18(2):47-52.
22. Patricia Marten-Daniel HDD. Toolkit and Guide for Administratorsof IEN Programs. 2013. 26 p.
23. Joughin G. Dimensions of oral assessment. Assess Eval High Educ. 1998;23(4):367-78. https://doi.org/10.1080/0260293980230404
24. Anastakis DJ, Cohen R, Reznick RK. The structured oralexamination as a method for assessing surgical residents. Am J Surg. 1991;162(1):67-70. https://doi.org/10.1016/0002-9610(91)90205-R
25. Hashim R, Ayyub A, Hameed S, Ali S. “Structured viva as anassessment tool?: Perceptions of Undergraduate Medical Students.” Pak Armed Forces Med J. 2015;65(1):141-4.
26. Dangre-mudey G, Damke S, Tankhiwale N, Mudey A. Assessmentof perception for objectively structured viva voce amongst undergraduate medical students and teaching faculties in a medical college of central India. Int J Res Med Sci. 2016;4(7):2951-4. https://doi.org/10.18203/2320-6012.ijrms20161983
27. Kramer G a, Albino JEN, Andrieu SC, Hendricson WD, HensonL, Horn BD, et al. Dental student assessment toolbox. J Dent Educ. 2009;73(1):12-35.

1. Associate Professor, Department of Oral Pathology, Bolan Medical College, Quetta.
2. Senior Demonstrator, Department of Prosthodontics, Bolan Medical College, Quetta.
3. Associate Professor, Department of Paediatric, Children Hospital, Lahore, Pakistan.
4. Former Professor, Department of Medicine, Sheikh Zayed Federal Postgraduate Medical Institute, Lahore.

Corresponding author: “Dr. Nabiha Farasat Khan” < nabihasaeed@hotmail.com >