Introduction: In this study, we compared the pretreatment conditions, treatment characteristics, and orthodontic outcomes of 3 groups of subjects selected for the American Board of Orthodontics (ABO) phase III clinical examination. One group was selected retrospectively by graduating residents just before their graduation.
The 2 prospective groups were treated at separate institutions. The students at 1 institution were not aware that these patients would be potential ABO cases (prospective, blinded), but the students at the second institution were aware that these subjects would serve as their pool of potential patients for the ABO examination (prospective, unblinded). In addition to comparing the 3 groups, all cases were categorized as passing or failing based on their total objective grading system (ABO-OGS) score to assess the ABO-OGS criteria that were the most challenging to meet. Methods: Chart histories and orthodontic dental casts (pretreatment and posttreatment) were collected for 133 subjects. Information regarding demographics, initialmalocclusion type, treatment modality, treatment duration, appointment frequency, and missed appointments were collected from chart histories.
Pretreatment dental casts were evaluated by using the discrepancy index; the index of complexity, outcome, and need; and the peer assessment rating. Posttreatment dental casts were evaluated with the peer assessment rating and the ABO-OGS. Results: The only significant pretreatment characteristic with predictive power for favorable orthodontic outcome was Angle Class I (3.1 odds ratio for passing the ABO-OGS) compared with the Class II subjects. The prospective unblinded group receivedmore extraction and headgear therapy than did the other groups. The retrospective group had significantly lower total ABO-OGS posttreatment scores and a higher passing rate compared with the prospective groups. Conclusions: Angle Class I malocclusions appear to have some advantage for achieving passing ABO-OGS scores, as does the retrospective selection of cases.
Successful board certification appears difficult to accomplish based on a prospective model for orthodontic graduate residents. New graduate candidates might be at a disadvantage compared with traditional candidates because they often cannot take advantage of the posttreatment settling phase. Alignment, marginal ridges, and occlusal contacts appear to be where most points are deducted in the evaluation of ABO-OGS certification cases.
Since its inception in 1929, the American Board of Orthodontics (ABO) has striven to certify as many practicing orthodontists as possible and elevate the standards of the practice of orthodontics.1,2 When the percentages of board-certified orthodontists were 13% to 17% in the late 1970s, the board began efforts to increase the numbers of board-certified orthodontists.
1 In 2001, the ABO began actively pursuing the idea and the feasibility of certifying graduating orthodontic residents in the resident clinical outcomes study, or the pilot study (PS). This led to a 4-year collaborative project between the ABO and 16 American orthodontic graduate programs accredited by the Commission on Dental Accreditation.3,4 This project investigated whether orthodontic residents could provide start-tofinish treatment for 6 patients with ABO-quality results.
At its inception, this PS was designed so that cases would be prospectively identified at the time of patient assignment, and residents, faculty, and patients at the participating programs would be aware of this PS designation.
Orthodontic program directors at the participating institutions were asked to prospectively designate 12 patients for each incoming 2002 resident, 6 of whomwould be presented to the board after treatment. Participating orthodontic residentswould treat the patients, frombanding
to debanding, and be eligible to present the cases to the ABO to earn a 10-year time-limited certificate. Sixteen graduate orthodontic programs in the United States agreed to participate in the PS. Only 1 orthodontic program agreed to participate under the requirement that
all persons involved in the treatment would be blinded to the designation of the prospectively selected PS patients. This method was chosen to prevent any differential treatment of the PS subjects.The subjects at the other 15 programs were treated in an unblinded fashion,
so that residents, faculty, and patients knew they were participating in the PS.
During the PS, the study protocol was altered. The participant was allowed to present 6 cases for certification that included only 1 from the previously selected 12 that were prospectively designated. An incentive of a 15-year time-limited certificate was offered to residents
presenting all prospectively selected PS cases, and 12% of PS examinees successfully earned this 15-year certificate.
5 The PS concluded in February 2006, when 50 participating orthodontic residents attended the ABO Clinical Examination. Forty-five candidates successfully obtained ABO certification, for a pass rate of 90%. This compared with 33 traditional candidates passing the examination at a rate of 85%.Therewas amean difference of 2.38 ABO-OGS points for passing cases between the resident and the traditional examinees in the PS. The board concluded that the cases presented had sufficient complexity, with an average discrepancy index (DI) score of 16.96 for the student cases, compared with an average DI score of 21.84 for the regular examinees. The PS participants presented 422 cases, of which 58% were from the original prospectively selected PS group.5 The board believed that this result positively affirmed that residents could treat toABOstandards during their orthodontic graduate programs. As a result, the board has now instituted new certification guidelines for recent graduates.4 The new certification process has provided the impetus for graduate programs to self-evaluate their patient populations and the quality of orthodontic
treatment provided in each residency program.
In this study, we aimed to determine, with the aid of the DI; the peer assessment rating (PAR); the index of treatment complexity, outcome, and need (ICON); and the ABO objective grading system (ABO-OGS), whether there were significant differences in pretreatment conditions, treatment characteristics, and orthodontic treatment outcomes between ABO cases selected by using the 3 methods.We determined whether any pretreatment characteristics had predictive value in determining the orthodontic treatment quality outcome.
Additionally, we categorically compared the ABO points deducted for failing vs passing cases, to determine which intraoral locations were the most difficult for the entire study sample.
MATERIAL AND METHODS
All study procedures were approved by the institutional eview board at the University of Washington.
The sample consisted of complete chart histories nd dental casts (pretreatment and posttreatment) of ll subjects. The sample comprised 3 groups. Groups (retrospectively selected) and 2 (prospective linded) were collected from the retention archives f a graduate orthodontic program participating in he PS. Group 3 (prospective unblinded) was collected rom another graduate orthodontic program participating n the PS.
No exclusion criteria were defined to prevent atients from participating in the study, as was the ase with the initial PS guidelines. Patients were ncluded irrespective of age, sex, race, or orthodontic roblem if they were nonsyndromic and comprehensive rthodontic treatment was planned.
The ABO stipulated hat ‘‘cases should be representative of a cross section of linical problems and of adequate difficulty to represent he resident’s ability to diagnose and treat orthodontic atients” in the original PS guidelines.5 When records ere collected for this study, subjects were excluded of they were still in active orthodontic treatment, had ncomplete records, had transferred treatment outside he assigned graduate clinic, or had never started reatment after the prospective PS designation.
According to these criteria, 49 records were initially ollected for group 1, which included subjects who were etrospectively selected by the graduating residents classes of 2003-2006) to be used in a simulated ABO xamination. Two subjects were excluded from this roup because of incomplete records, leaving 47 ubjects in group 1. Group 2, the prospective blinded roup, was prospectively selected to be part of the PS, nd their study participation was concealed from all persons nvolved in the orthodontic treatment. Group 2 initially ontained 57 subjects, but 16 were excluded based n the exclusion criteria, leaving 41 subjects. Group 3 ecords were gathered from another institution that reated patients in a prospective unblinded fashion. All ersons involved (faculty, residents, patients) were ware of the PS designation. Group 3 started with 50 subject records, but 5 were excluded, leaving 45 rospective unblinded subjects. For the reasons outlined bove, 23 subjects were excluded from the entire subject ample. All subject materials were deidentified nd labeled with an identification number to facilitate nvestigator blinding.
Subject records consisted of chart histories, and retreatment and posttreatment dental casts. Chart histories were reviewed to gather information about demographics, initial malocclusion, treatment type, treatment duration, frequency of appointments, and missed
appointments during active orthodontic treatment. In few cases, phase 1 treatment had previously been provided before the PS. In these circumstances, only information from the comprehensive phase of treatment was collected for these patients. This protocol is consistent
with the ABO’s evaluation of 2-phase patients in traditional board examinations.
Pretreatment dental casts were scored by using the DI, PAR, and ICON by 2 calibrated, independent examiners.
Posttreatment dental casts were scored by 2 examiners using the PAR, ICON, and ABO-OGS.6-10 The radiographic component of the ABO-OGS index was excluded, because many patients had no posttreatment panoramic radiographs. Additionally, numerous studies have questioned the usefulness of panoramic radiography to assess root parallelism because of inherent image distortion, especially in premolar extraction sites.11,12 The ABO-OGS scores were adjusted based on average PS radiographic deductions to account for this exclusion.
The pretreatment and posttreatment casts from each site were combined, deidentified, assigned identification numbers, and measured in random order. Two investigators measured all dental casts independently, and the mean score was used unless significant differences
were noted in the scores (weighted PAR, 5 points;
weighted ICON, 9 points; ABO-OGS, 4 points). When differences were greater than these values, the dental casts were rescored by consensus, and the consensus score was used. Twenty-one (15.8%) cases had to be rescored by consensus.
To determine intraexaminer error, 10 casts were rescored later by each examiner. Intraexaminer error was evaluated by using the intraclass correlation coefficient for all examiners involved in the study (Table I).
Pretreatment conditions and treatment characteristics were assessed and compared both qualitatively and quantitatively. These scores were compared with posttreatment conditions as assessed by the PAR, ICON, and ABO-OGS to determine treatment changes
and the quality of orthodontic treatment between the groups.
Descriptive statistics (means, standard deviations, and ranges) were calculated for pretreatment DI and ICON, pretreatment and posttreatment PAR, and posttreatment ABO-OGS scores. Descriptive statistics were also performed for patient demographics, initial malocclusion (type and severity), treatment type, treatment duration, and number of orthodontic appointments.
Analysis of variance (ANOVA) was used to test for differences in continuous variables between the groups. Pairwise between-group comparisons were carried out when ANOVA indicated differences between the groups. The Bonferroni adjustment to the significance level was used to correct for multiple comparisons in post-hoc analyses. This correction was applied to prevent inflation of the type 1 error rate caused by multiple comparisons.13 When the sample was divided into passing and failing cases, the average scores were compared with t tests.
Logistic regression was used to determine whether any pretreatment variables could be used as reliable predictors of successful board-quality treatment. A stepwise model-building algorithm was used to identify a subset of available covariates that was highly predictive
of successful board-quality treatment.
The statistician was blinded to treatment group identification until the analyses were completed. For all analyses, the levels of significance were set at P\0.05 and P \0.017 when the Bonferroni adjustment was performed.
Table I. Inraobserver error
Intraclass correlation coefficient
Examiner 1 (BHS) Examiner 2 (CJ) Examiner 3 (SH)
ICON 0.996 0.975
PAR 0.997 0.992
ABO-OGS 0.99 0.979
BHS, Blair H. Struble; CJ, Cameron Jolley; SH, Sara Haley.
