Results of 2018 USMLE Step 1 & COMLEX Level 1 Correlation Survey - r/step1 (2024)

Whoaaaa, what a year!!! I have some good news and some bad news. The bad news is that if you are going to be using the information in this post to help you prepare for the USMLE/COMLEX, it means you still have to take Step 1. Yikes!! Definitely happy to be done with that. The good news is that you all are going to crush the USMLE/COMLEX.

Data Acquisition : Google Forms. Subjects were required to login to their google account to complete the form (halfway through the data collection) to limit the number of invalid responses. With that said, I had/have absolutely NO access to personal identifiers stored by Google Forms (i.e. they are stored internally somewhere in their system).

Exclusion Criteria : Data points that were impossible for the given exam/item (e.g. subjects reporting a 340 for example on the USMLE were removed).

Time-to-Exam Consolidation : Time-to-exam ranges were consolidated for some of the practice exams to attempt to allow for an n > 40 subjects.

Statistics : Statistics were primarily performed through Microsoft Excel (quick and easy to navigate). Though, the Multilinear Linear Regression Line as well as the linear NBME & UWSAs were computed through SPSS (more accurate).

Raw Data without Institutional Information : Link

Overview : Most of the trendiness fit a polynomial function best.

General : All the general information reported in the "charting outcomes link" corresponds to the raw data from google forms (i.e. before some of the data was excluded). That's because it was too much work to consolidate word responses in excel.

USMLE : The average USMLE Step 1 score received (n = 604) was 245.48 +/- 1.14 (CI = 95%) with a standard deviation of 14.21. The minimum score reported was a 194 and the maximum score reported was a 275.

COMLEX : The average COMLEX Level 1 score received (n = 107) was 619.10 +/- 12.79 (CI = 95%) with a standard deviation of 66.73. The minimum score reported was a 443 and the maximum score reported was a 765.

QBanks : Of the QBanks being evaluated in this study, the UWorld QBank appears to be most greatly correlated with USMLE performance (predicted USMLE performance = 1.1277x + 164.39 with an R² of 0.5612). The number of passes does not appear to correlate with USMLE performance. However; there are various confounding variable that may account for the lack of correlation including potentially preferred personal study modalities, the importance of quality over quantity, time at which first pass was completed, etcetera.

NBME : Regardless of when the practice test was taken (i.e. evaluating the overall/summary scores per NBME), NBME 16 most strongly correlated with USMLE performance with an R² of 0.6057 (Predicted USMLE = 2E-05x3 - 0.0112x2 + 3.1163x - 73.678). However; using one’s average NBME score is a better predictor (not accounting for number of different NBME practice tests taken) with a Predicted USMLE per Excel = 0.6361x + 98.626 with an R² of 0.6842. However; stats were also run in SPSS which is more accurate, and a predicted USMLE was calculated to be = 0.623946x +101.556805 (adjusted R² = 0.679 with a Standard Error of 7.693506).

UWSA : Regardless of when the practice test was taken (i.e. evaluating the overall/summary scores), UWSA2 was more greatly correlated with USMLE performance when compared to UWSA1. The predicted USMLE per UWSA2 = 7E-08x5 - 9E-05x4 + 0.0406x3 - 9.4848x2 + 1098.3x - 50151 with an R² = 0.6504. Averaging the UWSA exams was less effective with an R² of 0.626 (2nd order polynomial). When a linear regression was evaluated for the UWSA1/2 Avg (per SPSS) and R² of 0.586 was obtained (Predicted USMLE = 0.644680x + 86.009656 with a standard error of 8.739950).

Average NBME & UWSA : Multinomial Linear Regression (per SPSS) with Assumption Testing was performed with Average NBME and Average UWSA used as the covariables and USMLE Score Received as the Dependent Variable. Unfortunately, it is a suboptimal analytical tool for predicting one’s USMLE score as multicollinearity was identified between both predictors with correlations >0.7. Thus, single variable analysis may provide a better degree of accuracy. Regardless, if one desires to use a multilinear regression, a predicted USMLE score may be computed by the following formula : Predicted USMLE = 0.256092(UWSA Avg) + 0.441792(NBME Avg) +80.144559; Adjusted R² = 0.714; Standard Error = +/- 7.27155

COMSAE : There was very little data corresponding to COMSAE and COMLEX performance. Only 22 reported taking COMSAE A, 12 taking B, and 15 taking C. COMSAE E was the only practice test with an adequate number of reports (n = 64).

Sleep : Sleep Duration as well as alterations in sleep patterns did not appear to be associated with performance on the USMLE.

Nutritional Overview : Nutritional products were difficult to code for as there was much variety in answers and only so much time in the day. What appeared to be the most popular answers were Daily MVI (37), coffee (n=26), caffeine (+/- theanine) not otherwise specified (26), individual vitamins, fish oil (n=6). Pre-workout (unknown if blends include stimulants), protein powder/bars, and creatine were used by a variety of subjects. It would be interesting to see how regular exercise regimen may effect ones performance. Some individuals endorsed the use of POTENTIALLY elicit substance use (based on whether he/she has/had a prescription and/or where he/she lives) : adderall (n=1), concerta (n=1) and marajuana (n=1).

​

Charting Outcome Links :

This study has significant limitations!!! The average USMLE score in 2017 per “The Match” website was a 229, however the average USMLE reported in this survey is a 245.48. This is most likely multifactorial in nature, with differences possibly accounting for the naturally high intellect of redditors ;-)… thoughhhh also significant under-reporting of sub-optimal scores is anticipated. Also, note that this study is a self-report with no true means to verify and validate the data provided. Thus, evaluate the data in this report with much scrutiny. Also, make the following considerations :

• Only evaluate your progress based on data points within the distribution ranges. Do NOT over-extrapolate what is presented. Also note that some of the data series have a smaller “n,” even though I attempted to adjust for this by combining time periods as needed.
• Do NOT be discouraged by your “predicted scores.” There is VERY LIMITED data for those who scored lower, thus these equations are primarily for those who are doing well overall. Also note that confidence, motivation, discipline, etcetera are all VERY important to succeed in any exam!! Furthermore, the data here principally focuses on correlation. If further analysis was conducted through SPSS for all the data, it would undoubtedly reveal large standard errors for each “predicted trendline.”

Work your hardest during dedicated and crush the USMLE regardless of whether you are scoring well or poorly on the NBME/UWSA self-assessments.

In the past years,u/aervien,u/Waygzh,u/OslerSenpai(and anyone I missed- I apologize) have done absolutely AMAZING work collecting & analyzing data with regards to the USMLE & practice exams. I genuinely thank them for their hard work and contributions. Special thanks also to all those who contributed to the data pool this year- it would have literally been impossible without you.

​

-----------------------------------------------------

**** I have been notified that the polynomial functions did not Graph Correctly. I have attempted to transfer my excel doc to SPSS, but I have had trouble adding "missing values" to the data sheet and re-run the stats. i attempted for several hours and eventually decided to call it quits. With that said, if further stats are desired or you would like to personally run it in SPSS, let me know and I can send you the SPSS doc. But with that said, I converted all equations and graphs to linear regression lines in excel and re-uploaded them.

**** I have been notified that the title columns and data bars (for the resources used) don't always neatly line up. I just looked into it and it appears that neither zooming in nor copying and pasting into google slides fixes the problem. It would be very timely to code for all the resources or I would. I believe last year he/she reported on that data.