Curve Ball: Why Hitting a Standardized Test Home Run is Becoming Harder
Background:
In year 2000, fewer than half of the top 20 students at the high school of one of Edison Prep’s founders did more than cursory preparation for the SAT/ACT. Many just took the PSAT and then did one practice test the week before the test. That scenario hasn’t been the case for a few years now. Because high-achievers now prepare more aggressively than ever, the top of the curve has stratified even more.
This means that, compared to those tests from a few years ago, each individual question at the top of the score spectrum (2000+) is worth even more points.
What’s the problem?
Approximately 20% of America is trying to fit into the top 7% (a 2000+). By definition, two-thirds of those students will fail to reach their goal.
The top end of the curve is evolving.
The first two tests of the Official SAT Blue Book — the gold standard of SAT books and the one almost all test prep companies use — are the actual SAT tests from Oct. 2006 and Jan. 2007. The grading grid in the graphic below shows the scores that students who took the test those years got for a given number of questions wrong. Let’s compare the Blue Book test results to three more recent tests from 2011, 2012, and 2013. As you can see, it’s a big difference. 10 to 40 points per section can add up to 100 points or more over the course of the test. 100 points is the difference between Emory’s average SAT score and the averages of some Ivy League schools, and it can make the difference in winning certain merit scholarships.
How much “meaner” was the January 2013 curve than the Blue Book Test #1 that most students use as a starting point?
Critical Reading Comparison:
Analysis:
Across the board, the January 2013 Critical Reading test was 10 to 50 points more stringent than the SAT Blue Book Test #1 Critical Reading section. (See some of the cells highlighted in yellow.)
These tests weren’t cherry-picked. For example, while a minus 1 was 770 on January 2013 Math, it was even worse for January 2012 Math — a 760.
By comparison, 2 of the first 3 actual math tests in the Blue Book allow you to miss one and still get an 800…something that has not actually happened since George W. left office!
How the curve is changing for the top 5% of students:
In the 50+ tests since the SAT was revamped in 2005, we don’t know of one until the January 2013 test where you couldn’t miss one on Reading and still get an 800 (the list of publicly-released curves comports with our data). In fact, the vast majority of the time, students could miss 2 and still get a perfect 800 on Reading. On the January 2013 test, however, minus 2 was already 770 and a minus 3 was already a 730 (loss of 4 raw points because of the wrong answer penalty).
When top students practice harder, what happens?
The SAT does a process called “equating” to make sure that the specific month a student takes it (March 2013 vs. May 2013) doesn’t affect his or her score. However, that doesn’t mean that, over time, an ever-larger, dedicated band of Type A students that does an obscene amount of practice can’t cause “lumps” at the top of the scoring scale. For example, if one month 2% of students aced all of the math questions, could they really make a minus zero something other than an 800? Some parents will see “680 Math?” and think “My gosh! What happened?” In January 2013, the answer would be “s/he missed a mere 5 questions out of the 54, or lost an average of 24 points per math question.”
“Omitting is Quitting”
Many tutors and SAT books debate whether omitting questions is acceptable if you would like a top 1% score on the SAT. Our answer is relatively simple: no.
Case Study:
What would happen if a student omitted just 5 questions on each type of question (Reading, Math, Writing) and aced the other 155 that he or she answered? On the January 2013 test, it would have been a 720 Reading / 700 Math / 700 Writing, or 2120 total (assuming 10 on the essay). A great score to be sure — top 3% — but that’s 280 points of damage just for leaving those few blank! And since it’s very hard to go 155 for 155, what would that person have gotten if s/he missed just 1 multiple choice question per section that s/he answered? A 1970 — 670 Reading / 650 Math / 650 Writing. Or, put another way, a mere 2-3 questions above UGA’s average of 1930.
So how do I get there?
To paraphrase the old Carnegie Hall joke…“How do you get to Carnegie Mellon? Practice, Practice, Practice!” We are really good at motivating our students to do homework, yet we’ve only ever told 5 or 6 students to calm down because they were over-practicing.
Are other people really practicing that much?
You don’t have to look hard to find that there are actually Atlanta SAT firms offering 128 hour SAT classes, and ostensibly, they are selling well, or they wouldn’t be around for as long as they have. Now to be clear: you will never see Edison Prep offer a 128 hour SAT Bootcamp. That’s pure torture and candidly, a bit silly. We cover all 500 pages of our book in the 24 hour SAT Group Class without too much trouble. It’s the volume of practice tests and homework outside of our tutoring sessions that doesn’t always happen, and especially for students in the 1850+ range, that’s a grave mistake.
Implications for Tutoring:
The case study above illustrates why we stress to our students that when it comes to earning a 2000+, practice test volume in between tutoring sessions is the most important variable. The most fertile score a student can start with if s/he wants a 2200+ is a 1900. At the same time, the most dangerous score (where a student might move less than 50 points) is also a 1900, if homework does not happen. Once you are at a 1900+ (a top 12% score), you’re only missing the hard ones. “Winging it” will not suffice.
So if your student is a high achiever, know that they are in the danger zone if practice tests and homework don’t happen. Be vigilant and check in regarding homework progress weekly.
UPDATE – 05/01/13 — We may be in luck…a 3rd edition of the SAT Blue Book is due in August 2013, hopefully with fresh material! Supposedly 6 fresh, new tests that will hopefully more accurately reflect the grading curves of recent tests! Link is here.
If you have questions, let us know — call us at 404-333-8573 or edison@edisonprep.com!
