Dec 28 2013, 6:58pm CST | by Forbes
Washington University in St. Louis has the “smartest” students in America, according to the second annual ranking of college students released by San Francisco-based Lumosity, the self-purported “leading brain training and neuroscience research company.”
In Lumosity’s analysis of 72,388 undergrads, ages 17-24, who played its signature –- and much-advertised –- “cognitive training games,” MIT (ranked 1st last year) and Princeton (a shocking 39th last year) came in at 2nd and 3rd respectively. They were followed by my undergraduate Alma mater of Northwestern (ranked 4th for the second year running).
Perennial ranking darlings Harvard (which dropped from 2nd to 8th) and Stanford (which dropped from 3rd to 9th) fell, while “Core Curriculum” powerhouse, the University of Chicago, rose from 16th to 6th. The Rose-Hulman Institute of Technology (9th last year) plummeted to 80th. Meanwhile, Ivy League stalwarts Cornell, Columbia and Brown came in at 23rd, 26th and 35th respectively.
Holding up the rear in Lumosity’s ranking of 456 colleges were Farleigh Dickinson University (453), The University of Texas-Pan American (454), the University of North Carolina at Pembroke (455) and the not-so-game Jacksonville State University Gamecocks (dead last at 456).
Privately held Lumosity claims that its results closely mirror a school’s median composite ACT and SAT scores. However, the fairly wide discrepancy between Lumosity’s rankings and conventional rankings from the likes of U.S. News & World Report begs a question: does Lumosity’s game-centered rankings approach provide an accurate measure of student intelligence? Moreover, are its results a measure of certain students’ ability to play games or take tests rather than a measure of what they empirically know?
I am not alone in raising these concerns. After my story on Lumosity’s 2012 rankings ran, I received a flurry of comments on the company’s ranking methodology. I brought these concerns to Lumosity after they released this year’s rankings (see the full 2013 list below).
Below are a summary of Lumosity’s responses to each concern in turn.
Selecting Based on Email Addresses May Include Graduate Students and Staff.
As I noted in my original post, Lumosity uses “self-reported email addresses and/or the web domain associated with” a student’s “IP address” in ascertaining whether a test-taker is a bona fide collegiate undergrad. As Crotty on Education reader Richard Campo commented, “They did not necessarily test students, let alone undergraduate students. They tested anyone at the university with an email address. That could be anyone from the maid to the cafeteria lady. All this study shows is that at MIT and Stanford the students have a separate email address from the rest of the university employees.”
In response, Lumosity spokesperson Erica Perng informed me that “this year’s analysis had additional inclusion criteria to increase the probability that the game scores were generated by a current student at the university. (1) We only included users who list their education level as having completed ‘high school’ (possibly freshmen), ‘some college’ (current students), or ‘bachelor’s degree’ (students who are about to or just recently graduated). (2) We included only users whose first five game plays occurred in the time period between July 2012 to October 2013.”
Nevertheless, Lumosity scientist Daniel Sternberg, Ph.D., admits that Lumosity’s “approach to identifying users as university students” is still “imperfect” in part because “a particular university domain is not necessarily an indicator of a lasting affiliation with a university.” Moreover, I still can’t tell how Lumosity clearly distinguishes between a bona fide undergrad and a preternaturally smart young janitor like Will Hunting.
The World’s Top University, Caltech, is Not Even on the List.
In the Times/Thomson Reuters World University Rankings for 2012-2013, the California Institute of Technology (Caltech) in Pasadena was named the top university in the world. Unfortunately, Walter “Heisenberg” White’s Alma mater did not make the Lumosity list again this year because it did not meet the cut-off of a 50-person sample of students with complete score and demographic data from the past year. Notes Perng, Lumosity “chose this cut-off in order to improve the reliability of our findings while still providing good coverage across a large number of schools. As a result, Caltech and several excellent small liberal arts schools (e.g., Amherst) were not able to be included. If we look at the data without taking into account the 50-person cut-off, Caltech is certainly at the top.”
Success at Lumosity Games is a Factor of Repetition and Gamesmanship, Rather than Innate Cognitive Ability.
As with its 2012 analysis, in 2013 Lumosity chose to analyze a user’s first game play in each cognitive area, so that, notes Perng, Lumosity would “not measure practice effects.” In addition, notes Perng, Lumosity does not base its results off a user’s Brain Performance Index because Lumosity finds that “the more people train, the better their Lumosity scores.” In other words, users cannot game the results in their school’s favor./>/>
Lumosity Users are Self-Selecting.
The Lumosity ranking data are based on college students from across the U.S. who’ve heard about Lumosity, came to the Lumosity website, and tried at least a few games. Therefore, the Lumosity data does not represent a random sample from each college. Rather, it represents a sample of the kind of people at a college who were interested in Lumosity and the kind of “self-improvement” that Lumosity proffers. As Perng notes, it’s “possible that the results could be biased by this, if the types of people who were interested in Lumosity and came to our site were systematically different from college to college.”
For me, this is a de-limiting factor since knowledge of Lumosity is largely dependent on receiving, let alone being receptive to, its marketing. Since there is a huge pool of foreign students at places like Caltech who reside outside the prevailing American media sphere combined with strong anti-media/anti-marketing bias at rigorous, classics-driven liberal arts colleges like St. John’s College Santa Fe (where I received my Masters), it’s possible that there might be a target customer more likely to respond to a Lumosity pitch, and, thus, play a Lumosity game.
Though Lumosity claims to have rigorously controlled for gender (49.6% of the Lumosity data sample was female) and age, Lumosity players may not be a representative sample of a college’s demographics, and, in fact, might merely represent the small subset of students likely to try online learning games. Moreover, there are many kinds of “intelligence” — developmental psychologist Howard Gardner identified nine — for which Lumosity does not test. The fact that Lumosity’s methodology has not been officially peer-reviewed makes all these caveats worthy of further investigation.
Nevertheless, the Lumosity rankings continue to fascinate, since, in an education landscape increasingly obsessed with making learning more like gaming – there’s even an egregious word for this phenomenon: “gamification” — they offer a doorway into how a generation immersed in video gaming might perform in school, the workplace, and beyond.
Below, in order, are the 20 Colleges with the Smartest Students:
1. Washington University in St Louis
5. Carnegie Mellon University
6. University of Chicago
7. Rice University
11. Tufts University
12. Stanford University
13. Georgetown University
16. Duke University
17. Bucknell University
18. Vanderbilt University
19. College of William and Mary
20. Boston College
For the complete list of 456 ranked colleges, and a full explication of Lumosity’s ranking methodology, please go here.
Where does your Alma mater rank? Let me know what you think in the Comments area below.
Source: Forbes Business
Forbes is among the most trusted resources for the world's business and investment leaders, providing them the uncompromising commentary, concise analysis, relevant tools and real-time reporting they need to succeed at work, profit from investing and have fun with the rewards of winning.
blog comments powered by Disqus