Inaccurate, Impossible: Experts Knock New Trump Plan to Collect College Admissions Data

“You’ve got this effort to add these elements without a mechanism with which to vet the new variables, as well as a system for ensuring their proper implementation,” said Cook. “You would almost think that whoever implemented this didn’t know what they were doing.”

Cook has helped advise the Education Department on the IPEDS data collection for 20 years and served on technical review panels, which are normally convened first to recommend changes to the data collection. Those panels were disbanded earlier this year, and there isn’t one set up to vet Trump’s new admissions data proposal.

Cook and other data experts can’t figure out how a decimated education statistics agency could take on this task. All six NCES employees who were involved in IPEDS data collection were fired in March, and there are only three employees left out of 100 at NCES, which is run by an acting commissioner who also has several other jobs.

An Education Department official, who did not want to be named, denied that no one left inside the Education Department has IPEDS experience. The official said that staff inside the office of the chief data officer, which is separate from the statistics agency, have a “deep familiarity with IPEDS data, its collection and use.” Former Education Department employees told me that some of these employees have experience in analyzing the data, but not in collecting it.

In the past, there were as many as a dozen employees who worked closely with RTI International, a scientific research institute, which handles most of the IPEDS data collection work.

Technical review eliminated

Of particular concern is that RTI’s $10 million annual contract to conduct the data collection had been slashed approximately in half by the Department of Government Efficiency, also known as DOGE, according to two former employees, who asked to remain anonymous out of fear of retaliation. Those severe budget cuts eliminated the technical review panels that vet proposed changes to IPEDS, and ended training for colleges and universities to submit data properly, which helped with data quality. RTI did not respond to my request to confirm the cuts or answer questions about the challenges it will face in expanding its work on a reduced budget and staffing.

The Education Department did not deny that the IPEDS budget had been cut in half. “The RTI contract is focused on the most mission-critical IPEDS activities,” the Education Department official said. “The contract continues to include at least one task under which a technical review panel can be convened.”

Additional elements of the IPEDS data collection have also been reduced, including a contract to check data quality.

Last week, the scope of the new task became more apparent. On Aug. 13, the administration released more details about the new admissions data it wants, describing how the Education Department is attempting to add a whole new survey to IPEDS, called the Admissions and Consumer Transparency Supplement (ACTS), which will disaggregate all admissions data and most student outcome and financial aid data by race and gender. College will have to report on both undergraduate and graduate school admissions. The public has 60 days to comment, and the administration wants colleges to start reporting this data this fall.

Complex collection

Christine Keller, executive director of the Association for Institutional Research, a trade group of higher education officials who collect and analyze data, called the new survey “one of the most complex IPEDS collections ever attempted.”

Traditionally, it has taken years to make much smaller changes to IPEDS, and universities are given a year to start collecting the new data before they are required to submit it. (Roughly 6,000 colleges, universities and vocational schools are required to submit data to IPEDS as a condition for their students to take out federal student loans or receive federal Pell Grants. Failure to comply results in fines and the threat of losing access to federal student aid.)

Normally, the Education Department would reveal screenshots of data fields, showing what colleges would need to enter into the IPEDS computer system. But the department has not done that, and several of the data descriptions are ambiguous. For example, colleges will have to report test scores and GPA by quintile, broken down by race and ethnicity and gender. One interpretation is that a college would have to say how many Black male applicants, for example, scored above the 80th percentile on the SAT or the ACT. Another interpretation is that colleges would need to report the average SAT or ACT score of the top 20 percent of Black male applicants.

The Association for Institutional Research used to train college administrators on how to collect and submit data correctly and sort through confusing details — until DOGE eliminated that training. “The absence of comprehensive, federally funded training will only increase institutional burden and risk to data quality,” Keller said. Keller’s organization is now dipping into its own budget to offer a small amount of free IPEDS training to universities.

The Education Department is also requiring colleges to report five years of historical admissions data, broken down into numerous subcategories. Institutions have never been asked to keep data on applicants who didn’t enroll.

“It’s incredible they’re asking for five years of prior data,” said Jordan Matsudaira, an economist at American University who worked on education policy in the Biden and Obama administrations. “That will be square in the pandemic years when no one was reporting test scores.”

‘Misleading results’

Matsudaira explained that IPEDS had considered asking colleges for more academic data by race and ethnicity in the past and the Education Department ultimately rejected the proposal. One concern is that slicing and dicing the data into smaller and smaller buckets would mean that there would be too few students and the data would have to be suppressed to protect student privacy. For example, if there were two Native American men in the top 20 percent of SAT scores at one college, many people might be able to guess who they were. And a large amount of suppressed data would make the whole collection less useful.

Also, small numbers can lead to wacky results. For example, a small college could have only two Hispanic male applicants with very high SAT scores. If both were accepted, that’s a 100 percent admittance rate. If only 200 white women out of 400 with the same test scores were accepted, that would be only a 50 percent admittance rate. On the surface, that can look like both racial and gender discrimination. But it could have been a fluke. Perhaps both of those Hispanic men were athletes and musicians. The following year, the school might reject two different Hispanic male applicants with high test scores but without such impressive extracurriculars. The admissions rate for Hispanic males with high test scores would drop to zero. “You end up with misleading results,” said Matsudaira.

Reporting average test scores by race is another big worry. “It feels like a trap to me,” said Matsudaira. “That is mechanically going to give the administration the pretense of claiming that there’s lower standards of admission for Black students relative to white students when you know that’s not at all a correct inference.”

The statistical issue is that there are more Asian and white students at the very high end of the SAT score distribution, and all those perfect 1600s will pull the average up for these racial groups. (Just like a very tall person will skew the average height of a group.) Even if a college has a high test score threshold that it applies to all racial groups and no one below a 1400 is admitted, the average SAT score for Black students will still be lower than that of white students. (See graphic below.) The only way to avoid this is to purely admit by test score and take only the students with the highest scores. At some highly selective universities, there are enough applicants with a 1600 SAT to fill the entire class. But no institution fills its student body by test scores alone. That could mean overlooking applicants with the potential to be concert pianists, star soccer players or great writers.

The Average Score Trap

Admissions data is a highly charged political issue. The Biden administration originally spearheaded the collection of college admissions data by race and ethnicity. Democrats wanted to collect this data to show how the nation’s colleges and universities were becoming less diverse with the end of affirmative action. This data is slated to start this fall, following a full technical and procedural review.

Now the Trump administration is demanding what was already in the works, and adding a host of new data requirements — without following normal processes. And instead of tracking the declining diversity in higher education, Trump wants to use admissions data to threaten colleges and universities. If the new directive produces bad data that is easy to misinterpret, he may get his wish.