Educational Rankings and Economic Success: How Clear is the Connection?

March 8, 2013

Following the release of the PISA 2009 scores, United States Secretary of Education Arne Duncan said:

The findings, I have to admit, show that the United States needs to urgently accelerate student learning to try to remain competitive in the knowledge economy of the 21st century. Americans need to wake up to this educational reality, instead of napping at the wheel while emerging competitors prepare their students for economic leadership.

Similarly, in this short video advocating for the Common Core Standards, the Hunt Institute makes reference to the U.S.’s PISA rankings and says:

Out students need better knowledge and tools to prepare them to compete in the global economy.

This is not a new argument, nor is it one restricted to the U.S. context. Advocates and detractors of education reform continually make reference to international competition, often using PISA and TIMSS rankings to bolster their argument. They draw an explicit link between these rankings and economic performance.

So, let’s take this argument at face value and test it on it’s own merit.

Using IMF data, here are the ten largest economies as measured by GDP PPP:

1. United States
2. China
3. India
4. Japan
5. Germany
6. Russia
7. Brazil
8. United Kingdom
9. France
10. Italy

Based on the arguments commonly heard from politicians, interest groups, and in the popular press, we may expect to see a significant overlap between this list and the 2009 PISA rankings. Here are the top ten performing nations and cities from that exam:

1. Shanghai, China
2. South Korea
3. Finland
4. Hong Kong
5. Singapore
6. Canada
7. New Zealand
8. Japan
9. Australia
10. The Netherlands

One nation appears on both lists: Japan.

Of course, one could argue that education is a lagging variable. That those educated today will not have meaningful participation in the global economy until years later. To test this, let’s look at the 1995 TIMSS 8th grade Math rankings:

1. Singapore
2. South Korea
3. Japan
4. Hong Kong
5. Flanders (Belgium used disaggregated data)
6. Czech Republic
7. Slovakia
8. Switzerland
9. The Netherlands
10. Slovenia

Again, only Japan appears on this list and the list of largest economies as determined by GDP PPP. Here one could argue that GDP PPP is a simplistic measure that fails to take population into account. As our final exercise, let’s look at GDP PPP per capita:

1. Luxembourg
2. Norway
3. Qatar
4. Switzerland
5. Macao SAR, China
6. Kuwait
7. Australia
8. Denmark
9. Sweden
10. Canada

This is even more problematic, not one nation (or region in the case of Macao) that ranked in the top ten in 1995 appears on the per capita list.

It should go without saying there are a number of problems with the simplistic analysis used in public policy debates.

For starters, although we often say “Nation so-and-so is ranked Xth in the world,” the reality is that the majority of the nations in the world don’t participate in TIMSS or PISA. Luxembourg, Qatar, and China did not take part in the 1995 TIMSS. We can hardly expect nations that didn’t take the test to appear on our top-performers list. In fact, only 43 nations in the world participated in that exam in 1995. That number has increased over the past eighteen years; sixty-four nations, cities and regions took the most recent PISA exam in 2012. However, we are still working with an incomplete dataset.

In addition, we’re not even using simple linear regression analysis in this blog post. We’re just comparing top-ten lists and looking for commonalities. Scholars studying comparative education know this is an invalid way on measuring the effect of education on economic growth. The link between a nation’s education system and its economy is a complex relationship between two adaptive, emergent systems. A relationship that can’t be measured by comparing GDP and PISA scores. We know this.

However, outside of the universities, think tanks, and multilateral development banks it is a different story. Politicians are not making complex arguments that take many variables into account. They are saying better rankings equals better a economy. President Obama:

It is an undeniable fact that countries who out-educate us today will out-compete us tomorrow.

The weaknesses in our argument above should serve to reveal the weakness in the president’s argument. Politicians define educational success with league tables and make ill-defined linkages to the nation’s economic rankings every day. Just by looking at the rankings it should be clear that this argument fails to hold up by its own logic.

—

UPDATE 03-09-13 0554AM – A reader asked why I said “nations” before the PISA rankings with Shanghai taking the top spot. I misspoke, that should have read “nations and cities” and has been fixed. This does point to another problem with the way the league tables are discussed in the popular discourse, as mixing up nation and city results is a common mistake. When the 2009 results were announced, many commentators reported that “China” had top the world rankings. Here’s Time Magazine:

China Beats Out Finland for Top Marks in Education