The Sour Grapes of Pisa

November 28, 2013
Still standing.

Still standing.


The new Pisa 2012 will be released on Tuesday, which for those who are unfamiliar with it is a recurrent survey on the performance of schoolchildren from all over the world. The winners in this survey tend to be the same over the years: various Chinese populations (Shanghai, Hong Kong and Singapore), Finland, Canada, Australia, Japan and South Korea.

A high rank is generally interpreted as the result of a good policy and a low rank will usually create headlines demanding reforms.

The Pisa Hall of Shame

At the bottom of the order we find poor and often Muslim countries like Kyrgyzstan, Azerbaijan, Peru, Panama, Qatar and Albania. But besides rich countries at the top and poor at the bottom, there is also the phenomenon of over- and underachievers, poor countries whose children perform well and vice versa. This measure is more interesting since it indicates a failed education policy or other factors that may have been overlooked.

Some of the worst underachievers (excluding tax havens and small oil countries) are USA, UK, Austria, Germany, Denmark and Sweden. How do people in these countries respond to the results of the survey?

America: Self-Criticism and Fear of China

When commenting on the results American media have mainly been comparing themselves with China and been surprisingly self-critical, as for instance from Stacie Nevadomski in the Huff Post shows,

“The truth, the real news, is that there is no news here. These results should be no surprise. The long slide in American student performance relative to global peers has been a constant drumbeat, paralleling the domestic failures of our schools shown in Waiting for ‘Superman’.”

Or education secretary Arne Duncan,

“The findings, I have to admit, show that the United States needs to urgently accelerate student learning to try to remain competitive in the knowledge economy of the 21st century.”

James Fallows in The Atlantic agrees but adds that that Shanghai, the winner of the Pisa 2009, isn’t representative of the whole of China – which is correct; neither is Hong Kong or Singapore who also rank at the very top. These are all elite populations. America scored better against the other Chinese regions of Macao and Taiwan and it would probably do even better compared to all of China. Although those who are familiar with unpublished results from other parts of China claim they are very respectable.

Regardless of how well America compares to China, it’s still a fact that 13 countries score better than America and all have significantly lower GDP per capita. Maybe that would be a more constructive focus.

European Skepticism

A more disturbing reaction has come from some of the European underachievers. Recently, the largest newspaper in Sweden, Dagens Nyheter (Today’s News), has featured an article about the upcoming Pisa 2012, with the headline “Several Countries Cheated with School Results”, suggesting that countries like Italy, Slovenia and the United Arab Emirates has falsified their results. The article is based on an unpublished study by German and Canadian sociology professors Jörg Blasius and Victor Thiessen. “The result means that the credibility of the Pisa survey can be questioned,” says Blasius.

This story is also getting attention in Denmark, another underachiever, where one of the major papers, Berlingske Tidende has an article about it. The article includes other criticism as well, mainly that of Svend Kreiner, a statistics professor at the University of Copenhagen. Kreiner has analysed earlier results. He is critical of how a lot of questions are omitted for some countries but included for others. He claims the methods of scoring are so arbitrary Denmark could be ranked second or 42th depending on arbitrary tweaks in the evaluation. In the article, president of the Danish Teachers Association, Anders Bondo Christensen, says it’s time to scrap the survey altogether.

In the UK (also an underachiever), there is a similar discussion on the TES educational community. In an article, TES’s William Stewart writes,

“Politicians worldwide, such as England’s education secretary Michael Gove, have based their case for sweeping, controversial reforms on the fact that their countries’ Pisa rankings have “plummeted”. Meanwhile, top-ranked success stories such as Finland have become international bywords for educational excellence, with other ambitious countries queuing up to see how they have managed it.”


“But what if there are “serious problems” with the Pisa data? What if the statistical techniques used to compile it are “utterly wrong” and based on a ‘profound conceptual error’? Suppose the whole idea of being able to accurately rank such diverse education systems is ‘meaningless’, ‘madness’?”

Sour Grapes?

However, fact is the alleged cheating is only concerned with follow-up questions to principals that have been found to be largely identical in many cases. It doesn’t concern the performance of the schoolchildren. It hasn’t even been established if it is actual fraud designed to make the countries in question look better or if it’s just a matter of laziness or even the fact that some principals are heads of more than one school.

Also for Kreiner’s analysis, Pisa’s own statistician, Andreas Schleicher, questions it on grounds that Kreiner is using a very small part of the data in spite of having access to all of it. He also questions the methods Kreiner used and suggests that they our outdated. As a response to alleged cherry picking, Kreiner replies by accusing Pisa/Schleicher of doing similar things. To me, that sort of rhetoric doesn’t exactly increase his credibility.

It’s not easy for a non-expert to make any sense of this, but I have to say that there is something disconcerting with the fact that Svend Kreiner is being awarded a prize for his critique while no one in Danish press is asking the questions that Schleicher’s comment raises. Is everyone in Denmark so familiar with statistics that it’s a non-issue? And big headlines about cheating even though it hasn’t been established?

Alternative Explanations

Rather than blaming the statistics, there could be other things behind why some countries underachieve. The most obvious thing would be changes in national IQs.

The Pisa survey (and similar tests) correlates strongly to intelligence tests; so much in fact that it actually is an intelligence test although it’s rarely referred to as such. This explains a lot of the rank order, because we know that intelligence is highly heritable and resistant to external forces – like education policies. Smart people like the Chinese are going to rank at the top and less smart people like Ugandans are going to be somewhere at the bottom. This is also a reason to be skeptical of the European sour grapes skepticism I mentioned earlier. If there was something seriously wrong with the Pisa it wouldn’t correlate so much with similar tests.

But intelligence alone can’t explain under- and overachievers. If we look at the latest national IQ estimates, the underachievers score like this,

Austria 99, UK 99.1, USA 97.5, Germany 98.8, Denmark 97.2, Sweden 98.6,

and, the three overachievers score like this,

Finland 100.9, Estonia 99.7 and Poland 96.1.

There is not much difference; the averages for these groups are 98.4 and 98.9. But maybe this snapshot disguises a trend in which underachievers are on the way down and vice versa?


I would suggest that this is the case, and that the reason for this is immigration. East Asian countries don’t have much immigration to speak of, but in Europe there has been a varying influx of people in recent years, especially from Muslim countries. The national IQs in these countries are usually around 85 so Western countries that receive a lot of these immigrants should see a larger decline in national IQ averages than other countries. If we look at PEW’s survey of Muslims in Europe, we can make a comparison between over- and underachievers. The most striking overachievers are Estonia, Poland and Finland, countries that all have extremely small Muslim minorities making up 0.1, 0.1 and 0.8 percent of the population respectively. Compare that with the figures for the underachievers Austria 5.7, UK 4.6, Denmark 4.1, Germany 5.0 and Sweden 4.9. Many immigrants are very young children who will take the Pisa survey in years to come or are taking it now but have yet to become adults and have an effect on the economy. Since the Pisa survey is just an intelligence test for children they simply reflect the influx of young and low IQ people. Underachievers have a larger influx so they score worse than you’d expect from the current national IQs and wealth because the effects on these metrics will kick in some years in the future. And overachievers are just maintaining their national IQs and consequently rising in rank since the rank order is relative.

So the way to improve the scores is not to reform the education system but to change the immigration policy.

So, Any Bets for Tuesday?

If I was to guess I would base it solely on national IQs, immigration and introversion scores, although that last one is a bit speculative. This would lead me to the safe bet that East Asians will stay at the top and no real low IQ countries will surprise anyone with a high rank. Judging by the immigration projections from PEW, Eastern Europe looks like it could be on the rise, or at least maintaining positions, although Russia and Bulgaria look problematic. The real winners here are probably small to medium sized countries that are relatively stable, like Estonia, Poland, Slovenia, Croatia and Hungary. Western Europe will show a downward trend, especially for countries that are increasing their share of the Muslim population from an already high level, like the UK, Austria, Sweden, Belgium and France.

But whatever happens, you can be certain that many people in the underachieving countries will keep blaming the test. Because changing your view on human nature and society is hard work and shooting the messenger is easy.

For more details about the Pisa survey, check out Steve Sailers blog which features several interesting posts on this subject.


The IQ Breaking Point – How Civilized Society is Maintained or Lost

September 24, 2013
West Virgina at IQ 96. Life just below the breaking point.

West Virgina at IQ 96. Life just below the breaking point.

It’s a well-known fact that intelligence corresponds to various kinds of life outcomes at the individual level, such as income, education, drug abuse, criminality etc. A little less known is the fact that national average IQs correspond to similar outcomes on the national level. This has been shown by among others psychologists Richard Lynn and Tatu Vanhanen, who have found staggering correlations between national IQ and things like health, education, income, crime, corruption, democracy.  To give you a rough idea, here are some of the findings,

Educational measures like TIMSS and PISA correlate 0.79-0.92

Number of scientists and engineers correlate 0.61

Measures of  per capita national income (GDP, GNP, GNI) correlate 0.51-0.89

Poverty and unemployment: -0.63 and -0.76 respectively

Crime in the form of homicide, assault and rape: -0.21 to -0.82 with distinctly higher correlations for the more accurate measure of homicide

Corruption: -0.27 to -0.68, all but one study above -0.54

Democracy: 20 of 22 measures show correlations around 0.60

Life expectancy: 0.51-0.82, expect for one study that found a negative correlation (possibly a typo)

Anyway, you get the picture. All the basic requirements for modern civilization – democracy, education, wealth, health and (lack of) crime and corruption – are strongly related to national IQs.

IQ 97: The Breaking Point

Seeing these correlations, it’s easy to assume that national IQ averages would correspond to specific degrees of development – that South Korea with an IQ of 107 would be much more civilized than Australia at 98, which in turn would be fairly similar to Russia or some other country at 97. But looking at the stats this is far from the case. Instead it seems like there is a point, somewhere around 97, above which a modern civilization can be maintained and below which things abruptly begin to fall apart. To illustrate this I’ll review some of the correlations mentioned above.

GDP (PPP) Per Capita

You can’t have civilization without money. It pays for education, healthcare, police etc. If we use the national IQs from, who appear to have the updated version of Lynn and Vanhanen’s data set (correct me if I’m wrong), and compare them with the GDP/PPP that takes local purchase power into account, we find clear support for the idea of a breaking point at 97 or thereabout. If we exclude oil nations, the top 20 with populations over one million, according to Wikipedia/IMF are,

Singapore, Hong Kong, USA, Switzerland, Canada, Australia, Austria, Netherlands, Ireland, Sweden, Germany, Taiwan, Belgium, Denmark, UK, Finland, Japan, France, Israel and South Korea.

In these top 20, there are nine nations in the 98-99 range, but only two countries below 98 – Israel and Ireland. Both can be partly explained by the fact that they receive plenty of financial aid from USA and the EU respectively. And it’s unlikely that Ireland will stay in the top 20 given its huge public debt and very high unemployment.

And if we look at more “normal” countries with IQs below 98, those without oil or rich friends, we find Slovenia with an IQ of 96 at 24th place, then Cyprus with an IQ of 91 at 26th, and Greece with an IQ of 92 at 27th. And Greece may well have lost this spot as I write this. So at 98 there are plenty of wealthy countries, but at 97 it suddenly seems to evaporate. And this isn’t just about money; if we turn to corruption we find a very similar picture.


According to the Corruptions Perceptions Index (CPI), the top 20 countries (again with a population of over one million) most free of corruption are,

Denmark, Finland, New Zealand, Sweden, Singapore, Switzerland, Australia, Norway, Canada, Netherlands, Germany, Hong Kong, Belgium, Japan, UK, US, Chile, Uruguay, France, Austria and Ireland.

Strictly speaking this is 21 nations since Austria and Ireland share the 20th spot and it seemed a bit random to exclude the one or the other. Anyway, we find 10 countries in the 98-99 range – including the top 4 and 6 in the top 10. The only three countries below 98 are Ireland (see above), Chile with an IQ of 90 and Uruguay with an IQ of 96. Again we see how everything is fine at 98, but at 97 and below things go south.


Turning to crime, so to speak, we find the top 20 countries with the lowest homicide rates (and more than one million inhabitants) according to Wikipedia/UNODC are,

Singapore, Hong Kong, Japan, Bahrain, Norway, Austria, Oman, Slovenia, Switzerland, United Arab Emirates, Spain, Germany, Qatar, Denmark, Italy, New Zealand, China, Saudi Arabia, Sweden, Australia.

This clearly breaks the pattern since we have five nations in the 98-99 range and just as many low-IQ nations, all from the Arab world. However, if we are looking for a point at which modern civilization begins to crumble, then we shouldn’t look at dictatorships at all. They can keep the murder rate low with social control and oppression, but at in doing so they disqualify themselves as candidates for civilized societies. So if we limit our sample to at least somewhat civilized countries we get a different picture. This is of course a rather subjective method but I think we can all agree that the countries above can’t be a part of it. I have included Hong Kong but excluded China so that’s where I draw the admittedly somewhat arbitrary line. The top 20 at least moderately civilized countries with the lowest homicide rates then become,

Singapore, Hong Kong, Japan, Norway, Austria, Slovenia, Switzerland, Spain, Germany, Denmark, Italy, New Zealand, Sweden, Australia, Poland, France, Netherlands, Ireland, UK, Portugal and Serbia (IQ 89).

Again, this became 21 countries because the two last share the 20th place. We find nine nations in the range 98-99 and four countries below that. The anomaly here is Slovenia at 6th place, a country that was also a mild anomaly with 24th on the GDP per capita list. But overall, this confirms the previous findings in that there are plenty nations in the 98-99 range but at 97 or less they are few and nowhere near the top of the list.

Indeed looking at where the sub-98 countries are on these lists we find that they made places 9 and 18 on the GDP list; 17, 18 and 20 on the corruption list and 6, 18, 20, 20 on the homicide list. So they are not only few but mainly at the bottom.


Finally, let’s have a look at the Economist Democracy Index for 2012. The top 20 nations with more than a million people are as follows,

Norway, Sweden, Denmark, New Zealand, Australia, Switzerland, Canada, Finland, Netherlands, Austria, Ireland, Germany, UK, Czech Republic, Uruguay, Mauritius, South Korea, USA, Costa Rica and Japan.

As you might expect by now, around half of these, nine, are in the 98-99 range whereas only four are below that. Again, Ireland is breaking the pattern, as are Uruguay, Mauritius and Costa Rica with IQs of 90 and 89 respectively. We also find that four of the top five have 98 IQs whereas those below this level are at 11, 15, 16 and 19.

An Index of Civilization

So what happens if we add these measures up? I did this by scoring point in reversed rank order starting from 100 for each measure to create an civilization index. A statistician can probably come up with something better, but it will give a rough idea of the overall picture. The chart below shows the result,


As expected, there is a general pattern of higher degrees of civilization as intelligence increases. But the most civilized countries are all in a cluster with IQs between 98 and 101. Below 98 the level drops dramatically as can be seen by the lack of dots in the upper left part of the chart. The only distinct anomaly left is Ireland which scores 360 points for the 12th spot and, less striking, Slovenia at 20th with 335. Or, given what I’ve mentioned about the Irish economy, it may be Slovenia that is the more genuine anomaly here – clearly not as horrible as it’s portrayed in the Hostel movies. At any rate, combining these factors eliminates anomalies and further strengthens the support for an idea of a breaking point at approximately 97.


I haven’t found any official statistics for state-level IQs in America. There are however some estimates made by the Audacious Epigone, based on the NAEP scores, and educational measures are strongly correlated with IQs. He has also set the American average at 98 which matches the figures I’ve used above. If we exclude District of Columbia, which is common in these cases since it’s full of government money and temporary inhabitants, the top 20 GDP per capita (I haven’t found any PPP numbers), are as follows,

Delaware, Alaska, Connecticut, Wyoming, Massachusetts, New York, New Jersey, Virginia, Washington, Colorado, California, Maryland, Minnesota, Illinois, South Dakota, Nebraska, Hawaii, Iowa, North Dakota and Louisiana.

As in all the previous calculations, we find that states in the 98-99 make up roughly half of the sample, once more dominating the top positions – 1, 2, 4, 6, 8. Below this level we find only three states, California with an IQ of 95 at 11th position and Hawaii with an IQ of 96 at 17th and Louisiana with an IQ of 96 at 20th place. There may be some explanations for these anomalies although I’m not that familiar with American politics. But it seems clear that California is in at least as much trouble as Ireland.

Conclusions and Implications

I’m not going to say that every conceivable measure on every conceivable level will show this pattern, but overall I think we have to conclude the existence of an IQ breaking point somewhere close to 97, at which we see drastically different outcomes depending on whether a country is above or below this level. A country can of course still fail above this point, like Italy and most likely North Korea. But a national IQ above 97 represent a necessary, but not sufficient, condition for success.

One implication, if this turns out to be true, is that immigration could pose a serious threat to the West, especially those countries and regions that are closest to the breaking point. The sad part is that since everything is going so well at 98 they may dismiss this risk. This is especially true for those countries and states who are right at 98 and whose immigrants have the lowest IQs, for instance France or Texas. As for California that state has now clearly passed the breaking point, it will be interesting to see if they will break the pattern. I suspect they won’t, given that the few countries that do fairly well below 98 (Ireland, Slovenia, Uruguay etc) are all small and lacking in diversity, which is the opposite of California.

It’s fully possible that someone else already has noted this breaking point, but since I hadn’t heard of it before and it didn’t seem like common knowledge, I figured it was worth sharing. I have no idea why 97 would be a magic number; maybe it’s just something in the human condition. I’ll be updating this article whenever I find any relevant information on this.

%d bloggers like this: