Education has been given the task to turn our children into globally competitive workforce. It is thus no surprise that results of international assessments such as the PISA and TIMSS are closely watched by policy makers and the media as an indication a nation’s education quality and their future competitiveness. While I have serious reservations about making “global competitiveness” as an honorable purpose of education and the reduction of complex phenomenon such as education and human capacity into simplistic numbers, I am unable to resist the seduction of numbers and rankings. In doing research for my new book World Class Learners: Educating Creative and Entrepreneurial Students, I came across a set of data that appear to show something very interesting.
The Global Entrepreneurship Monitor (GEM) is annual assessment of entrepreneurial activities, aspirations, and attitudes of individuals in over 50 countries. Initiated in 1999, about the same time when PISA began, the GEM has become the world’s largest entrepreneurship study. Thirty-nine countries that participated in the 2011 GEM also participated in the 2009 PISA and 23 out of the 54 countries are considered “innovation-driven” economies, which means more economically developed countries.
Comparing the two sets of data shows clearly that countries score high on the PISA do not have the level of entrepreneurship that match their stellar scores.
Ranking of 23 “Innovation-driven” economies by 2009 PISA math scores. Data source: http://www.pisa.oecd.org/dataoecd/54/12/46643496.pdf
Ranking of 23 “Innovation-driven” economies by 2011 Perceived Entrepreneurial Capabilities (% of people confident in their ability for entrepreneurship). Data source: http://www.gemconsortium.org/docs/download/2201
Putting the two lists together reveals that countries with higher PISA scores have fewer people who are confident in their entrepreneurial capabilities. Out of the innovation-driven economies, Singapore, Korea, Taiwan, and Japan are among the best PISA performers but their scores on the measure of perceived capabilities or confidence in their ability to start a new business are the lowest.
For the more statistically inclined, the following scatterplot using raw scores instead of ranking shows the same pattern: countries with higher PISA scores have lower confidence in entrepreneurial capabilities:
Interestingly, Tom Loveless found that there is a negative correlation between student confidence in their math abilities and their math scores on the TIMSS. The same pattern was found between enjoyment of math and math scores. Basically it seems that countries that produce better math scores have students who are less confident in their math ability. They also enjoy math less.
Source: Tom Loveless (2006): How Well Are American Students Learning
So what does this mean?
Home: Learning in the Age of AI 
27 responses to “Test Scores vs. Entrepreneurship: PISA, TIMSS, and Confidence”
[…] (typeof(addthis_share) == "undefined"){ addthis_share = [];}Professor Yong Zhao has done a fascinating analysis of the international PISA math assessments and found […]
LikeLike
[…] http://zhaolearning.com/2012/06/06/test-scores-vs-entrepreneurship-pisa-timss-and-confidence/ […]
LikeLike
One thing I have noticed is that people are afraid to tell children they are wrong or that they need to improve in an area. Children think they are great at math, because no one told them they were actually just mediocre. I don’t think we need to become task masters or drill sargents, this will cause them to dislike math. We need to find the happy middle ground. We do children a great disservice when we allow them to fail and tell them they are doing a great job. In the end they don’t know the subject, how to study and improve themselves, or the basic skills they will need later in life.
LikeLike
[…] Yong Zhao has done a fascinating analysis of the international PISA math assessments and found […]
LikeLike
[…] TIMMS/PISA vs. Entrepreneural Spirit: http://zhaolearning.com/2012/06/06/test-scores-vs-entrepreneurship-pisa-timss-and-confidence/ […]
LikeLike
[…] have massive research and many evidences available…[ie: Diane Ravitch is declaring a bunch of them here, […]
LikeLike
I am reminded of some quote that I can not faithfully reproduce here but it says that just because there is correlation among data sets, this doesn’t necessarily explain things. I don’t know what this correlation between the test scores and innovation scores means; BUT I don’t think, don’t believe, that encouraging innovation / entrepreneurship as an integrated component of effective learning (e.g., PBL with an innovation / entrepreneurship defining question) will reduce the effectiveness of the learning.
Of course, as always, there certainly can be efforts expended ostensibly on innovation / entreprenureship that indeed would detract from effective learning. No question about that. But of course, there can also be efforts expended on effective learning improvement itself that end up reducing effective learning!
Again, I truly believe that facilitating PBL with expressed links to innovation not only improve the skills and excitement of students for entrepreneurship activities explicitly but also improve the levels of effective learning!
LikeLike
I am reminded of a quotation that I am unable to faithfully reproduce; but it says as I remember it that just because there is a correlation among data sets, this doesn’t necessarily explain things. I don’t know what this correlation between the test scores and the entrepreneurship scores means; BUT I don’t believe that encouraging innovation / entrepreneurship as an integrated component of effective learning (e.g., PBLwith an innovation / entrepreneurship defining question) will reduce the effectiveness of the learning.
Of course, as always, there certainly could be efforts expended ostensibly on innovation / entrepreneurship that indeed would detract from effective learning. No question about that. But, of course, there can also be efforts expended aimed ostensibly at effective learning improvement directly that have a disastrous impact on the effective learning!
Again, bottom line, I truly believe that facilitating PBL with intentional links to innovation / entrepreneurship not only improves the skills and motivation to consider entrepreneurship opportunities as a career or within an established company career (often called intrepreneurship) BUT also improves the levels of effective learning. And yes, effective learning does improve standardized test scores such as PISA and TIMMS!
LikeLike
I believe that there was another study done in relation to these where the students were asked if they thought they did well on the tests. Those that ranked high on the test were less likely to be confident that they did well, but the U.S. ranked the highest in this study, having the lowest scores, but thinking that they did better than they actually did? Is this a good way to be or doesn’t it point back to the way the U.S. prides itself on claiming they are better then they actually are?
An Alien thought!
LikeLike
It would be interesting to develop an assessment that measures creative thinking and see how the scores on such an assessment would line up with both the PISA and the GEM. My guess is that high GEM scores correlate with high creative thinking.
LikeLike
John Bennett, I think what you meant is correlation does not equal causation. And I think Dr. Zhao has been very careful as to not make causal implications.
Zhao, I found the negative correlation between PISA scores and perceived entrepreneurial capabilities fascinating. But I guess a more important question to ask is: so what? Why do you think the correlation is there? And what can policymakers do about it? I think one of the substantive explanation is that: countries with higher PISA scores are also countries facing more severe internal competitions. Therefore, people in high-achieving countries are skeptical about their entrepreneurial capability when taking their equally competitive peers into consideration. That’s just my interpretation. I look forward to reading your insights on that. Thanks!
LikeLike
John Bennett, I think what you meant is correlation does not equal causation. And I think Dr. Zhao has been very careful as to not make causal implications.
Zhao, I found the negative correlation between PISA scores and perceived entrepreneurial capabilities fascinating. But I guess a more important question to ask is: so what? Why do you think the correlation is there? And what can policymakers do about it? I think one of the substantive explanation is that: countries with higher PISA scores are also countries facing more severe internal competitions. Therefore, people in high-achieving countries are skeptical about their entrepreneurial capability when taking their equally competitive peers into consideration. That’s just my interpretation. I look forward to reading your insights on that. Thank you.
LikeLike
Could it be that the more you know, the more you know that you don’t know (a case of academic humility). On the other hand, the less you know, the more you think that you know (hence a false confidence). Just because you are less confident in your mathematical ability does not mean that you have less mathematical ability.
LikeLike
[…] [5] http://zhaolearning.com/2012/06/06/test-scores-vs-entrepreneurship-pisa-timss-and-confidence/ […]
LikeLike
Alien Thought,
You hit it on the nail! What difference does the level of confidence about your knowledge and your actual skills level make, if in the end your performance does not match your perception of yourself or your skills level. Just thinking you are good at something when you are not is not going to help your performance.
A study was done a while back ago: a chinese student and an American student were given a test with their parents outside. When their parents got the results, each reacted differently. The American parent told the kid he had done his best and that he was okay. The Chinese parent looked at what the child had gotten wrong and explained everything to him and had him study his mistakes telling child that he was not up par. Guess who had done better next time? Asians do not spend much time boasting while we Americans spend our learning time boasting about qualities we do not even possess. That is the difference in the outcome of the study above.
American education would improve if children were expected to work hard and diligently and be respectful. In actuality, it all starts with the parents.
LikeLike
Fascinating. I came across this article after our local newspapers bemoaning Australia’s performance in the TIMSS.
My brief experience visiting Singapore gives me the feeling that the negative correlation between the TIMSS and entrepreneurship in Asian countries may be due to cultural and educational factors. In Singapore, people seem to work themselves to death. My taxi driver had previously been in the military, and said he quit after decades of running on four hours of sleep everyday. In the medical system, doctors had deep knowledge of concepts that will of little importance to actual patient care, but lacked social skills and creativity. 60 and 80 hour work weeks are the norm rather than the exception.
My feeling is that the education system may be teaching students strong discipline and brute force techniques for solving problems, by providing heavy work loads and lots of instruction time spent on maths and science. However, I feel that time spent on tasks that will teach creativity, independent thinking, leadership, social skills, and psychological health, such as sport, music, arts, group work, philosophy, and free time to be a kid, may be getting neglected in Asian countries.
I would also comment that the lack of a 0 point on the vertical axis of the PISA/Entrepreneurship graph makes the minor differences in test scores between countries appear much larger than they actually are.
LikeLike
[…] algunos artículos publicados recientemente en su blog sobre creatividad, emprendizaje y educación, el investigador Yong Zaho, desvela, en base al […]
LikeLike
Having difficulty accepting these test-score comparisons that show US students lagging because something about the samples feels inconsistent. Are we sure that the groups of students being compared are equal in ability, support, socio-economic background, etc? Dies PISA compare apples to apples?
LikeLike
[…] correlates strongly with entrepreneurship. Countries that come out on top with the PISA test also tend to have low levels of confidence and low levels of entrepreneurship. The United States devotes more money and student time to extracurriculars and less to in-class […]
LikeLike
[…] correlates strongly with entrepreneurship. Countries that come out on top with the PISA test also tend to have low levels of confidence and low levels of entrepreneurship. The United States devotes more money and student time to extracurriculars and less to in-class […]
LikeLike
Hi
I have read your book and found your results very interesting. I recently blogged about this article because one of our major parties want to implement grades from the fifth grade.
While I wrote the blogpost something struck me: The countries with low score on entrepreneurship are solid, stable economies, with the exceptions of the immigrant countries the US and Australia. So it is possible for people to find work within the economy. The other countries don´t have the same financial stability. Does this affect your results? How much does the mentality of a country affect entrepreneurship? Also, do you find a correlation between PISA and TIMMS score and entrepreneurs within a country?
LikeLike
[…] an extremely revealing June 2012 blog, Zhao dissects the relationship between students’ perceived levels of creativity and […]
LikeLike
[…] of all, it is well worth it to read a variety of analysis of the data, my favorite being Yong Zhao test scores vs entrepeneurship, who has done a nice job pointing out the negative correlation between high PISA scores and low GEM […]
LikeLike
Hmmm….Spain, Greece, Slovenia, Ireland….at the top of the list. I guess “Confidence” doesn’t equal Competence when creating an innovation economy.
LikeLike
[…] with high math / Science test scores, and countries with high levels of innovation… (Yong Zhao) http://zhaolearning.com/2012/06/06/test-scores-vs-entrepreneurship-pisa-timss-and-confidence/ Educating Creative and Entrepreneurial Students, a book by Yong Zhao… […]
LikeLike
[…] wealthiest students in with the middle class and poor. They say American students are failing when it’s just the poor kids. And even when you add them all together, we’re in the middle, and we’ve always been in the […]
LikeLike
[…] with creativity, innovation, or entrepreneurship as Yong Zhao has recently shown with PISA scores here and here. Educational enterprises are designed to compete with one another, but are utterly […]
LikeLike