The article appeared in the Views column of CNBC TV18  on 31^st July 2020.

https://www.cnbctv18.com/views/view-the-Learning-power-of-Pisa-6508081.htm

• Participating in international benchmarking can provide insights that cannot be garnered solely from looking within the Indian system.
• Imagine if India had not taken the PISA in 2009-10—we would have continued to believe, as the government believed then, that Himachal Pradesh and Tamil Nadu are at world-class levels.

An article by Mr Anil Swarup, former Secretary, MHRD, Government of India, (“Evaluation and assessment: The ‘learning tower’ of PISA“, July 23) makes the case that an international test like PISA should not be a priority for India.

The article essentially says

• We do not need more assessments or diagnosis; we need action on the sufficient diagnoses we  already have. The problems related to school education are pretty well-known—in many cases, the solutions are also known too
• PISA is foreign and expensive
• Testing Chandigarh, the Kendriya and Navodaya Vidyalayas does not make sense as they are not  representative of India
• PISA’s findings may not be valid for India considering its sample size and India’s diversity.
• The National Achievement Survey conducted by the NCERT has been good since 2017 and should be built on

Many of these represent commonly-held perceptions that seem correct on the surface but do not hold up to deer scrutiny.

Well-developed, large-scale assessments can provide clear indicators on where a system needs to focus to improve. Such assessments need to be undertaken regularly just like an annual health check-up should not be skipped either because we think we are healthy or because we know of a problem which we feel we have not addressed. PISA, specifically, has less diagnostic value because it tests students at the end of the schooling years (class 9 / 10).

The power of PISA comes less from its diagnostic power than from two aspects not discussed in the article: Benchmarking and Signalling. Mr Swarup acknowledges that PISA has helped some countries improve their learning outcomes. Let us understand why.

Benchmarking: The most important value that PISA brings to all participating countries is providing information on how students of the same age in the rest of the world perform on an equivalent test designed to test application of language, mathematics and science skills. It may seem at first glance that this is an unfair comparison because different countries have vastly different resources at their command—a very common criticism of PISA in India.

This criticism is easily addressed by seeing how countries actually perform. Many rich countries like the US do not perform that well. Countries like China and Vietnam have done well in spite of a lower per capita income. Research suggests that rather than a country’s wealth, the effectiveness of its education spend determines its performance.

Benchmarking also tends to show what strategies work and what do not. If India aspires to become a world power building on its human capital, it needs to be able to do well in tests like PISA and learn from what works wherever in the world.

It is also critical for India to do International Benchmarking—something the NCERT National Achievement Survey (NAS) which benchmarks only among Indian states completely misses this point. Participating in international benchmarking can provide insights that cannot be garnered solely from looking within the Indian system. Imagine if India had not taken the PISA in 2009-10—we would have continued to believe, as the government believed then, that Himachal Pradesh and Tamil Nadu are at world-class levels.

Signalling: An equally powerful impact of PISA is the signalling effect it has within a country. The first signal sent by participating in PISA and debates about it is that learning outcomes are important. It is natural to think of education in terms of inputs (like days of teacher training, the physical size of classrooms and computers provided) and outputs (like teacher attendance, enrolments and education TV channels started). PISA—like any other good assessment—emphasises learning outcomes with conceptual understanding. Research has shown that inputs and even outputs do not achieve the outcomes we value, and at times, outcomes are achieved—for example by a dedicated teacher or a poor country that emphasises education—even when inputs are lesser than ideal.

But PISA signals much more—it triggers debates at various levels about learning and topics related to learning. Why are children in China performing so much better? What strategies is Vietnam (or Kerala or Delhi) using to improve education, and did they work? These debates lead to many preconceptions being questioned—for example, many teachers believe that poor children are incapable of learning well because their parents are not able to support them at home. Data from PISA has largely overturned the international perception that smaller class sizes mean better learning.

Signalling also aligns stakeholders towards a common goal. Mr Swarup’s article seems to assume that the responsibility for good education is primarily with the government. But students, parents, private schools, companies and even tuition classes play an important role. Assessments help make learning visible and thus shows these different stakeholders see elements of the problem that they could work on. It is not even important that stakeholders agree on what they take away from results like PISA’s as long as the assessment is scientifically designed and analysed. Complex problems like education need complex solutions, and a sense of positive competitiveness towards a meaningful goal—children learning better—is definitely welcome!

The power of benchmarking and signalling is that they show us what is possible! It helps us understand the relative strengths and weaknesses of our own system, identify best practices, and inject energy into making improvements. This increase in interest often generates momentum for all stakeholders, thereby allowing to implement targeted reforms.

Improving the quality of assessments and thus reducing rote learning: It has become clichéd to say that India has a highly rote-based learning system. Yet, this root cause seems to have actually worsened in India while many countries —including the poorest in Africa—have done better on this score. A recent working paper by Dr Newman Burdett indicated that in Indian Board Exams, “higher-order skills were almost entirely lacking… In the two African countries (studied), this rote-learning approach seemed less extreme.” Though the causes of rote learning are many, good understanding-based questions in important exams are one solution. PISA can—and has been—pushing for better questions and better exams. This exam reform will itself help improve the quality of learning as students realise that understanding is more important than mere recall.

Mr Swarup says that PISA is expensive but it is not—the fees to OECD at Rs 50 lakh and even the cost of testing fewer than 10,000 students is not very significant. Most other expenses related to taking the PISA test are actually directed towards improvements in learning (as Mr Swarup wants) though I agree they may not be well-planned or executed. The PISA sample is scientifically drawn and hence the small sample size is enough to give a fairly accurate picture of where learning stands. Of course, a clearer picture will emerge as more states participate in PISA (which is currently the hope for 2024/5 and 2027/8)

Mr Swarup also says that we should build on the NCERT’s NAS. I agree completely and many of us have worked to try and improve the NAS. But even today, however, NAS papers and results are not easily available, openly discussed or seen as providing useful insights about student learning in India. The NAS report is large and has data but few compelling insights about actual student performance in the way PISA and other good assessments provide. Government has been very resistant to allowing private companies or research agencies to undertake large scale assessments like the NAS. This is not good for a large country like India or fair to its people. Why does Pratham’s ASER (which manages to duck the need for government permission) have a much higher national and international standing than the NAS?

What is the real role of assessments in education? The real role of assessments is not to label a child or a country though all of us—parents, teachers, policymakers and the press – end up doing do that a lot. The power of good assessments is to spark reflection, discussion and debate. Though they are no panacea, if we choose to learn from PISA and act on it—a point I completely agree with Mr Swarup on—we can give India the education system its people truly deserve.

The post The learning power of PISA appeared first on EI blog.