Over the last few months, and even more so, over the previous few days, the term “assessments” seems to have become quite the buzz word. The National Education Policy (NEP) has drawn considerable focus on Assessments by mentioning census-based bi-yearly assessments in grades 3, 5 and 8 (1). It also notes assessment-led reforms in grades 9 to 12 and the setting up of a National Assessment Centre called PARAKH to streamline these reforms. Recently, the World Bank-aided STARS project (Strengthening Teaching-Learning and Results for States), which will catalyze work on assessments in 6 states, has also received a nod from the ministry and the media. Despite these dictums, reform in education is ultimately in the hands of the states, and it is the states that control the switch to unleashing the power of assessments.

Source: Dr Ramesh Pokhriyal’s (Minister of Education) twitter account, https://twitter.com/DrRPNishank

This brings up the question – what does it take for a state to conduct large-scale assessments? What do states need in order to get ready in terms of systems and processes to continuously conduct rigorous assessments that help it achieve this reform in its education systems? A few states such as Haryana, Madhya Pradesh, Gujarat, and Himachal Pradesh have been conducting large-scale assessments over the last few years. Educational Initiatives (EI) has been a pioneer in conducting several landmark large-scale assessments across as many as 20 states in the country. Most recently, EI supported the administering of the Haryana Saksham Ghoshna assessments.

Haryana Saksham Assessments

Since 2017, the state of Haryana has conducted over ten rounds of the Saksham Ghoshna assessments (the word ‘Saksham’ itself means ‘competent’ in Hindi). Essentially, these have been third-party-led assessments that the state has used as a form of certification. In its initial avatar, the 119 blocks in the state were allowed to ‘nominate’ themselves for the Ghoshna. Once a block nominated itself, some schools from the block would be randomly selected and tested in the next round of Ghoshna (the block had no control over which schools and which students would get tested). If at least 80% of the students tested in a block scored a minimum chosen score in each subject (this score was decided by the third-party), then the block was victoriously declared to be ‘Saksham’ or capable. By the end of Ghoshna 1.0, after nearly 2 years, over 90% of the blocks in the state had been declared ‘Saksham’.

In its second avatar, Ghoshna 2.0, the state decided to up its ante by covering more grades (grades 3 to 8) and by testing each child in more subjects (Math, Hindi, Science. Social Studies). Blocks were no longer nominating themselves, instead, all blocks were tested over two rounds of Ghoshna. Over 7 months, randomly selected schools across all blocks were tested twice. In a state the size of Haryana – which is mid-sized being about the 17^th largest in terms of population in India – this meant testing at least 5,00,000 unique students over 6 months. These students together gave about 20,00,000 unique assessments — all in a short span of 6 days, facilitated by more than 2000 personnel from the state. At the end of Ghoshna 2.0, 22 blocks were declared to be ‘Saksham’ because 80% of the students tested in these blocks had scored above the decided minimum score.

As the third-party assessment agency during Ghoshna 2.0, EI had the opportunity to watch closely and understand how a state readies itself to conduct large -scale assessments continuously. While several elements culminated in achieving this feat, some that appeared most critical are highlighted below:

• Streamlined systems & processes – Right from printing the assessment papers (for which many states need to go into a cumbersome tender process) to dispatching the papers to training invigilators, Haryana’s systems and processes have been tried, tested and now stand robust. Papers get printed and dispatched in a matter of 2–3 weeks, trainings get effectively organised in a mere 3–4 days and the assessments are carried out all over the state at the same date and time over just 2 days.
• Embedding assessments into other activities – Even if the state had managed to conduct large-scale assessments multiple times successfully, it’s quite futile if the rest of its activities don’t take inputs coming from these assessments. Haryana has created a Saksham Taalika, which maps its curriculum to skills. This allows teachers to teach skills in classrooms versus simply teaching chapters out of a book. The state also has a dedicated unit in its SCERT that creates the Student Assessment Test (SAT) papers on these mapped skills to test learning every two months – the SATs are additional assessments the state carries out without a third party
• Activated ecosystem – Assessments have become a part of the vocabulary of the state. Whether you meet a teacher, a block or a district official or even a parent, everyone believes in the power of Saksham Ghoshna. More importantly, they understand that assessments are central to helping the state progress in its journey of having children learn better.

As the state sets its eyes on PISA, here are a few thoughts to progress from good to great:

• Keeping it low stakes – The upside of high stakes is the energy and enthusiasm that it generates. (the Saksham assessments are more or less a movement on ground). On the other hand, the downside is that with so much at stake, sometimes schools and blocks will do anything to get the right results. High stakes often leads to higher mass assistance and therefore often the results captured don’t actually reflect the true level of learning in
• Test fewer students – Shift the focus from the blocks to the state. This will allow testing a much smaller fraction of students through scientific sampling while still gaining a complete picture of learning levels in the state. Doing this will reduce the logistical pressure of conducting large scale assessments across the state and will allow much smoother implementation of the assessment, thereby switching focus from implementation to quality.
• Keep re-enforcing the feedback loop – The most important part of assessments is to have the insights and data feed back into the learning system. The data from these assessments should inform teaching-learning from policy to blackboard, ultimately reaching teachers and students in classrooms.
• Keep exposing students to ‘good questions’ – While the Saksham Ghoshna assessments put good questions in front of students and teachers, it is important to continuously expose students to such good questions. This could be in the form of a good question a day or even in the SAT assessments – with more exposure to good questions, students will learn better with more understanding.

Keys to Conducting Large–Scale Assessments

While the path and strategy chosen by every state will be unique to its challenges (size, diversity etc.), the case-study of Haryana helps us isolate some critical elements of conducting large-scale assessments.

1. Processes

It is vital for states to invest in systems and processes. This would include seemingly trivial elements like printing assessments or setting up state-wide training sessions or disseminating results down to every teacher. Including checks and balances across these systems and processes will ensure they’re set up well and are capturing true levels of learning.

2. People

Assigning specific roles in the process to people across the system helps each person play their part. Designating roles such as monitors, trainers, field invigilators, consultants etc. helps get the desired outputs from each person and reduces chaos. Having a specific assessment cell with dedicated staff will help states make better questions and better assessments.

3. Perceptions

Ultimately the education system is made of people and people are driven by their personal motivations. It is important for teachers to understand that while their students may not have been tested, such assessments contain insights on learning levels for the entire state and therefore these insights would be relevant for them in their classrooms. Similarly, each district official needs to understand why properly conducting training for field invigilators is important and critical for the assessment and how her or his district will ultimately learn better because of this. It is important to spend time in helping people understand why the assessments are being done and how they will transform education in the state to motivate all players to participate fully in them.

With news reports of programs like STARS, centralized assessment agencies like PARAKH, I look forward to seeing many more states go deeper into assessments in the next few years. While such assessments might appear daunting from a distance, states can strategize and maximize their returns. Focusing on quality rather than quantity is what will ultimately move the needle – there’s no need to measure too much, just the need to measure well. Additionally taking a sample-based approach, choosing key subjects, testing yearly (rather than monthly or quarterly) and keeping it low stakes will help get most out of the process. After all, as highlighted even in the NEP, assessments are the key to educational reforms (2).

References

1. https://www.mhrd.gov.in/sites/upload_files/mhrd/files/NEP_Final_English_0.pdf
2. https://www.livemint.com/opinion/columns/nep-assessment-is-the-key-to-unlock-better-learning-11597018842931.html

The post Conducting Large-Scale Assessments – What Does it Take? appeared first on EI blog.