NEEV Hybrid Program: Making A Case For Bichronous Online Learning
Why at-home digital learning?
It takes a village to raise a child and definitely more than the traditional school system to prepare them for today’s competitive career landscape. Particularly in the current times where effective teaching hours in schools are much below that mandated by experts[1], COVID lockdowns have had a substantial detrimental effect on students’ academic levels[2] and competition for quality colleges and careers is increasing[3].
One of the best bets to address this gap in effective learning time for a typical school going Indian child to actually achieve successful life outcomes is at-home digital learning. In a quest to add more meaningful learning hours to students’ academic life, Avanti has tried synchronous digital learning solutions with proven success in the form of live classes. Today, more than a thousand students attend Avanti live classes every day making it one of India's largest live class programs in this specific demographic.
Fig 1: A glimpse of NEEV Live Class Program
From Synchronous to Asynchronous Learning:
In Haryana, live classes are conducted as part of the NEEV Live Class program where hour-long live classes are conducted three times a week for math remediation of middle school students. These live classes focus on 125 foundational Mathematics learning outcomes for grade 9th and 10th government school students. Roughly 30% students attend these classes meaningfully, i.e. attend more than 2 hours of live classes every week. This is much higher than similar high-touch models across the education sector, where the typical attendance is around 5%. Students who attend these classes regularly typically improve their performance by 10 to 24 percentage points from baseline to endline.
Given the positive results of our synchronous digital at home learning intervention, our next step was to find a lighter-touch, more scalable version of the remediation program. To do this, we decided to substitute some synchronous components of the NEEV program with asynchronous components like our Quiz product or Plios.
Fig 2: A Glimpse of Avanti’s Quiz Product
Who are the primary users for asynchronous learning products?
Is asynchronous learning suitable for all learning types? Our previous projects with asynchronous learning told us no. Asynchronous learning is a low touch model with minimal human intervention. It also is most likely to be adopted by a user set showcasing demonstrated intent of using solving the problem the product seeks to resolve. Which of our students have demonstrated intent towards self-directed learning with minimal human touch?
Based on our experience of working with them during live classes, the answer was our academically proficient students. Based on our selection, this would be the top tercile of students as per their performance in our baseline assessment.
What is the persona of these students? What are these students like?
Fig 3: Characteristics of Academically Proficient Students
Typically academically proficient students are:
Self-motivated to learn: They typically seek academic resources outside of typical classroom learning. As per our internal survey, a number of them use YouTube videos to study.
Need autonomy in how they learn: Compared to other student types, academically proficient students are most likely to be selective about their learning experience, choosing their own mode and facilitator of learning.
Curious and/or Competitive: Academically proficient students are mostly both curious and competitive and seek to know where they stand academically. They also are more likely to demonstrate meta-cognition skills and be more interested in tracking their learning journey. This is a good opportunity to build leaderboards and user profiles.
Need Quick Gratification: Most of these students are impatient and need quick replies to questions, doubts, etc.
Student-Centered Product Design:
Given that academically proficient students seek to design their own learning experiences, if we wanted adoption of our product, we needed to co-design it with them. Thus started a series of conversations with 13 students of GSSS Chhapra and GSSS Balsamand co-design the hybrid NEEV Program.
The 1st Question: Which is the most suitable mode of asynchronous learning?
Given a choice of assessments, videos, and plios, the students selected plios to be the most suitable form of asynchronous learning. Plios are interactive videos where a video is overlaid with questions at regular intervals. As a user watches a video, questions pop up that they can answer (or skip) and continue watching the video.
The students’ choice of plio also corroborated with Avanti’s earlier learnings where we found student engagement on plios to be five times higher than regular YouTube videos.
The 2nd Question: Which plio type would students prefer?
Till date Avanti has experimented with two types of plios. The first type are concept plios which are regular animated or non-animated videos with little to no human presence. The second type are exam plios where, similar to live classes, a teacher is featured in the video and uses a whiteboard to teach the concept.
Based on conversations with students, as well as previous qualitative and quantitative data, students prefer videos with human presence in them.
The plios designed for the hybrid program are hence similar to teachers teaching a live class, with pop up questions substituting the real time Q&A session. Here’s a sample of a hybrid program plio. Only one feature was missing- Doubt Resolution.
The 3rd Question: What should be the program structure?
The super enthusiastic students of GSSS Chhapra and GSSS Balsamand, after multiple rounds of trials and feedback cycles, helped us arrive at the following structure:
Fig 4: Weekly structure of the synchronous NEEV program vs. the hybrid NEEV program with asynchronous and synchronous components
After a round of open-ended interviews with students, the structure that emerged was substituting live classes teaching new concepts with plios, followed by a live class for doubt resolution.
Through previous pilots we know only around 14% students of this profile ask doubts organically. Hence, to make doubt classes more effective, plio results are shared with teachers beforehand to inform them of which topics were potential doubt areas. In addition, an assessment on both concepts is also conducted weekly and acts as a formative assessment for teachers to plan doubt classes.
The final structure for the program was concept teaching via plios on alternate days in the first half of the week. This would be followed by a weekly assessment and a doubt class to resolve doubts and low-scoring concepts covered in the plios and assessments. All throughout, the center for communication for all learning sessions and followups were whatsapp groups.
Fig 5: An example of social influence- Students sharing plio results on whatsapp communities motivates other students to attempt plios too
A Higher Adoption Rate
Since it was being co-designed with students, the NEEV hybrid intervention was piloted in three stages. Overall, we saw a healthy adoption of more than 50% regular users in each stage of the pilot. For reference, this percentage of regularity is 10 percentage points higher for when these same students attend live classes. This is because of the flexibility asynchronous learning provides in terms of time and consumption.
Pilot 1: July 2022: The MVP was codesigned and piloted in 2 schools with 16 grade 10 students over a duration of 2 weeks. 69% students were weekly active users (WAU) who attended at least 1 learning session per week, 62% students were regular students who attended >50% learning sessions per week.
Pilot 2: August-September 2022: The second pilot was conducted in 3 schools with 32 schools over a duration of 4 weeks. 78% students were weekly active users (WAU) who attended at least 1 learning session per week, 78% students were regular students who attended >50% learning sessions per week.
Pilot 3: November - December 2022: The program was piloted with 124 grade 10 students over a duration of 5 weeks. 100 students were onboarded to the program, i.e. they attended at least one learning session. 80 students were active weekly and 65 students attended more than two learning sessions per week. In the table below, the attendance of the same set of students in live classes conducted at the end of the year is shown for comparison of adoption in a 100% synchronous program.
Fig 6: Adoption of Hybrid NEEV Program vs. NEEV Live Class Program
Out of the three components of the hybrid program, plios have the highest usage, with the ratio of average users per component being 35:26:29 for plio, assessment and doubt class, respectively. The average duration of our plios is 20 minutes, out of which the average captured watchtime in 72% or 14.5 minutes.
Fig 7: Comparative adoption and feedback on individual components of hybrid program
Impact on Learning:
In the 2nd pilot, the 25 students (78%) who engaged with the program regularly showed an increase of 28pp on score from baseline to endline.
To test the impact on learning on a larger cohort, we conducted a second baseline assessment during the third pilot. The baseline was conducted prior to beginning the intervention and scores were compared with the corresponding endline assessment. The positive impact of the program is clearly visible as the number percentage point increase from baseline to endline increases with level of student engagement with the program.
Passive students who did not engage with any component of the hybrid program showed an increase of 15 percentage points, while irregular students who engaged with less than 50% of learning sessions per week showed an increase of 19.5 percentage points. Students who engaged the most with more than 50% of learning sessions per week showed an increase of 25.15 percentage points.
Fig 8: Impact on learning increases with meaningful engagement with the Hybrid Program
Fig 9: Increase in pp change from baseline to endline assessment with increasing engagement
Other Observations:
Online communities keep students engaged: Whatsapp groups served as online communities where students shared their results, asked doubts and communicated with teachers and each other. Online research also shows that when synchronous communication features are integrated with asynchronous features, the online course is more engaging, increasing learning outcomes, positive attitudes, and retention.[4]
Online communities boost adoption: As students of this academic category are eager and competitive, the sharing of scores by one batch member triggers the rest of them to also complete their learning session and share scores
Results are teaching style-agnostic: Adoption and results of the intervention were agnostic to the teacher type. This pilot was conducted with three teachers, each very different from the other in terms of teaching style and level of interaction with students. By design, even the teacher featured in the plio and the one taking the doubt class of the batch was different.
Low Offline Field Engagement: Unlike NEEV live class program, which required monthly school visits to engage with students and boost adoption, school visits did not play a major role in this program. The team visited schools once in two months, only for the purpose of conducting assessments.
Way Forward: Next, we plan to scale the intervention to 15 schools, around 400 grade 7th students in a year-long intervention and check for impact on long-term learning. The model will also be tested in Avanti’s Test Prep program for grades 11th and 12th.
References:
Involvement of Teachers in Non-teaching Activities and its Effect on Education, NIEPA, Ministry of Human Resource Development
Impact of COVID-19 Pandemic on Lives of Children, Volume 1, Issue-1, Children First, DCPCR
Is The Pressure Of Competitive Exams Ruining An Entire Generation?, Atrijo Banerjee, Jun 28, 2021, Youth Ki Awaaz
Bichronous Online Learning: Blending Asynchronous and Synchronous Online Learning, Martin et al, Educause Review
