Da Vinci, a family streaming service, provides award-winning educational TV programs for learning enthusiasts on multiple platforms, such as TV, mobile and web apps. Geared towards children aged 7 and above, their mission is to help create a kinder smarter world. The mobile app includes educational games and quizzes, serving a dual purpose: boosting engagement and providing tangible educational benefits.

My thoughts on the existing quiz feature

V1 of the quiz feature (aka Badges) was designed and launched prior to me joining Macademia, so I began my discovery phase by conducting a UX review to identify things that I believed to work well, and what needed to be redesigned for this new iteration.

It was very early in the project, but I presented my findings to the some of the senior stakeholders to confirm alignment with the business's perspective on the feature and to help identify key stakeholder requirements.

I was only at the start of the discovery phase but already my imagination was firing up! Initial ideas for the new concept included:

• Include video durations or a "recommended completion time" for badges
  
• Suggest badges based on recently earned ones
  
• Revise quiz format: Bronze for 3/5, Silver for 4/5, Gold for 5/5 correct answers
  
• Enable swiping between badges instead of exiting and reselecting
  

How kids interacted with the

existing quiz feature...

I wanted to observe kids try to complete 2 or more quizzes, to understand:

• Why did they pick the quizzes they did?
• How easy did they find the quiz?
• What would be the next quiz they’d play?
• Do they play any other games where they have to collect badges?

I conducted user testing at a local school with four kids from the target audience

Common interactions

As this was a redesign project I wanted to conduct a data analysis but ran into a bug!

I knew that user engagement wasn’t great but I wanted to study the data further to validate assumption and answer questions I had about user interaction.

Assumptions

1. The most played quizzes are the ones at the top of the page
   
2. The most interacted with quizzes are the ones with video content sub 5 minutes
   
3. Older children engage with quizzes containing longer videos, younger with shorter
   
4. Younger children have more attempts at a badge than older children
   

Questions

1. How many kids have engaged with the quizzes since launch?
   
2. What are the ages of the kids that play the most quizzes?
   
3. Is there a specific category (science, maths etc…) that kids are engaging with more?
   
4. Can we track quizzes started but not completed - is there a correlation to length/topic?
   

I was able to identify some reliable insights which included:

• 64% of badges were awarded to kids aged 7 and under, with 5 year olds being the biggest “engagers” - the feature was appealing to a much younger audience than we wanted
  

• At an average of 1.5k weekly active app users in the past quarter, only an average total of 300 badges (quizzes) were awarded each week -
  

• The recurring retention of the feature is low. Only 17.5% of users who played a quiz do it on two days per week, this number is 50% for users watching or playing content.
  

• The events that had been set up to track when V1 was launched weren‘t comprehensive enough to give insights about engagement with individual quizzes
  

Finding inspiration from other quiz based apps

I carried out a competitor and UX analysis of our competitors and some of the industries leading apps. The full analysis can be viewed here, and below are the strongest ideas and opportunities identified..

• Prioritise points over badges for better focus, leveraging a streak-based formula
• Enhance accessibility with features like question previews, skip options, optional timers, and video speed controls
  
• Sustain engagement by implementing a feature/button to check for new badges or avatar updates
  
• When a user gets a question wrong provide a pop up tip or option to view a snippet of the video
  
• Always have quiz/game instructions easily accessible if needed
  

Exploring what makes our quizzes unique

A crucial element to retain in the redesign was Da Vinci's distinctive approach of integrating quizzes with the channel's content. In V1, users watched a Da Vinci show episode and answered related questions. Show durations ranged from 5-20 minutes, with the quiz consistently featuring at the end. User testing had already revealed that engagement and retention declined as video duration increased.

Assisted by the content team, and working closely with our Education Specialist, I set about studying the shows we’d be using for V2, looking for patterns or structures that could be utilised. The full study can be viewed here.

Key takeaways

• Short (<5 mins) fast-paced shows were better suited to having the quiz at the end to limit disruption to the user experience

• The content we had to work with for V2 (American licensed, prime time, non-animated) limited us to mostly shows with a duration of >20 mins

• Of that content, episodes typically included 2-3 individual stories played as alternate “chapters” some ending on a cliffhanger...

• Existing segments in the longer shows, like the "coming up" feature or music interludes, were ideal for inserting quiz questions

• Having a way for kids to save their progress when playing a badge that is built around a long show would be useful

• Inserting quizzes into full length shows gives us an edge over competitors - with the exception of The Pet Rescuers all content can be found elsewhere, this could be a great selling point

• We should categorise the quizzes. A strategy could be to work backwards and create them around the content that’s available. Possibly base the quizzes around professions...
• Animals & Nature - Marine biologist, Animal rescuer, Conservationist
  
• Science - Forensic scientist, Chemist, Biologist
  
• Technology - Engineer, Architect, Mathematician
  
• Health & Medicine - Doctor, Paramedic, Surgeon
  

Tapping in to existing research

Understanding child psychology is a significant task! Whilst I regularly engaged in user testing with kids and conducted interviews with parents and teachers, consulting reputable literature quickly provided relevant behavioural insights for specific Da Vinci projects. Some of the research insights I felt relevant to this project are below.

• Learning becomes enjoyable for children when it aligns with their interests. To foster effective learning, encourage your child to explore topics that captivate them [1]
  

• Some kids only encounter a sense of control in education. When a child feels overly controlled or lacks autonomy in their learning, they may disengage. While guiding them is crucial, granting children control over their learning experience is equally important. [2]
  

• 9 year olds will pursue their interests with diligence and focus. They will want to work on a subject, topic, or a particular part of the curriculum until they become skilled and master it. [3]
  

• Gamification cleverly employs psychology. Ed-tech product dashboards can be crafted like a race with multiple levels to clear, each level earning the user awards. This not only boosts engagement but also guides users through their learning journey, providing a sense of control. Gamification fosters a desire to learn and progress to the next level, rather than feeling forced into learning.[4]
  

• Motivate by using an optimal challenge which is one that is slightly more difficult than what a child has already mastered, but is still achievable through practice and hard work. [5]
  

• Ways to help an unmotivated kid develop the right motivation include;
  
• Pique their curiosity in a new subjects by showing them the different uses of it
  
• Let them choose activities according to their interests without pressure
  
• When children are stuck at a problem, help them view it as a “challenge they can conquer”, not a “difficulty they need to overcome”. [5]
  

The full research document can be viewed here

How will we know the redesign worked?

Having concluded the discovery phase, I defined some basic KPI’s to measure the success of this project.

A change in business strategy meant a

change to our target audience

Our user base had traditionally been children of "tiger parents" seeking academic success and excellence through Da Vinci's platform. However, with a shift in business focus toward schools and homeschools, a new target audience emerged— including kids unfamiliar with or uninterested in learning.

Leveraging insights from the discovery phase about child psychology, engagement, and information retention, I worked on crafting a new user persona for this project, Mason.

Unlocking my inner 8-year-old mindset

with user journeys!

User journeys reveal how personas interact with the product and help identify pain points to address while generating new ideas. Creating user multiple journeys for Mason yielded insights and ideas, including:

1. Detail 2-3 key learning points that the challenge/quiz will teach, so the kid knows to focus on those specific parts and to also set the scene
   
2. Kids like Mason who aren't into science or maths, need to be able to play enough challenges that interest them, to get hooked, and then given the subjects they're not interested in a try
3. Provide quiz recommendations based on interactions and interests
   

Translating research findings into user stories

Consolidating insights from competitor analysis, user testing, UX review, data analysis, and literature review I crafted user stories for Mason which included:

• As a kid who finds it tough to learn, I want to watch short videos, like under 5 minutes, and then take a quiz. This way, I can understand things better and not get overwhelmed by long videos. Learning becomes more fun for me!"
  

• As a kid playing educational games, I want to control my learning. If it's too bossy or limits my choices, it gets boring. I like some help but love making my own choices for fun learning!
  

• As a kid who knows practicing makes perfect (just like in hockey), I want chances to keep trying things until I'm really good at them, so I can feel super confident and happy with my awesome results!
  

• As a kid who loves being the best at games, whether it's hockey or Ninjago, I want to see how much I'm improving so I can stay excited and keep trying my very best
  

• As an 8-year-old gamer, I want games to use names that make sense and are easy to understand, so I can quickly find the cool stuff I'm looking for and have even more fun!
  

• As a kid who learns computer games from my big brother (who's not always around), I want games to give me help when I need it, so I can finish them on my own without waiting for my brother!
  

• As a kid who gets lost in books, games, and shows, I want 'endless games' that keep adding new challenges and levels, so I don't have to keep searching for new games after I finish them
  

I needed an overarching concept before delving into features and functionality...

Based on the show exploration exercise, an idea I wanted to explore was creating categories of quizzes from the content available and base them around professions to create a sense of something more tangible and aspirational. These could be SEL and STEM focused to align with our content.

Research indicated that children thrive when learning about topics they’re interested in and when given the freedom to shape their own learning journeys. With this in mind I came up with the following concepts...

The importance of weekly catch ups

Regular weekly catch-ups with my Product Director proved invaluable. During one session, I presented my initial concepts, based on flexibility and freedom, but they weren’t aligned with the vision held by the senior stakeholders.

It came to light that after starting the project, senior stakeholders had become very interesting in the success of Duolingo and wanted to replicate it for Da Vinci with a fixed linear learning path.

Duolingo’s audience and offering was different to ours, and what worked for them wouldn’t necessarily work for us. I fought hard for my research backed concept and we agreed on a compromise, so I began my concept exploration again with the following goal:

Back to the concept drawing board...

NOW I can delve into features and functionality!

Aligned with senior stakeholders' thinking, I began contemplating various solutions for distinct features. Utilising mind maps, I jotted down ideas for specific features and explored design solutions from there.

True ideation involves generating more questions than answers, right?!

During the ideation phase when questions arose that I couldn’t answer straightaway or I wasn’t sure about the feasibility, I jotted them down and reached out to relevant teams to help me answer them.

Collaboration to refine ideas and identify issues

I shared my most promising concepts and ideas in our weekly product meetings often leading to in-depth discussions on a whiteboard! As a result, we opted to divide the workload into three iterations.

New core features chosen for V2 included:

Introducing Engineering to the V2 concept

I aim to introduce engineering early on to projects, to assess feasibility and address any issues before finalising wireframes for handoff to the UI team. I walked them through a greyscale prototype in order to focus on the functionality and tech requirements easier.

Following the meeting a few changes were needed. Streaks, points and the option to play one challenge into another were out for V2, as it required too much backend work.

The landscape quiz experience was also out as quizzes were constructed on a web-based platform, and lacking web developer resources for this project, we had to use the existing web template and keep the quiz experience as portrait.

Scaling fidelity to pitch for sign off

Several iterations later I was ready to present the concept to the senior stakeholders for sign off. I knew this presentation would benefit from a step up in terms of visual design so I created higher fidelity wireframes and walked them through a prototype.

They approved the overall concept but asked to remove a couple of “nice to have” features in order to speed up the build and meet the necessary launch deadline, crucial for upcoming partnership business pitches.

When reviewing test builds I realised the need for an additional step in our process...

My work was done for now. V2 wireframes and prototypes were approved and passed on to the UI team who worked their magic tweaking the designs and getting them pixel perfect for the developers.

Afterward, I collaborated in reviewing test builds, ensuring functionality, and maintaining product usability. It was at this stage I realised we needed an additional step in our process!

Several changes had been made to improve the visual appeal of the feature, but these changes negatively impacted the user experience I’d carefully crafted.

A UX Designers worst nightmare...

To streamline error tracking, we used a spreadsheet categorising issues by severity. Critical concerns were addressed pre-launch, while less severe ones were tackled based on resource availability.

However, despite categorising some issues as severe, we proceeded with the launch without fixing them, the most severe being no intuitive way to actually exit the quiz!

User testing post launch isn’t ideal...

I wanted to test the concept with kids from our target audience before finalising and handing over the designs but I ran into practical constraints.

Building an MVP to test wasn’t an option so it was agreed to conduct user testing with V2 upon release.

conducted a user testing session at a local school, with the following research goals:

• Do they understand what they need to do in the quest feature without any explanation being given?
  
• Are they watching the whole video before attempting the quiz or skipping to the quiz?
  
• In which order do they decide to tackle the quests and challenges?
  
• How long were they happy to play the quests before losing interest?
  

The results disproved some assumptions and validated others

Common issues observed included:

Lessons learnt!

• Establish terminology early on. This was needed when introducing a brand new concept to a global team.
  

• If working with a UI team, always check the final designs before hand off to developers to ensure no changes have been made that affect the functionality of usability of the feature.
  

• Different stakeholders have different definitions of severe errors. Sometimes you have to launch and find a quick fix after. Having a good relationship with the developers helps when trying to quickly troubleshoot issues.
  

• Compromising on a concept or a feature can result in a really poor user experience. Sometimes it’s better to concede for the overall good of the user experience.
  

• Using existing assets in concepts and designs will quickly make you everyone’s friend!
  

• Kid’s pick things up incredibly quickly and are more patient than you think!
  

• When launching a product/feature decide on what analytics to track and check in on them a week after launch to spot any bugs.
  

My Work

Let’s Chat

Sarah.lindleyuk@gmail.com

+447404588302

Resume