Neurobit PSG
Introduction
We will explore the design process of ideating and building Neurobit PSG, a clinically validated and deep-learning-enabled platform for auto-scoring and precise sleep event detection.
My Role
Timeline
Multiple iterations and improvements over 5-6 Months
Team
1 Product Manager, 2 Frontend Engineers, 2 Backend Engineers and Myself
Problem
Manual scoring is a long and time-consuming process while the current CLI solution has major usability and experience issues
Understanding the Problem
An estimated 50–70 million people in the world have a sleep disorder. Polysomnography (PSG) remains the gold standard for recording sleep, involving monitoring various physiological signals like EEG, ECG, airflow, and more.
Sleep scoring is done on this recording. This process involves:
Dividing the entire recording into 30-sec segments, and allocating a sleep stage to each segment
Identifying events such as oxygen level, awakening, breathing obstructions, shallow breaths, and leg movements, in turn, identifies various sleep disorders
Traditional vs Automated Sleep Scoring
Traditionally, a sleep technologist can take between 3-4 hours to manually score a single PSG file, which requires strong expertise and consistent attention for reliable results.
Automated sleep scoring solutions can help reduce the burden on sleep technologists, provide accurate results and create major impacts by saving patient’s diagnosis time.
Manual Scoring (before Neurobit)
Cadwell Software visualising a PSG file. The column on the left shows all the physiological signals while the two graphs on the bottom show the sleep stages & breathing events. A Sleep Technologist will use such tools to manually score a PSG file and identify any anomalies.
Neurobit PSG V1.0 (old version)
The first version of Neurobit PSG (known as Z3Score) was developed before my arrival. This version did not have a User Interface and used Command Line Interface to create and configure montage.
Neurobit PSG V2.0
The main goal was to make take the Command Line Interface and make a new UI with a major focus on accessibility and User experience. In order to achieve this we had to reimagine the entire flow of sleep scoring.
User Research
We decided to keep the user research approach straightforward and conduct just two user interviews, both the users were from research and clinical backgrounds and were a part of the National University of Singapore’s (NUS) Sleep and Cognition Lab.
Points of discussion
Current Sleep Scoring Workflow
Can you provide a step-by-step walkthrough of your current sleep-scoring workflow, both manual and when utilizing Z3Score?
Workflow Challenges
What are some of the fundamental problems you face in the workflow? The frequency of such problems?
Usability Test of Neurobit PSG v1.0 (Z3Score)
Along with the interview, we also did a usability test on Z3Score to determine the system’s limitations and assess the interactions of existing users. The goal was to validate some of our assumptions regarding the usability challenges associated with Z3Score and give us more insights into the users’ ability to interact with a complex system.
Insights and Opportunities
Manual scoring is a long and time-consuming process
Usually, two sleep technologists will manually go through ~8 hours of sleep data and then compare the scores for validation, taking anywhere between 3-4 hours for each file.
CLI Interface is not ideal
The current Command Line Interface (CLI) solution has a steep learning curve and demands significant time for routine tasks, leaving users feeling they lack control and causing doubt.
Traceability of actions is not present
There is no way to trace back performed actions which causes the user to be extra careful while doing even the basic tasks which hinders the user’s experience
No system to manage Licenses and Credits
As the new version was also going to use a similar system of Credits and Licences, it was crucial for us to find a way to find a way to make it easy for the users to manage it.
Journey Map for Neurobit PSG v1.0
As the process for manual scoring is completely different, it made more sense to use version 1 of Neurobit PSG as the reference to find pain points and opportunities for Neurobit PSG.
Opportunities
HMW reduce the complexity of creating and configuring a montage?
HMW give them the ability to trace their actions and allow them to rectify them?
HMW make the new flow familiar for the user?
HMW design a solution that caters to user's limited technological understanding?
Note: While various factors, like technical constraints and credit & license management, were considered, the points highlighted above emerged as the most pivotal factors influencing our ideation process.
Ideation and Exploration
Leveraging our key findings and establishing important goals, we aimed for a user-friendly solution with easy montage creation and configuration, user control, simplified interactions, and traceability. Our agile approach involved breaking down tasks, sharing ideas & designs early, testing with stakeholders, and iterating based on feedback. This was a great way to reduce risk and build momentum.
This page was our scratchpad on Figma where the entire team explored ideas, tried different patterns and gave feedback. Most of the initial collaboration happened here and on the office’s whiteboard.
Iterate, Test and Improve
As mentioned above, we worked on smaller timelines and tested our ideas with the appropriate stakeholders. Following the creation of significant design components or flows, our initial step was to conduct internal testing within our team. We then conducted user testing by sharing a prototype or a quickly developed MVP. This was a great way to gather additional insights, validate our findings and improve quickly.
Actionable Insights (Major Goals)
Overall, the feedback from the user was positive and as we shared the progress with them, it gave us confidence that we were moving in the right direction and we proceeded to develop the final product. Some of the actionable insights gathered were:
Reduce Touchpoints
In initial iterations, users found the channel mapping process too complex, leading to longer learning times due to limited technical understanding.
Reusable Configurations
Most users preferred 1-2 configurations for bulk PSG file scoring. We needed to enable them to reuse the same configuration across multiple files.
Management of Credits & Licenses
Targeting sleep labs and organizations with large teams, we aimed to develop a separate platform for managing non-scoring elements like credit transactions, licences and API key management.
Solution and Delivery
Most portion of this section is under NDA and can be discussed in detail over a call.
Montage Creation & Configuration Flow
We chose a streamlined workflow for the tedious task of montage creation and configuration. We aimed to simplify a complex process into an intuitive and efficient flow.
User flow for Montage Creation & Configuration
New Montage Configuration Flow
Our emphasis centred on minimizing touch-points, automating redundant tasks, implementing traceability of user actions and directing users' attention solely towards the critical aspect of channel mapping.
Prototype made on Figma
Resuable Montage Configurations
Users typically configured 1-2 montages for bulk scoring PSG files. Our solution allows them to download a pre-configured montage in JSON format.
Import Montage Flow
Usage History and License Information
We also added a Usage History page which allowed the user to trace every single file and see all the relevant info related to a particular processed file. Along with this, we added a License Information page which contained all data related to that particular API Key like Services & Biomarker info, License type and Credits info.
Usage History
License Information
Analytics Portal for better management of Credits, API keys and Licences
We designed and developed a separate platform for admins to manage other aspects of the product such as credits management & tracking, creation of API keys, viewing Activity Logs, etc. This allowed us to keep the main workflow of sleep scoring separate from other product functions.
This falls under NDA and can be discussed over a call.
Results
Currently, Neurobit PSG is being used by Universities and Research Centres around the world. While it's clinical adoption is pending due to FDA clearance, we have had extremely positive responses from Hospitals and Sleep Labs who have tested this product.
Some Numbers:
The average sleep scoring time for approximately 100 hours of sleep data decreased significantly, from 3-4 hours to <20 mins
User adoption and learning time were less than 1 week due to major improvement in the experience.
Neurobit PSG has been adopted by 6+ researchers specializing in sleep studies, with the respective research findings subsequently published in academic papers.
Other metrics cannot be disclosed publicly.
Next Steps and Learnings
While Neurobit PSG has given us a solid foundation to build off of, many factors still have to be considered to make a product that completely replaces manual scoring. Technical factors such as multi-format support, adding more scoring services, etc. are all important, which means more brainstorming sessions, more iterations and more designs.
At Neurobit, the entire team follows a unified product philosophy:
A digital product is never done. We can always make improvements, we can always make it better. Whatever we design, develop and create should quite plausibly be the best version of that thing users will experience.