SenseMath

After building a prototype during a week-long jumpstart, in 2021 we developed an app together with Royal Visio that allows blind students to understand graphs through sound. SenseMath makes math more accessible, useful and fun for everyone!

The idea
Visually impaired students generally interpret mathematical graphs using physical aids such as paper on which the graphs are embossed. That way students can feel them. But that takes a lot of time and students can easily make a mistake when reading. Mathematics is not getting any easier for them. Visio, a center of expertise for blind and partially sighted people, came up with the idea to convert graphs into sound using an app. The aim was to help blind and partially sighted students understand and interpret graphs more quickly and easily. And above all, let them do maths more independently.

Validation
To validate the different starting points of the initial idea, we decided to do a jumpstart of a week. First of all, we found out whether the assumption is correct that sound makes it easier to understand graphs. We then started researching what kinds of sounds we would need to provide insight into tough mathematical assignments. Finally, we answered the question of how we could offer this in such a way that students and teachers can use it immediately.

Prototype
In close collaboration with Visio, teachers and students, we made a prototype app within that week. In this we created a lot of settings and examples to present to the target group and see what works well. Two blind Visio employees were present during the entire jumpstart to advise us, think along and immediately test everything we made. Halfway through the week we visited a Visio school in Amsterdam, where we could do a first test with students and teachers. With these findings in hand, we further developed the prototype app.

Superpowers
After the jumpstart and the successful test phase, the iOS app was actually realized in 2021: SenseMath. SenseMath converts a mathematical formula or function, such as y is x squared plus 3, into sound. Mathematical elements such as intersections and peaks are made audible. Entering a graph is similar to entering a graph into a graphing calculator (although SenseMath is certainly not a graphing calculator). With the audio reproduction of a graph, a blind person can get an overview of the various properties of a graph in a short time: up to 12x faster than with a tactile drawing.
We have created a tool that ensures that blind students can quickly and independently keep up with their sighted classmates. By working together with students with a visual impairment on a tailor-made solution, we have achieved that they are empowered. It is an app that fully meets their needs and capabilities. SenseMath gives blind students superpowers!

Focus on (pleasant) sound
With SenseMath, accessibility is not an afterthought. From the start, users with disabilities have been central and helped develop the app. Together with visually impaired students and a sound engineer, we invested a lot of time in finding sounds that are functional but not annoying to listen to for a long time. The app must sound pleasant when doing homework. In the app itself, students can choose from three options the type of sound that works best for them.

Especially for VoiceOver users
The Dutch and English app has been developed specifically for VoiceOver users, but can be used by anyone. All the graphs from the math book are already ready in the app, so they don't all have to be re-entered when students are doing their homework. They can view assignments from their math book directly in the app, enter a function themselves and listen to and analyze the graph with the sound that they like. Using the VoiceOver rotor, they can quickly turn all math elements on or off. When they want to enter a function themselves, they can do so with a specifically designed keyboard in the app that is displayed visually and aural. This special math keyboard works with VoiceOver's touch typing feature. All mathematical functions can be read in both speech and Braille, via a linear or mathematical representation.

Impact
The target audience of this app may be very small. Think back to your own high school. How many blind students were there in your class and how many of them chose mathematics? This does not alter the fact that the impact of this free app is very large. SenseMath does not only empower hundreds of students in mathematics education. Math teachers are now humming graphs to a class of blind students!
And that's not all. SenseMath proves that you can offer a very nice interface with sound. A purely auditory interface turns out to be just as rich for a blind or partially sighted person as a visual interface for the sighted. Just as beautiful, just as funny, just as smooth and above all just as intuitive. In short: by creating a completely new way of interaction, we have been able to turn a limitation into an opportunity with SenseMath.

Awards
The SenseMath app won two golden Dutch Interactive Awards in 2022. The DIA are the prizes for the very best interactive online work by Dutch agencies and have been presented annually since 2010. SenseMath received the award in both the Tools category and the Future Interfaces category. The jury praised the impact and worldwide potential of this product.

In 2022, SenseMath also won two bronze Lovie Awards, Europe's most prestigious internet award. The app was awarded in the categories Experimental & Innovation and Best Practices. And at the Dutch Creativity Awards, SenseMath received a bronze Spin in the Digital Services category, where the jury was impressed with the results of the project and the reactions from students and teachers.