The dance app that helps assess Parkinson’s
A few years ago a dance craze called the Vosho swept South Africa. The dance involves variations on a hop, a kick, a drop and a pop — hop on your left foot, kick out your right, drop your backside into a crouch and pop back up. Not for the weak-kneed, it is both endlessly adaptable and quite specific — you know a good one when you see one.
Babusi Nyoni, a developer and designer, saw in the dance an opportunity to test a new machine learning model called Tensorflow — which was released by the Google Brain deep learning artificial intelligence unit — that can estimate and judge a body’s movements.
In December 2018, he and his team at Triple Black, the Amsterdam-based agency he co-founded, released Vosho Fo’sho, a mobile app that judges users’ ability to perform the perfect Vosho.
“The intention was just to take something that was fresh off the shelf — the latest in the trend of computer vision — and put it in the most unexpected space, and have it have this context that the creators of that technology would never have envisioned,” says Nyoni, 32.
Nyoni soon realised he could take it somewhere even more unexpected and, he says, meaningful — a home for the elderly in his hometown Bulawayo in Zimbabwe. The Vosho app was, at its core, an assessment of movement over time, which is the kind of assessment necessary to diagnose Parkinson’s disease.
“The idea behind that was: ‘How can we build this world-first application through an African lens?’” he says. “I’m building the tech. I’m going to do the research in a small city in Zimbabwe. We’re going to build this for black health practitioners, for black patients and then in a sense . . . reverse engineer it to the rest of the world.
“Everything else is done with the west first and then [applied to Africa],” he adds. “So it was a very deliberate act of Africa-first innovation.”
Nyoni presented a prototype of his Parkinson’s diagnosis app at a conference at Oxford university in April last year, but then got caught up in another AI project he had launched — a Facebook Messenger chatbot that answers medical questions (see box below).
In January, Nyoni refocused on the Parkinson’s app, called Patana AI, partnering with the Ekuphumuleni Geriatric Nursing Home in Bulawayo.
The app launched in March in a small pilot phase, as Nyoni and his team collect data and feedback on usability and the performance of the AI tech. It is designed to work on the low-end smartphones that are common across Africa, so that users can simply point and shoot in order to perform the assessment.
Nyoni has a history of turning seemingly trivial technology toward more meaningful use. In 2016, while working at an advertising agency in Cape Town, he developed a custom AI football commentator for the brewer Heineken, which was deployed during that year’s Uefa Champions League. The algorithm collected football-related tweets from around the world and converted them into language that allowed a Twitter bot to banter with fans about the sport in a natural way.
But Nyoni then realised that he could repurpose it to predict forced migration patterns across Africa. His team removed the football banter and plugged in information including population growth, food production, extreme weather and gross domestic product changes.
The results revealed patterns showing when countries had reached tipping points that led to mass migration in the past — and, crucially, can help to predict under what conditions countries might reach those points in the future.
That project led to a partnership with UNHCR, the UN Refugee Agency, trying to predict the movements of displaced people in Somalia. However, the project hit a snag when they realised the available data was too limited to provide insight into how or when people might move.
Then the project team discovered that people sell their goats — a key vessel of wealth in rural Somalia — when they are preparing to move, which sends prices down. Those types of unorthodox data points are now included in the system, which provides for a more complete picture of potential mass displacements.
Nyoni says his ability to tap into innovative solutions is borne of his impoverished upbringing in an economically devastated Zimbabwe.
“Growing up in an environment where you have to always constantly be solving in some way or another, just to improve the quality of your life, is something that I guess prepared me for that moment where I could look at what I built and then imagine the use case for it that would be much more impactful than a dance move assessment or a football commentator.”
How small talk made a big difference
Babusi Nyoni’s biggest success so far is Sis Joyce, a Facebook Messenger chatbot that answers medical questions. It was purpose built for a chat app because many young Africans use them to communicate.
“Africa has some of the worst healthcare in the world, and there are opportunities for tech to plug some of the gaps that governments fail to,” Nyoni says. “But Africa also has some of the most expensive internet in the world. What it means is that innovations have to leverage existing behaviours as opposed to creating new ones — don’t try and invent an app culture where there isn’t one, because there isn’t one for a reason.”
Coronavirus-related messages to Sis Joyce have jumped in recent months, prompting Nyoni to partner with the city of Bulawayo to provide users with real-time data and health trends, as well as a resource centre to address citizens’ Covid-19 questions.
Sis Joyce is based on a natural language processing model trained on 800,000 doctor-patient transcripts. It was rolled out in eight African countries last year but one problem was that the model was not trained on social interactions.
Nyoni added a second layer of natural language that could both engage in small talk and answer simple questions such as “what is a CAT scan?”. Sis Joyce is now growing in the US, even though it has not been promoted there.