NSERC

Research story

Technology with a human touch
Group photo of a research team. Hanlin Yu holding a display screen, Jian Gao holding an instrumented violin and John Madden holding a new full length violin fingerboard.

From artificial muscles made with everyday materials to robotic skin with human-like sensitivity, Dr. John Madden brings together materials science, electronics and robotics to develop technologies that sense and move like living systems. His work explores how machines can better interact with the world and how we can use them to improve health and mobility and support learning.

You have led research in many facets of electrical and computer engineering over your career. What motivates this interdisciplinary approach?

What motivates me is curiosity. I’m always keen to learn about new areas and connect knowledge from different fields. This began during my master’s degree, when we wanted to build micro-robots. To do this, we borrowed methods from electrochemistry to make structures for the robots. These structures needed to move as fluidly as the human body does, yet nothing we created moved quite like a muscle.

We then investigated how we could create a muscle-like material. That work reminded us of an important fact: human muscles don’t work in isolation—they have the nervous system to guide them. So, if we were going to successfully mimic muscles with our actuators (the muscle’s mechanical equivalent), we needed sensors and electronics to act as an artificial nervous system.

There’s a range of problems we would like to solve with technology—and now, we have so much computing power. But that power requires more information from sensors to guide it.

You work on developing artificial muscles and smart materials. What recent breakthroughs are you most excited about?

I’m excited about our work with Honda to create fully sensorized, artificial skin for the Honda Avatar Robot’s hand. The hand has sensors that can distinguish really fine textural details, achieving fingertip sensitivity that’s comparable to human touch. Now that this is developed, we would like to bring it to market so it can be used in robots. It’s great to have programs—like NSERC’s Idea to Innovation grants—that help get these projects out of the lab.

We also found a way to make artificial muscles from everyday materials, like nylon and polyethylene—the stuff used in clothing and packaging. The artificial muscles expand and contract by heating and cooling, which generates surprisingly strong and fluid movements. An early application is putting these fibres into compression stockings―stockings that contract and expand, helping return blood from the legs to the heart. We hope this will speed recovery after intense exercise. In addition, many people experience dangerous swelling due to blood clotting in their legs and feet. This could improve quality of life for millions of people.

In partnership with Yamaha Canada Music, you are now looking into musical instrument technology. How can invisible sensors make learning to play an instrument more accessible and rewarding?

As a child, I endured a lot of violin lessons. Practice was difficult, so I stopped playing. I hope that our technology will make learning an instrument much more enjoyable – with immediate feedback making it more interactive.

Jian Gao, a master’s student in the lab and a violinist, was looking for new applications for our sensors and a PhD project. The Yamaha Guitar Group in Victoria (a leader in innovations for musical instruments) suggested we outfit a violin with built-in sensors. These sensors give the violinist feedback about their finger positioning, timing, posture and transitions. In a sense, the musicians have a companion helping them learn. This technology will allow us to understand how musical virtuosos move their fingers to create distinct sounds, which would in turn help us reproduce their performance and understand what makes it unique.

To provide violinists with feedback, our initial prototype will have a visual interface where the learner sees their sheet music on a screen and compares the note they played with the note they were aiming for. The interface will also show timing errors. The next version will be similar to a video game with a points system. Gamifying the process would make it fun and potentially competitive.

How could this sensor technology be applied to other devices?

There are applications that will help elderly people, whose muscles get weaker with age. We’re exploring ways to track a muscle that people don’t always think about: the tongue. A 20% loss of tongue strength can cause trouble swallowing, which is problematic for obvious reasons. We would like to use sensors to help people improve and rehabilitate swallowing, avoiding the need for feeding tubes as well as conditions such as pneumonia.

We’re developing a device that is a cross between a mouth guard and a retainer. It is placed on the roof of the mouth and detects the position and motions of the tongue. We will use the collected data to understand normal versus abnormal swallowing. Eventually, we hope the device will help the wearer correct their swallowing.

We will also test a similar device to support speech therapy. With this tool, those with speech impediments or trying a new language could learn quickly.

What is your next research challenge?

My next challenge is to work with the colleagues on my team—many of whom are trained in entrepreneurship—to transition lab technologies (like robotic skin, artificial muscles, or speech-assist devices) into products that companies and people want.

I want to create an ecosystem where innovations from the lab become impactful products that benefit society, the economy and students. It’s a daunting challenge because I’m a professor, not a businessperson, but I’m motivated by the opportunity to make a real difference in people’s lives.

This interview has been edited for conciseness and clarity

About Dr. John Madden

Dr. John Madden is a professor of electrical and computer engineering at the University of British Columbia and is an associate faculty of the School of biomedical engineering. He also serves as the director of Mend the Gap, a global, interdisciplinary team working on spinal cord repair.