The system was created at the Robotics Institute of the School of Computer Science and uses two cameras and a laser to detect fast, low-amplitude vibrations. Vibration can be used to reconstruct sound, capturing the isolated sound without distortion.
So far, the research team has made several successful demonstrations of the effectiveness of their system. They captured the isolated sound of the guitars playing at the same time and the sounds of music coming from individual speakers playing different songs at the same time. Scientists did this by, among others, tuning fork vibration and vibration analysis of … a packet of chips.
The dual camera system is able to capture vibrations from objects that are in motion, such as the movement of a guitar, and at the same time “sense” individual sounds from many other points.
The system works on an exercise basis analysis of differences in pattern of points on photos taken with the global shutter and rolling shutter.
Rolling shutter is an image capture method where a snapshot scans scene data that is not captured all at once, but in fragments, vertically, horizontally or rotated.
A special algorithm calculates the difference in point patterns from two recordings and transforms these differences into vibrationsto reconstruct the sound.
The pattern of dots refers to the way light behaves in space under reflection from a rough surface. Engineers create these patterns by targeting laser into the surface of an object that produces vibrations, such as a guitar body or a packet of chips. The pattern of the dots changes as the surface vibrates.
The rolling shutter quickly captures an image while scanning it, creating an image by stacking one row of pixels on top of another. On the other hand global snapshot captures an image in one photo.
One use for an optical microphone is creating computer visionsthat will be used in space, especially in autonomous vehicles. The new system is to help recognize and track objects in space.
The new system will enable better vision of imperceptible high-frequency vibrations.
Additionally, the optical microphone can be extremely useful for music creation or in monitoring the vibrations of individual machines in the production hallto detect early signs of a repair need.
“If your car starts making a strange noise, you know it’s time to watch it. Now imagine a hall full of machines. Our system allows you to monitor the condition of each of them by detecting their vibrations from one stationary camera.” Sheinin said.
The first visual microphones were developed by scientists at MIT in 2014.
For many years we have heard that we can be overheard on our phones, tablets or computers. Currently, there are at least a few spyware programs that can remotely activate the microphone on a given device and record sounds coming from the immediate surroundings.
Earlier Edward Snowden revealed that the CIA was using systems to spy on smartphones that even used phone game apps.
The new system has the potential to give spies almost unlimited new possibilities in their work.