Now, a programmer duo Or Fleisher and Dror Ayalon have worked out a similar tech that can add motion tracking capabilities to almost any webcam with the help of machine learning. For their project called Skeletron, they used Google’s open source library TensorFlow and the game engine Unity.
The task that the system can do justifies its name. “Skeletron is a system that predicts joints and human skeleton position in 3d from real-time video taken by any (cheap) RGB camera, such as a webcam,” writes Ayalon.
“The system sends the data about the position of the human body to Unity” which can “allow engineers, artists, and creative technologists to use it to develop digital experiences.”
Their experimental project might not have out of the world technology, but it does its job of identifying human movements in real time in 3D without any sensors. And considering its price (a cheap open source software and low-cost webcam), it could be an alternative to expensive motion tracking solutions which need powerful processors and advanced sensors.