Guitarobot– Zohar Stephan Shalev (Israel)
Additional team member: Ilya Kuntsevych
The Guitarobot combines a Raspberry Pi computer, 30 servo motors, visual processing and machine learning software and a conventional guitar to produce a musical instrument that can read and play music on command. RealVNC is used to connect Guitarobot to the server where the machine learning computations take place.
In addition to reading music written in the guitar tabs format, Guitarobot can also play songs from a library of music stored in memory, or by an operator sending commands directly from a keyboard (via RealVNC Connect).
Guitarobot was created by Zohar and Ilya as their final electronics project before graduating high school last Spring.
Two students create a robot guitarist
For the final project of their high school electronics course, Zohar and Ilya were looking for an idea that would both impress their teachers and be an entertaining challenge. At the suggestion of Zohar’s father, they decided to create a self-playing guitar.
After struggling with an initial design that kept running into problems, the pair discovered a YouTube video that set them on the right course. It featured a guitar played by motorized fingers, all controlled by 1980s Commodore 64 computer. They decided to take the concept of motorized fingers one enormous step forward by adding modern visual processing and machine learning: a self-playing guitar that could also read printed musical notation!
Zohar and Ilya were both guitar players themselves, so they understood the complexity of the task they had chosen. Early on they decided the Guitarobot would be powered by a Raspberry Pi computer. Besides keeping the costs down, they both already knew how to code in Python, the main language used for developing Raspberry Pi software.
To keep things relatively simple, they decided Guitarobot should read a type of musical notation called “tabs,” which is specifically used for guitar music and (crucially) is less complicated than conventional sheet music. Still, getting the machine learning algorithm to recognize the circles and numbers in the guitar tabs took many attempts before they could get it to work.
Their teachers were impressed. Zohar and Ilya passed their electronics project with flying colors, and both received the “Bagrut” certificate which will allow them to attend university in future.
Notes on Guitarobot
Guitarobot’s Raspberry Pi computer runs on custom-made software programmed in Python and controls a camera and 30 servo motors for holding down or plucking strings.
When the Raspberry Pi camera detects guitar tab notation, it sends an image of the notation to a nearby laptop computer using RealVNC Connect. The laptop then renders the image into code, using visual processing and machine learning. Once this is completed, the code is sent back to the Raspberry Pi, which it uses to control the servo motors to produce music.
Each new piece of music Guitarobot learns using this process is saved and can be played on command from memory. An operator can also play Guitarobot directly using a computer keyboard, also connected using RealVNC Connect.
Development of Guitarobot is currently on hold while the two pursue other goals. In the future, Zohar hopes to attend university in the UK or Australia and, if he gets the chance, would love to make the Guitarobot even better at reading and playing music.
