Synthetic Exercises

In the late 1980s a former classical flute student, Andy Hildebrand, changes his career path and moves on to study advanced digital signal processing. Having become a specialist in stochastic estimation theory (and by now a doctor), Hildebrand starts working for Exxon, one of the world’s largest oil and gas companies. There he develops a very idiosyncratic solution to the one simple question that the corporation asks him—where is the oil? Hildebrand develops a program to process data from reflection seismology, using seismic waves to locate hidden gas and oil sources. By sending sonic signals into the earth and using advanced algorithms to correlate and predict (in other words, attune the data), Dr. Hildebrand could finally locate the oil as if by magic.

Around ten years later a small US-based software company called Silent Talker begins developing an algorithm to detect micro-facial expressions. Through the decades that follow the company becomes the world-leading developer of automated deception recognition systems, or put simply, algorithmic lie detectors. Then a few former employees come up with their own version of the software, which they call “iBorderCtrl.” Developed with the border patrols of Spain, Greece, and the UK, this software was commissioned and funded through the European Union’s Horizon 2020 program. The algorithm scans the facial micro-expressions of migrants and asylum seekers entering the EU and, based on their facial movements, determines their level of veracity—essentially whether they are “lying” or not.

One day over dinner Dr. Hildebrand is jokingly challenged by a friend to “invent something.” The friend then proceeds to suggest that Andy invent something that makes him sing in tune. It dawns on Dr. Hildebrand that the technologies of oil extraction and voice tuning have a lot in common—correlation (statics determination), linear predictive coding (deconvolution), synthesis (forward modeling), and formant analysis (spectral enhancement).

He rushes home and soon after patents a new technology called “Auto-Tune,” a software to automatically tune the human voice in an organic and imperceptible manner. This software becomes the dream tool of music studios and singers, as it eliminates the possibility of notes going out of tune ever again. The algorithm becomes a dirty secret of the music industry, as nobody publicly admits using it until the 1998 pop hit “Believe” by Cher. The song uses the now-ubiquitous auto-tuning sound effect—its first massive recognition and commercial success.

Having no scientifically reasonable way to train their algorithm in the art of lying, the iBorderCtrl team trained it using thirty-two hired actors, who simulated deceptive and truthful situations in a lab setting. This data was then used to train the algorithm to determine whether migrant are being deceitful about their intentions for entering the EU.

Synthetic Exercises uses a sound library I compiled while working on another piece for which I subverted and retrained the iBorderCtrl algorithm. In Synthetic Exercises, this library, which was used to train the algorithm to “vocalize” the data, became my main point of fascination. As I trained the algorithm, it started spitting out early, noisy variants—not quite the human voice, not quite the finished sonority either. It produced a library of its own—artefacts, vocal errors, and sonic by-products, all documenting the birth of a particular digital vocalization. Synthetic Exercises is an arrangement of these sonic events, rendered heavily through auto-tune, exercising speech without words.