The Fischinger Doodle was a beautifully realized art project that focused attention on the music visualization genius of artist and animator Oskar Fischinger on the occasion of his 117th birthday.
It also demonstrated a music composition interface that musically untrained users learned to use with only the help of a one line text prompt to compose a one measure music loop in a few moments time. An amazing accomplishment.
Judging by the blizzard of tweets and Facebook posts that resulted on June 22nd, thousands of small compositions were created and quickly published. An eruption of unexpected musical invention from people who didn’t wake up that morning expecting to write their own original piece of music.
This was extremely impressive, since the act of music composition is usually thought to require the acquisition of a body of musical knowledge that most take years to master.
What was the composing interface that was built into that art project? It’s clearly valuable. Clearly, we should document what that was, as best we can.
The doodle, as it launches, invites visitors to “Click Around To Create Your Own Visual Music Composition”. On first click or touch within the window, this prompting text disappears, and an array of small hollow, diamond shaped graphic objects is shown.
The diamond graphics displayed are arranged into a latticework of 11 rows and 16 columns. As soon as a click is made within the interior of one of the diamond shapes, its interior is filled in and a sound chimes. It’s apparent that a filled-in diamond represents a fixed musical pitch.
As soon as the first click is made, an animation starts in synchrony with a music loop.
As a user makes clicks within the diamond grid, they come to discover that all of the diamonds in a row play the same pitch, and that the pitches of any filled diamonds that reside in the same column all start to play the same instant.
They notice that the filled in diamonds that play in an instant, are highlighted while they sound and lose their highlight when they become silent. The instant one column stops sounding, the filled diamonds of the next column to the right, highlight, and begin to sound.
By listening to the music track that they’ve created one click at a time and watching the diamonds of each column, highlight, play, then unhighlight in synchrony with the music that’s synthesized, they realize that the pattern of highlighted columns within the lattice moves to the right at a fixed rate. Each column plays its pitches for an identical duration.
They come to understand understand that they are hearing a musical loop, as the sequence of highlighted columns moves back to column 1, the instant column 16 stops playing.
They also quickly intuit that each column of diamond shapes represents one unit of equally measured time within the loop that is performed.
Once they happen to click on an already filled diamond, they discover that it becomes unfilled again, showing only its outline, and then discover that the sound corresponding to that diamond no longer sounds once the animation reaches that column of the grid again. The changes made to the synthesized music track occur without any apparent interruption to the music track that is played.
Using the doodle, a visitor eventually discovers that the location of diamonds within the lattice has significance. It ends up that clicking in the highest row of the grid, produces lowest sounding pitches, and that clicking in the next lower row, produces a higher pitch than those played by the row above. The rule is if you click higher in the grid, lower pitches are produced, and if you click lower in the grid, higher pitches are produced. (To me this seems backwards. C'est la vie.)
When the user clicks in the diamond grid to make a musical change, they receive instant aural and visual feedback for that change. This extremely responsive feedback allows a user to quickly decide if a change they’ve made to their loop is a keeper, and to toggle it away if it produces a bad effect on their music.
Was that composing idea an invention of Google’s engineers, or did they borrow a great set of ideas to realize the composing portion of their project?
Even though this music composition interface is masterfully integrated into the texture of the Fischinger Doodle, this is the third time that this idea has made a notable splash in the marketplace of music software ideas.
In 2005, this kind of interface was introduced in a hand-held tablet sold by Sony that was loaded with embedded synthesis software, called a Tenori-on. The surface of the Tenori-on showed an array of 16 rows and 16 columns. Each cell of the grid contained a single lighted, circular touch sensitive button. Touching an unlighted button, caused it to light up. Touching a lighted button, turned it off.
In the Tenori-on, touching a higher button in the grid to light it, would cause higher sounding pitches to sound and touching lower rows of the grid would cause lower sounding pitches to play.
Other than this opposite, more natural arrangement assignment of pitches to grid rows that was used for the Tenori-on, it played a synthesized music loop in synchrony with an animation that using the same interface rules that were used for the recent Fischinger doodle.
It was instantly noticed how responsive the Tenori-on was to user action, and how easy it was to create a music loop using the device. The Tenori-on was a very nicely designed device, and generated a season of excitement but eventually died in the marketplace.
Why did the public lose interest in the Tenori-on? Why has it mostly been forgotten?
It was quite an expensive device for what it did, about $1400 in the US. The number of pitch sets that could be assigned to the rows of the tenor-ion was limited, so it seemed that the melodies that it was possible to produce were too similar to one another after awhile.
Also, any ideas you invented with the Tenori-on, had to be performed by the Tenor-ion. If I remember correctly, there was no facility provided to transfer a representation of the music that was invented to another more capable synth or sequencer where the ideas invented could be made part of a larger work.
What was the second notable appearance of this music invention interface?
In 2011, software developer Andre Michelle, the principal of the Audiotools website, created a software only analog of the Tenor-ion music editing interface that ran in a web page, which he called a tonematrix, as a means to demonstrate the features of a software sequencer that he was developing for use on web pages.
It was very cool to see this interface abstracted from its original home running on a dedicated hardware device to run smoothly within a webpage. A major accomplishment.
It seems that he did not make a concerted effort to publicize the existence of his tonematrix software. The focus of his efforts was to demo the audio processing capabilities available on the Audiotools site.
Even though this was not the main focus of his effort, he did expand the state of art by making it possible to share the loops that were invented using his tonematrix animation.
An empty Tonematrix on the Audiotools website was launched using the URL
https://tonematrix.audiotool.com
By adding a parameter list to the end of this URL, any tonematrix loop that a user invented could be recorded by adding parameter values to the end of the tonematrix URL, which encoded the state of the tonematrix that played the user’s invention.
For exmaple, this URL launches https://tonematrix.audiotool.com/_/g.0.20.0.80.40.0.g0.0.100.80.10.20.0.g0.200
performs a tonematrix loop that I composed a few seconds ago. I acquired that encoding of my invented loop by pressing the Copy Link button on that page, and then pasting the captured URL on this page.
Michelle loaded the rows of his tonematrix implementation with the pitches of a pentatonic scale, a pitch set that provides the basis of many varieties of folk melody throughout the world.
Why isn’t the Audiotools Tonematrix better known?
At the bottom of his original tonematrix page on auditors.com, Michelle included a comment thread. Whenever someone new stumbled across the tonematrix page, they would make a loop that they liked and then wanted to share their work. They would post a “hey, listen to this” message to the end of the page’s comment thread. Eventually, hundreds of user compositions were submitted to this thread.
At a certain point, auditors became irritated by the accumulation of these user contributions and purged the message thread. It seems that a community of neophyte music creators was forming in the tonematrix message thread. For whatever reason, audiotools decided to bulldoze that community’s meeting place on their site.
To me, it seems that Audiotools ignored a blazing signal that excitement was building around an incidental demo that was built for their site, and for whatever reason decided that it didn’t fit with their vision of the development of their site.
The Fischinger Doodle demonstrates, this time to an enormous audience, the value of this interface design for allowing one to quickly invent original music ideas.
The big question: What would one want to build around this interface, if you wished to create a first class music composing tool?
It seems likely that this click in an animated matrix interface idea could become extraordinarily valuable to trained musicians and to neophytes alike if it is extended in a number of useful ways.
Submitted June 28, 2017 at 06:10AM by davidlu https://www.reddit.com/r/edmproduction/comments/6jzexi/the_potential_of_the_music_interface_that_was/?utm_source=ifttt