Guest blog by Tuomas Eerola

How is emotional expression initially conveyed by music? Primarily through the colour of the sound, known as timbre, before other musical parameters such as tempo, harmony, pitch, and structure have a chance to impart their stamp on the expression. Musical instruments differ radically in timbre despite them being able to play the same pitch and loudness. The timbre differences may be characterised in terms of adjectives such as bright, warm, nasal, or harsh, and these have a tangible connection to emotional expression of music.

Timbre conveys an immediate impression about the identity of the sound to the listener. Listeners are able to make a distinction between neutral and emotional expression (Filipic et al., 2010), determine the genre of music (Gjerdingen & Perrot, 2008) in a quarter of a second (examples below), and, estimate reliably whether the sound is vocal or not from tiniest (8 ms, that is 1/125th of a second!) of fragments (Suid et al., 2014).

To get a sense of this, feel free to listen few excerpts from Gjerdingen and Perrot's classic study:

We infer softness, warmth, movement and size, energy, and even possible hostility about the sound source. Some of these properties describe the physical aspects of the sound, but also about the social intentions or states. This idea is related to our own body-states in affective experiences: Pleasant affective states are reflected in faucal and pharyngeal expansion, which is physiological jargon for wide relaxed voice. This manifests in relatively more low- than high-frequency energy. The high-arousal emotions such as anger is related to an increase in high-frequency energy and its variability. In fact, these couplings are even partly shared by some primates. Hence, the mapping between acoustic properties and emotional expression is rooted in biology.

The past research has explored the good, bad and ugly aspects of sounds. My personal favorite in this line of work is titled "Scraping sounds and disgusting noises" by Trevor Cox. He had over million people rating a small selection of horrible sounds, including vomiting, microphone feedback and the noise from many babies crying at the same time, which were actually the worst sounds in the selection.

In a musical domain, disgusting sounds are not typically sought after but a broad palette of sound colours are of course utilized. In my work on emotions and music, I wanted to know whether musical instruments themselves and their sounds carry clear affective associations. To explore this, I got hold of 1-second long instrument sounds that were identical in frequency, loudness and duration, asked listeners to evaluate their affective qualities. They rated a reedy saxophone [example below] to be tense and energetic, a guitar pluck as pleasant and relaxed and so on. The affective qualities of all clips could then be directly connected to the acoustic qualities of the sounds.

This experiment was simply scratching the surface of the possibilities that timbre can offer to research of emotions in music. Timbre talks to us in a very direct fashion about the affective qualities of other beings. Composers – from Wagner, Scriabin, Messiaen, to Saariaho, and Lachenmann – have been keen to exploit the possibilities of timbre, first with sophisticated orchestration techniques and later with the aid analog and digital technology. In research, we are catching up the ideas of the great composers by turning more attention to the intriguing ways timbre is able to convey affects, sensations and associations. The new collaboration between artist Adinda van ‘t Klooster (Leverhulme sponsored Artist in Residence) and music scholars at Durham University (Collins and Eerola) offers a great opportunity to explore these connections simultaneously from artistic and empirically-grounded scientific vantage points.

References

• Cox, T. J. (2008). Scraping sounds and disgusting noises. Applied Acoustics, 69(12), 1195-1204.
• Eerola, T., Ferrer, R., & Alluri, V. (2012). Timbre and affect dimensions: Evidence from affect and similarity ratings and acoustic correlates of isolated instrument sounds. Music Perception, 30(1), 49-70.
• Filipic, S., Tillmann, B., & Bigand, E. (2010). Judging familiarity and emotion from very brief musical excerpts. Psychonomic Bulletin & Review, 17(3), 335-341.
• Gjerdingen, R. O. & Perrott, D. (2008). Scanning the dial: The rapid recognition of music genres. Journal of New Music Research, 37(2), 93-100.
• Suied, C., Agus, T. R., Thorpe, S. J., Mesgarani, N., & Pressnitzer, D. (2014). Auditory gist: Recognition of very short sounds from timbre cues. The Journal of the Acoustical Society of America, 135(3), 1380-1391.