2021

Retaining Pianistic Virtuosity in #MIs: Exploring Pre-Existing Gestural Nuances for Live Sound Modulation through a Comparative Study. (Book Chapter)

Authors

Niccolò Granieri, James Dooley and Tychonas Michailidis

Abstract

This paper focuses on Reach, a keyboard-based gesture recognition system for live piano sound modulation, and the comparative user testing conducted to evaluate it. Reach is a system built using the Leap Motion Orion SDK, a custom C++ OSC mapper and a Pure Data environment. It provides control over the sound modulation of a live piano feed, taking advantage of pre-existing gestural nuances offering a touch-free experience to the pianist.

The user testing compared the Reach system with two commercially available keyboard-based systems for augmented live sound modulation: Seaboard and TouchKeys. The approach taken during the user tests is illustrated and test results are discussed. The results that emerged suggest an underlying importance of recognising and utilising the musician’s existing technique when designing Digital and Augmented Musical Instruments (#MIs), and the potential of reducing the requirement to learn additional instrumental technique. The comparative user testing discussed in this paper is part of a larger research project that seeks to study and understand how a low degree of invasiveness in digital systems for live sound modulation can reduce the learning curve of new systems, allowing greater access to music making with technology.

Downloads

BibTex

@inbook{GranieriDooleyMichailidis:2021,
    address = {London},
    author = {Granieri, Niccolò and Dooley, James and Michailidis, Tychonas},
    booktitle = {Innovation in Music. Future Opportunities},
    pages = {-},
    title = {Retaining Pianistic Virtuosity in #MIs: Exploring Pre-Existing Gestural Nuances for Live Sound Modulation through a Comparative Study},
    year = {2021}}

2020

Augmenting the experience of playing the piano: controlling audio processing through ancillary gestures. (PHD Dissertation)

Authors

Niccolò Granieri

Abstract

Pianists spend many years practicing on their instrument. As a result they develop alongside their pianistic technique a set of gestural nuances that enable them to perform expressively and establish their own acoustic signature on the piano. This \textit{mute layer} of nuanced gestures is rarely taken into consideration when developing new keyboard-based gestural interfaces. These often usually require new gestural vocabularies to be learned resulting in a disruptive experience for the pianist. The main objective of this research is to investigate how new keyboard-based gestural interfaces can enable musicians to control and transform live piano sound through the gestural nuances embedded in their technique. Specifically, how keyboard interfaces with nuanced gestural control can extend the creative possibilities available to classically trained pianists, thus stimulating new approaches to build intuitive interfaces for musical expression, and new ways of learning and playing digital instruments. Towards this goal, interviews, user tests and case studies were conducted with a range of pianists coming from different musical backgrounds, and Reach, an augmented instrument for live sound modulation controlled by gestural nuances embedded in the pianistic technique was developed.

Downloads

BibTex

@phdthesis{Granieri:2020,
    author = {Granieri, Niccolò},
    title = {Augmenting the experience of playing of the piano: controlling audio processing through ancillary gestures.},
    school = {Royal Birmingham Conservatoire, Birmingham City University},
    year = {2020}}
NIME Publication Ecosystem Workshop. (Workshop)

Authors

Alexander Refsum Jensenius, Andrew McPherson, Anna Xambó Sedó, Charles Patrick Martin, Jack Armitage, Niccolò Granieri, Rebecca Fiebrink and Luiz Naveda

Abstract

How can we develop an open, future-oriented, multimedia-rich, and institutionally recognised publication ecosystem for NIME practitioners and researchers? This workshop will continue previous discussions about the need for a NIME journal, and for solutions to share ideas, hardware designs, code, scores, and performances, systematically. Concerns about C19, climate change, and accessibility make these discussions urgent and demand reimagining our expectations for a publication venue. What solutions can we start implementing right away, and which goals do we have as a community? This open workshop will lay the ground for concrete experimentation in the year(s) to come.

Downloads

-

BibTex

@inproceedings{JenseniusMcPhersonXamboEtAl:2020,
    address = {Birmingham},
    author = {Jensenius, Alexander Refsum and McPherson, Andrew and Xambó Sedó, Anna and Martin, Charles Patrick and Armitage, Jack and Granieri, Niccolò and Fiebrink, Rebecca and Naveda, Luiz},
    booktitle = {New Interfaces for Musical Expression},
    pages = {-},
    title = {NIME Publication Ecosystem Workshop},
    year = {2020}}

2019

Microgestural implementation for the creation of an expressive keyboard interface. (Book Chapter)

Authors

Niccolò Granieri, Tychonas Michailidis and James Dooley

Abstract

Musicians spend a great deal of time practising their instrument. As a result, they develop a unique set of microgestures that define their personal sound: their acoustic signature. This personal palette of gestures has been identified as one of the most distinctive aspects of piano playing and varies from musician to musician, making their sound unique and enabling them to expressively convey their music.

By using radar millimetre waves to capture micromotions and microgestures, it is possible to achieve a high level of expression without the need to modify the keyboard instrument itself or requiring additional technique. The aim of this research is to build on existing instrumental technique and remove the steep learning curve typical found when performing digital or augmented musical instruments. This approach enables the pianist to retain and focus on his or her technical control and musical freedom resulting in a less disruptive experience.

The paper describes through the implementation of microgestural sound control, how performers can gain a wide control over digital sound processing through their existing technique. The study is also meant to identify which musicians will mostly benefit from the interface analysing their musical background, level of expertise on the instrument, familiarity with digital instruments and music environments.

Downloads

BibTex

@inbook{GranieriMichailidisDooley:2019,
    address = {London},
    author = {Granieri, Niccolò and Michailidis, Tychonas and Dooley, James},
    booktitle = {Innovation in Music. Performance, Production, Technology, and Business},
    pages = {269-282},
    title = {Harnessing Ancillary Microgestures in Piano Technique. Implementing Microgestural Control Into an Expressive Keyboard-Based Hyper-Instrument},
    year = {2019}}
Reach: a keyboard-based gesture recognition system for live piano sound modulation. (Demo Paper)

Authors

Niccolò Granieri, James Dooley

Abstract

This paper presents Reach, a keyboard-based gesture recognition system for live piano sound modulation. Reach is a system built using the Leap Motion Orion SDK, Pure Data and a custom C++ OSC mapper\footnote{\url{https://github.com/NiccoloGranieri/Reach}}. It provides control over the sound modulation of an acoustic piano using the pianist's ancillary gestures.

The system was developed using an iterative design process, incorporating research findings from two user studies and several case studies. The results that emerged show the potential of recognising and utilising the pianist's existing technique when designing keyboard-based DMIs, reducing the requirement to learn additional techniques.

Downloads

BibTex

@inproceedings{GranieriDooley2019,
    author = {Granieri, Niccolò and Dooley, James},
    title = {Reach: a keyboard-based gesture recognition system for live piano sound modulation},
    pages = {375--376},
    booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
    editor = {Queiroz, Marcelo and Sedó, Anna Xambó},
    year = {2019},
    month = jun,
    publisher = {UFRGS},
    address = {Porto Alegre, Brazil},
    issn = {2220-4806},
    doi = {10.5281/zenodo.3673000},
    url = {http://www.nime.org/proceedings/2019/nime2019_paper072.pdf}}

2018

Reach – Designing Keyboard Instruments with pianists in mind. (Poster)

Authors

Niccolò Granieri

Abstract

This poster focuses on the comparative user testing conducted to evaluate Reach, a gesture recognition system for live piano sound modulation. The user testing compares the Reach system with two existing keyboard-based systems for keyboard live sound modulation: ROLI Seaboard (Lamb and Robertson, 2011) and TouchKeys (McPherson, 2012). The study analyses ease of use, learnability and creative freedom based on two jazz improvisations each on all three systems by the participants. This is presented along with user experience questionnaire (UEQ) data. The poster illustrates results from the test, focusing on the relationship between the learning curve and creative barrier in digital instruments and showing promising results for touch-free digital musical instruments (DMIs) like Reach.

The comparative user testing taken into analysis is part of a larger research project that seeks to investigate how a low degree of invasiveness in digital systems for live sound modulation can reduce the learning curve and eventually make electronic music more accessible.

Downloads

BibTex

@unpublished{Granieri2018,
    author = {Granieri, Niccolò},
    title = {Reach – Designing Keyboard Instruments with pianists in mind.},
    booktitle = {Poster presented at the Sound, Image and Interaction Design Symposium},
    year = {2018},
    month = sept,
    address = {Madeira, Portugal}}
Improvising through the senses: a performance approach with the indirect use of technology. (Journal Article)

Authors

Tychonas Michailidis, James Dooley, Niccolò Granieri and Balandino Di Donato

Abstract

This article explores and proposes new ways of performing in a technology-mediated environment. We present a case study that examines feedback loop relationships between a dancer and a pianist. Rather than using data from sensor technologies to directly control and affect musical parameters, we captured data from a dancer’s arm movements and mapped them onto a bespoke device that stimulates the pianist’s tactile sense through vibrations. The pianist identifies and interprets the tactile sensory experience, with his improvised performance responding to the changes in haptic information received. Our system presents a new way of technology-mediated performer interaction through tactile feedback channels, enabling the user to establish new creative pathways. We present a classification of vibrotactile interaction as means of communication, and we conclude how users experience multi-point vibrotactile feedback as one holistic experience rather than a collection of discrete feedback points.

Downloads

BibTex

@article{MichailidisDooleyGranieriEtAl:2018,
    author = {Tychonas Michailidis and James Dooley and Niccolò Granieri and Balandino Di Donato},
    title = {Improvising through the senses: a performance approach with the indirect use of technology},
    journal = {Digital Creativity},
    volume = {29},
    number = {2-3},
    pages = {149-164},
    year  = {2018},
    publisher = {Routledge},
    doi = {10.1080/14626268.2018.1511600},
    url = {https://doi.org/10.1080/14626268.2018.1511600},
    eprint = {https://doi.org/10.1080/14626268.2018.1511600}}

2017

From piano to piano. (Poster)

Authors

Niccolò Granieri

Downloads

BibTex

@unpublished{Granieri2017,
    author = {Granieri, Niccolò},
    title = {from piano, to piano},
    booktitle = {Poster presented at the Research Conference},
    year = {2017},
    month = may,
    address = {Birmingham, United Kingdom}}
Expressing through gesture nuances: Bridging the analog and digital divide. (Performance)

Authors

Niccolò Granieri

Abstract

This piano performance has been composed to explore bridging the gap between acoustic instruments and the digital world. The audience will be placed in front of a musician that is stripped at first of all his human traits and gestural capabilities, being forced to play the instrument through machine like objects and movements. Wooden sticks will be used to strike the piano keys, making the act of playing mechanical, binary. Throughout the short piece, he will slowly regain control over all of his musical gestures, abandoning the objects that constrained him, finding a different instrument in front of him, one that transcends the classical concept of a piano.

He will explore this new instrument and slowly realise that his technique is being enhanced by the instrument itself and the explorable sound landscape is much more vast than he thought. The sound coming from the piano will be processed and effected following the pianists sound-accompanying gestures: what is usually made in response to the sound, in this case becomes responsible for the sound itself.

The steep learning curve on digital interfaces, often poses a creative barrier to musical creativity. New digital interfaces require years of practice to attain a certain fluency, thus pushing away instrumentalists that have spent a lifetime perfecting their own instrument and technique. The border between these two worlds is clear, and one that this research aims to dissolve.

The goal is to create an interface that takes advantage of the gestures and technique of classically trained pianists, and enhances the sound possibilities of the instrument throughout a non-invasive technology. The performance is meant to make the audience question if technology could actually enhance a performance without being obtrusive both to the audience itself and to the musician.

Downloads

-