SIEMPRE - Social Interaction Entertainment Using Music Performance

Publications

    In Process

  1. Labbé, C., Torres-Eliard, K. & Grandjean, D. (in prep). Dynamic judgments of perceived and felt emotions in music listening.
  2. Torres-Eliard, K. & Grandjean, D. (in prep). Dynamic judgments and temporal structure of the emotions expressed by music.


    3. Submitted

    Articles in press and submitted

  3. Badino, L., D’Ausilio, A., Fadiga, L. & Metta, G. (In Press.). Computational validation of the motor contribution to speech perception. Top Cogn Sci.
  4. Badino L., D’Ausilio A., Glowinski D., Camurri A. & Fadiga L. (submitted) Sensorimotor communication in professional quartets.
  5. Bartoli E., D’Ausilio A., Berry J., Badino L., Bever T. & Fadiga L. (submitted) Listener-speaker perceived distance predicts the degree of motor contribution to speech perception.
  6. Camurri, A., Dardard, F., Ghisio, S., Glowinski, D., Gnecco, G., Sanguineti, M., (submitted) Exploiting the Shapley value to estimate the position of a point of interest for a group of individuals.
  7. Canevari C., Badino L., D’Ausilio A., Fadiga L. & Metta G. (submitted) Modeling speech imitation and ecological learning of auditory-motor maps.
  8. Clerget E., Andres M., Winderickx A., D’Ausilio A., Fadiga L. & Olivier E. (submitted) The role of left BA44 in recognition of composite biological actions.
  9. D’Ausilio A., Bartoli E., Maffongelli L., Berry J. & Fadiga L. (submitted) Ultrasound images of tongue movement bias auditory speech perception.
  10. Glowinski, D., Mancini, Cowie, R., Chiorri, C., Doherty, C. & Camurri, A. (submitted) The movements made by performers in a skilled quartet: a distinctive pattern, and the function that it serves.
  11. Gnecco. G., Glowinski, D. & Camurri, A. (submitted) On the Detection of the level of attention in an orchestra through head movements.
  12. Labbé, C. & Grandjean, D. (submitted) Self-reported motor and visceral entrainment predicts different feelings during music listening.

    13. Submitted Conference papers

  13. Cowie, R., Sawey, M., Doherty, C., Jaimovich, J., Fyans, C., Stapleton, P. & Knapp, B. (submitted). GTrace: General trace program compatible with EmotionML. Affectice Computing and Intelligent Interaction Conference, Geneva.
  14. Doherty, C., Cowie, R., Fyans, C., Jaimovich, J., Knapp, B., Ortiz-Perez, M. & Stapleton, P. (submitted). Development of a quality of experience questionnaire for use during live music performances. Frontiers in Psychology.
  15. Maestre, E., Scavone, G. P., & Smith III, J. O. (submitted). Digital modeling of bridge driving-point admittance from measurements on violin-family instruments. Stockholm Music Acoustics Conference.
  16. Mayor, O., Llimona, Q., Marchini, M., Papiotis, P. & Maestre, E. (submitted). repoVizz: a framework for remote storage, browsing, annotation, and exchange of multi-modal data. 21st ACM international conference on Multimedia, Barcelona.
  17. Ott., T., Eliard., K., Glowinski, D., Baron, N. & Grandjean, D. (submitted) Analysis of violinists' expressive gestures, Sixth International Conference of Students of Systematic Musicology (SysMus13).
  18. Riolfo, A., Glowinski, D. & Chiorri, C. (submitted) Can you distinguish between a musician playing alone or within an ensemble?”, Sixth International Conference of Students of Systematic Musicology (SysMus13).
  19. Rovegno, R. & Glowinski, D. (submitted) Multiple 3D density maps to visualize musician expressive behaviour, Sixth International Conference of Students of Systematic Musicology (SysMus13).

    20. Published journal articles

  20. Camurri, A., Nijholt, A., Reidsma, D., Volpe, v. (2012). Special Issue of Entertainment Computing Intl Journal, Elsevier (In Press).
  21. Camurri, A., Costa, C., Volpe, G., (2011) Proc. Intl Conf INTETAIN 2011 - 4th International ICST Conference on Intelligent Technologies for Interactive Entertainment, In cooperation with ACM – SigCHI, Genoa.
  22. Elsner, C., D’Ausilio, A., Gredebäck, G., Falck-Ytter, T. & Fadiga, L. (2013). The motor cortex is causally related to predictive eye movements during action observation. Neuropsychologia. 51(3):488-492.
  23. D’Ausilio, A., Bufalari, I., Salmas, P. & Fadiga, L. (2012) The role of the motor system in discriminating degraded speech sounds. Cortex, 48(7), 882–887. PDF
  24. D'Ausilio, A., Badino, L., Tokay, S., Aloimonos, Y., Li, Y., Craighero, L., Canto, R. & Fadiga, L. (2012) Leadership in Orchestra Emerges from the Causal Relationships of Movement Kinematics. PLoS ONE 7(5): e35757. 
  25. D'Ausilio A., Bufalari I., Salmas P., Busan P., Fadiga L., (2011) Vocal pitch discrimination in the motor system. Brain & Lang. 118, 9-14.
  26. Fadiga L., D'Ausilio A. (2011). From action to language: evidence and speculations. M. Sanz, I. Laka and M. K. Tanenhaus Eds. The Cognitive and Biological Basis for Linguistic Structure: New approaches and enduring Themes Oxfor University Press: Oxford, UK.
  27. Watanabe, T. & Miwa, Y. (2012) Duality of Embodiment and Support for Co-creation in Hand Contact Improvisation. Journal of Advanced Mechanical Design, System, and Manufacturing, Vol.6(7), 1307-1318.

    28. Published conference papers

  28. Badino, L., Camurri, A., D’Ausilio, A., Fadiga, L., Glowinski, D., Gnecco, G., Sanguineti, G., Varni, G. & Volpe, G. (2013) Analysis of music ensemble performance as a test-bed for social interaction: Methods from Operations Research and preliminary results. AIRO Winter 2013 Conference (Operational Research for Development, Sustainability and Local Economies), Champoluc, Aosta, 28 Jan – 1 Feb.
  29. Bartoli, E., D'Ausilio, A. & Fadiga, L. (2012) The role of the inter-speaker articulatory variability in the motor contribution to speech perception: a TMS-MDS study. Mirror neurons: new frontiers 20 years after their discovery. Erice, Aug 31 - Sept 6.
  30. Bartoli, E., Maffongelli, L., D’Ausilio, A. & Jacono, M. (2012) Fadiga L. Affordance encoding in the motor cortex: a state-dependent TMS study. SIPF 2012 (Società Italiana di PsicoFisiologia). Venezia, 23-24 Nov.
  31. B. Bortz, S. Salazar, J. Jaimovich, R. B. Knapp, G. Wang (2012) ShEMP: A Mobile Framework for Shared Emotion, Music, and Physiology, ACM-ICMI SBM workshop, Santa Monica, California, USA. PDF
  32. Camurri, A. & Glowinski, D. (2013) Automated measure of non-verbal expressive gesture and social signals in joint music actions, International symposium “Music, gestures and emotions”, University of Geneva, June 2013.
  33. Cowie, R., Doherty, C., Fyans, C., Jaimovich, J., Knapp, B., Ortiz-Perez, M. & Stapleton, P. (2013). Measuring audience experience in a live music environment. In Collaborate to Innovate Conference Access Horizon 2020, Dublin
  34. D’Ausilio, A., Bartoli, E., Berry, J., Maffongelli, L. & Fadiga L. (2012) Studio sulla mediazione sensori-motoria nella genesi dell'effetto McGurk. Congresso annuale SINP (Società Italiana di Neuropsicologia). Rome, 9-10 Nov.
  35. D’Ausilio, A., Bartoli, E., Maffongelli, L., Berry, J. & Fadiga, L. (2012) McGurk effects require a motor coding. SIPF 2012 (Società Italiana di PsicoFisiologia). Venezia, 23-24 Nov.
  36. D’Ausilio, A. & Fadiga, L. (2012) The role of the motor system in decoding speech. ICP2012 (International Congress of Psychology), Cape Town, 22-27 Jul.
    37. < li> D'Ausilio, ., Badino, L., li, Yi., Tokay, S., Craighero, L., Canto, R., loimonos, Y., Fadiga, L. Communication in orchestra playing as measured with Granger Causality. Intetain 2011 (Intelligent Technologies for Interactive Entertainment), Genova, 25-27 May. Also in: A. Camurri, C. Costa, and G. Volpe (Eds.): INTETAIN 2011, LNICST 78, pp. 273-275, 2012.
  37. Delogu, F., Brunetti, R., D'Ausilio, A., Del Gatto, C. & Del Grosso, E. (2012) L'integrazione audio-motoria nei pianisti: un approfondimento EEG degli effetti di facilitazione e inibizione. Musical (E)motion, Rome, 23 Apr.
  38. Doherty, C. & Cowie, R., (2013). Assessing the quality of experience of an audience in a live musical scenario. 3rd international conference on Music & Emotion, Jyväskylä.
  39. Finisguerra, A., Maffongelli, L., Bassolino, M., Jacono, M., Pozzo, T., Fadiga, L. & D’Ausilio, A. (2012) Can mirror-Like properties generalize across effectors? A TMS study. SIPF 2012 (Società Italiana di PsicoFisiologia). Venezia, 23-24 Nov.
  40. Glowinski, D., Gnecco, G., Piano, S. & Camurri. A. (2013) Expressive Non-Verbal Interaction in String Quartet, Conference on Affective Computing and Intelligent Interaction (ACII 2013).
  41. Glowinski, D., Mancini, M., Camurri. A & Cowie, R. (2013) How action adapts to social context: the movements of musicians in solo and ensemble conditions, Intl Conference on Affective Computing and Intelligent Interaction (ACII 2013). PDF
  42. Glowinski, D., Chiorri, C., Riolfo, A., Torres-Eliard, K., Grandjean, D., & Camurri, A. (2013) Evaluating emotional expression in music context: the contribution of perceived social context and expertise, International Society for Research on Emotion meeting (ISRE2013).
  43. D. Glowinski, L.Badino A.D'Ausilio, A.Camurri, L.Fadiga (2012) Analysis of Leadership in a String Quartet, ACM-ICMI SBM workshop, Santa Monica, California, USA. PDF
  44. D.Glowinski, K.Torres-Eliard, C.Chiorri, A.Camurri, D.Grandjean (2012) Can naïve observers distinguish a violinist's solo from an ensemble performance? A pilot study", ACM-ICMI SBM workshop, Santa Monica, California, USA. PDF
  45. D. Glowinski, M. Mancini, N. Rukavishnikova, V. Khomenko, A. Camurri, (2011) Analysis of Dominance in Small Music Ensemble. Proceedings of the AFFINE satellite workshop of the ACM ICMI 2011 Conference, Alicante, Spain PDF
  46. Glowinski, D., Camurri, A., Chiorri, C., Coletta, P., Schenone, A., Volpe G. (2010). Multi-Scale Entropy Analysis of Dominance in Social Creative Activities, Proc ACM Multimedia Intl Conference, Firenze, ACM Press PDF
  47. Harding E., Sammler D., D'Ausilio A., Fadiga L., Koelsch S. (2010) Music and Action: Do they share neural resources? ICMPC (International Conference of Music Perception and Cognition), Seattle, 23-27 Aug.
  48. Hayashi, N., Miwa, Y., Itai, S. & Nishi, H. (2013) Bodily Expression Media by Dual Domain Design of Shadow, HIMI/HCII 2013, Part III, LNCS 8018, pp. 195-202, 2013
  49. Itai, S., Endo, Y. & Miwa, Y. (2013) Design of Space for Expression Media with the Use of Fog, HIMI/HCII 2013, Part III, LNCS 8018, pp. 210-218, 2013
  50. Jaimovich, J., Coghlan, N., Knapp, R. B, & Bortz, B. (2013) "Music & Emotion Listening Experiments: design, implementation & perspectives", International Conference on Music and Emotion (ICME), Jyväskylä, Finland.
  51. Jaimovich, J. & Knapp, R.B. (2013) "Creating Biosignal Algorithms for Musical Applications from an Extensive Physiological Database", submitted to the 5th biannual Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII), Geneva, Switzerland.
  52. Jaimovich, Javier, Niall Coghlan, and R. Benjamin Knapp (2012) Emotion in Motion: A Study of Music and Affective Response. In Proceedings of the 9th International Symposium on Computer Music Modeling and Retrieval (CMMR) Music and Emotions, pp.29-44. Queen Mary University of London. PDF
  53. Jaimovich, Javier, Miguel Ortiz, Niall Coghlan, and R. Benjamin Knapp (2012) The Emotion in Motion Experiment: Using an Interactive Installation as a Means for Understanding Emotional Response to Music. In Proceedings of the 2012 Conference on New Interfaces for Musical Expression (NIME), pp.457-8. University of Michigan, Ann Arbor: Division of Computer Science & Engineering, University of Michigan. PDF
  54. Jaimovich, J., Coghlan, N. & Knapp, R.B., 2012. Emotion in Motion: A Study of Music and Affective Response. In Proceedings of the 9th International Symposium on Computer Music Modeling and Retrieval (CMMR) Music and Emotions. Symposium on Computer Music Modeling and Retrieval. London, England, p. (In Press).
  55. Jaimovich, Javier, Nicholas Gillian, Paolo Coletta, Esteban Maestre, and Marco Marchini (2011). “Workshop on Multi-Modal Data Acquisition for Musical Research.” In *Proceedings of the International Conference on New Interfaces for Musical Expression*, 551. Oslo, Norway.
  56. Maestre, E., Scavone, G. P., Smith III, J. O. (2011) Modeling of a violin input admittance by direct positioning of second-order resonators. 162nd Meeting of the Acoustical Society of America (ASA).
  57. Maestre, E. (2011) Synthesis of bowing controls applied to violin sound generation. 162nd Meeting of the Acoustical Society of America (ASA).
  58. Maestre, E., Pérez, A., Ramírez, R. (2010). Gesture sampling for instrumental sound synthesis: violin bowing as a case study.International Computer Music Conference.
  59. Maestre, E., Ramirez, R. (2010). An approach to predicting bowing control parameter contours in violin performance. Intelligent Data Analysis. 14(5), 587-599.
  60. Marchini, M., Papiotis, P., Maestre, E. (2012) Timing synchronization in string quartet performance: a preliminary study. International Workshop on Computer Music Modeling and Retrieval (CMMR12), London, UK (2012)
  61. Marchini, M., Papiotis, P. & Maestre, E. (2013). Investigating the relationship between expressivity and synchronization in ensemble performance: an exploratory study. In International Symposium on Performance Science, Vienna.PDF
  62. Marchini, M., Ramirez, R., Papiotis, P., & Maestre, E. (2013). Inducing rules of ensemble music performance: a machine learning approach. In 3rd international conference on Music & Emotion, Jyväskylä.PDF
  63. Mayor, O., Llop, J., Maestre, E. (2011) RepoVizz: A multi-modal on-line database and browsing tool for music performance research. International Symposium on Music Information Retrieval (ISMIR), Miami. http://mtg.upf.edu/system/files/publications/RepoVizz_ISMIR_2011_v5.pdf
  64. Miwa, Y., Itai, S., Watanabe, T. & Nishi, H. (2013) Generation Dynamics of Sympathetic Embodied Awareness in Hand Contact Improvisation, Proceeding of IASDR 2013, 2013
  65. Panagiotis Papiotis, Marco Marchini, Alfonso Peréz, and Esteban Maestre (2012) Measuring music ensemble synchrony in a string quartet through analysis of intonation adjustments. International Journal of Arts and Technology (IJART), submitted.
  66. Panagiotis Papiotis, M. Marchini, E. Maestre, and A. Perez. Measuring ensemble synchrony through violin performance parameters: a preliminary progress report. In Proceedings of the 4th International ICST Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN 2011), Genoa, Italy (2011) PDF
  67. Panagiotis Papiotis, Marco Marchini, and Esteban Maestre (2012) Computational analysis of solo versus ensemble performance in string quartets: Dynamics and Intonation. In Proceedings of the 12th International Conference of Music Perception and Cognition (ICMPC12), Thessaloniki, Greece. PDF
  68. Papiotis, P., Herrera, P., Marchini, M., & Maestre, E. (2013). Aural-based detection and assessment of real versus artifically synchronized string quartet performance. In 3rd international conference on Music & Emotion, Jyväskylä.PDF
  69. Papiotis, P., Marchini, M., & Maestre, E. (2013). Multidimensional analysis of interdependence in a string quartet. In International Symposium on Performance Science, Vienna.PDF
  70. Papiotis, P., Marchini, M., Pérez, A., Maestre, E., Andrzejak, R. (2011) Synchronization of intonation adjustments in violin duets: towards an objective evaluation of musical interaction. International Conference on Digital Audio Effects (DAFX), Paris, France. http://recherche.ircam.fr/pub/dafx11/Papers/83_e.pdf
  71. Pérez, A., Bonada, J., Maestre, E., Blaauw, M., Guaus, E. (2012) Performance Control driven Violin Timbre Model based on Neural Networks. IEEE Transactions on Audio, Speech, and Language Processing, Vol. 20, No. 3.
  72. Rovegno, R., Gagliardi, M., Glowinski, D., Ghisio, S., Coletta, P., and Camurri, A., Using the Qualisys MoCap System for Multimodal analysis of string quartet performance: a feasibility study, Intl ISSSM2011, University of Jyvaskyla (Finland), 8-18 August 2011.
  73. Torres-Eliard, K., Labbé, C., & Grandjean D. (2011). Towards a Dynamic Approach to the Study of Emotions Expressed by Music. Proceedings 4th International ICST Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN 2011), Genoa, Italy, May 25-27, 2011, LNICST 78, A. Camurri, C. Costa, and G. Volpe (Eds.), Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, 2012, 252-259 PDF

SIEMPRE Project Public Deliverables

  1. D1.1 Research requirements
  2. D2.1 First series of experiments
  3. D3.1 Initial version of techniques for data acquisitionand multimodal analysis of eMAP signals - Appendix I The SIEMPRE Library
  4. D4.1 Results from the first series of experiments and first evaluation report
  5. D1.3 Models and algorithms for analysis of creative social interaction (ver.1)
  6. D2.2 Second series of experiments