Research projects
Multimodality and narrative and pragmatic abilities
The project of my PhD thesis aims to analyze the role that multimodality plays in the narrative and pragmatic development typically developing children and of children with Autism Spectrum Disorder (ASD) and Developmental Language Disorder (DLD). For the thesis, we will be carrying out different studies:
A correlational study in which we will evaluate the relationship between multimodal skills and linguistic, narrative and pragmatic skills in TD, ASD and DLD children (in collaboration with Prof. Courtenay Norbury and Dr. Mariia Pronina)
Two intervention studies that will experimentally test the value of a multimodal intervention (MultiModal Narrative) in promoting narrative and pragmatic development in a classroom context and in a speech-language therapy context.
The outcome of these studies will highlight the role of multimodality in development and its effects for promoting the development of oral narrative and pragmatic abilities.
In collaboration with:
Dr. Alfonso Igualada (UOC)
Dr. Pilar Prieto (ICREA/UPF)
Project supported by the Education Department (Generalitat de Catalunya)
Related research:
Florit-Pons, J. (in prep.). Multimodality as a tool for boosting narrative and pragmatic abilities in typical and non-typical language development [PhD thesis]. Universitat Pompeu Fabra. Directed by Pilar Prieto (ICREA-UPF) and Alfonso Igualada (UOC).
Florit-Pons, J., Igualada, A., & Prieto, P. (2022, July 13-15). The effects of an inclusive multimodal narrative-based intervention on narrative and pragmatic abilities in preschool children: A classroom investigation. Presentació oral al 9th Conference of the International Society for Gesture Studies (ISGS). University of Chicago, Chicago, IL, USA.
Florit-Pons, J., Prieto, P., & Igualada, A. (2022, July 13-15). Can multimodality boost narrative and pragmatic abilities of preschool ASD and DLD children? Evidences from a preliminary study. Poster presentation at the 9th Conference of the International Society for Gesture Studies (ISGS). University of Chicago, Chicago, IL, USA.
Florit-Pons, J., Prieto, P., & Igualada, A. (2021, September 20-22). A meta-analytic review of the effects of multimodal language on DLD children’s language development. Oral presentation at 1st International Developmental Language Disorder Research Conference. The DLD Project: Australia.
Videos related to this project
15-minute video presented at 1st International Developmental Language Disorder Research Conference
Improving Catalan use and Catalan phonology in preschoolers
The goal of this project is to determine whether the MultiModal Narrative (MMN) intervention can also help boost the use of Catalan language and improve Catalan phonology of Spanish-dominant preschoolers.
In collaboration with:
Mireia Gómez (University College Cork)
Claire Luong (UPF)
Alba Buendía
Dr. Josefina Carrera-Sabaté (UB)
Dr. Alfonso Igualada (UOC)
Dr. Pilar Prieto (ICREA/UPF)
The development of gesture, speech and information structure
This project is concerned about children's multimodal development, specifically on the temporal relationship between gesture and prosodic prominence and on how children use gesture to mark information structure in discourse.
In collaboration with:
Dr. Ingrid Vilà-Giménez (UdG/UPF)
Dr. Patrick Louis Rohrer (UPF/UN)
Sara Muñoz-Coego (UPF)
Dr. Pilar Prieto (ICREA/UPF)
Main findings:
The use of non-referential gestures increases significantly from ages 5-6 to 7-9.
By age five, children have acquired the ability to temporally integrate gesture and speech in narrative discourse.
Children use non-referential gestures to mark information that updates speaker's and listener's common ground (i.e., new referents, topic and comment constituents).
Related research:
Florit-Pons, J., Vilà-Giménez, I., Rohrer, P. L. & Prieto, P. (2023). Multimodal development in children’s narrative speech: Evidence for tight gesture-speech temporal alignment patterns as early as 5 years old. Journal of Speech, Language and Hearing Research, 66(3), 888-900. doi: 10.1044/2022_JSLHR-22-00451
Muñoz-Coego, S., Florit-Pons, J., Rohrer, P. L., Vilà-Giménez, I., & Prieto, P. (2022). The prosodic and gestural marking of information status in children's narrative speech: A longitudinal study. Proceedings of the 11th International Conference on Speech Prosody. Lisbon, Portugal.
Rohrer*, P. L., Florit-Pons*, J., Vilà-Giménez, I., & Prieto, P. (2022). Children use non-referential gestures in narrative speech to mark discourse elements which update common ground. Frontiers in Psychology, 12: 661339. doi: 10.3389/fpsyg.2021.661339
* Shared first authorship
Videos related to this project
20-minute video presented at XIIè Workshop sobre la prosòdia del català
10-minute video presented at Gesture and Speech in Interaction (GESPIN)
5-minute video presented at Budapest CEU Conference on Cognitive Development (BCCCD21)
Annotation of multimodal corpora
The team has developed the MultiModal MultiDimensional (M3D) labeling scheme for the annotation of multimodal corpora, which offers a dimensionalized approach to the annotation of communicative body movements (i.e., manual gestures, head movements, and other articulators) in terms of their form, their semantic and pragmatic contributions to speech, and their relationship to speech prosody.
In collaboration with:
Dr. Patrick Louis Rohrer (UPF/UN)
Dr. Ingrid Vilà-Giménez (UdG/UPF)
Dr. Núria Esteve-Gibert (UOC)
Ada Ren (MIT)
Dr. Stefanie Shattuck-Hufnagel (MIT)
Dr. Pilar Prieto (ICREA/UPF)
Related research:
Rohrer, P. L., Vilà-Giménez, I., Florit-Pons, J., Gibert, N. E., Ren, P., Shattuck-Hufnagel, S., & Prieto, P. (2021). The MultiModal MultiDimensional (M3D) labeling system. https://doi.org/10.17605/OSF.IO/ANKDX. Accessible at OSF
Rohrer, P. L., Vilà-Giménez, I., Florit-Pons, J., Esteve-Gibert, N., Ren, A., Shattuck-Hufnagel, S., and Prieto, P. (2020, September 7-9). The MultiModal MultiDimensional (M3D) labelling scheme for the annotation of audiovisual corpora. Oral presentation at Gesture and Speech in Interaction (GESPIN). KTH Speech, Music & Hearing and Språkbanken Tal: Stockholm, Sweden.
Videos related to this project
20-minute video presented at Gesture and Speech in Interaction (GESPIN)
The predictive role of pragmatic skills on language development
This longitudinal project's aim is to analyze whether early expressive pragmatic skills at 3-4 years of age can be predictive of later linguistic skills at 5-6.
In collaboration with:
Dr. Núria Esteve-Gibert (UOC)
Dr. Nadia Ahufinger (UOC)
Dr. Mariia Pronina (UPF)
Dr. Pilar Prieto (ICREA/UPF)
Related research:
Esteve-Gibert, N., Ahufinger, N., Pronina, M., Florit-Pons, J., & Prieto, P. (2023, July 9-14). Children’s linguistic abilities at 5-6 years of age can be predicted by expressive pragmatic skills at 3-4 years of age. Oral presentation at the 18th International Pragmatics Conference. Université Libre de Bruxelles: Brussels, Belgium.