2.png){kind=logo}
Email this article How word-image relationships in children’s picture books adapt across levels: A multimodal analysis
Bernadette Kushartanti, Universitas Indonesia, Indonesia
Dwi Purwanto, University of Colorado Boulder, United States
Abstract
Images in picture books are not solely decorative elements, nor are the words narrowly for narration; their relationship is way more complex. This complexity has been extensively explored, but there remains a gap to investigate whether the complexity adapts across levels. Applying Halliday’s systematic functional linguistics (2004), Kress and Leeuwen’s social semiotic approach (2021), and Nikolajeva and Scott’s approach to picture books (2006), this research qualitatively and quantitatively investigated thirty-four text-image pairs from two differently-levelled children’s picture books for complexity changes. The results confirmed that a higher-levelled picture book demonstrated broader types of text-image relationships compared to a lower-levelled one. These results highlight the need for authors and educators to design or select picture books that progressively introduce multimodal complexity. Future studies are encouraged to expand samples of picture books with a more rigorous criterion.
Keywords
References
Al-Yaqout, G., & Nikolajeva, M. (2015). Re-conceptualising picture book theory in the digital age. Barnelitterært Forskningstidsskrift, 6(1). https://doi.org/10.3402/blft.v6.26971
Badan Standar, Kurikulum, dan Asesmen Pendidikan. (2022). Pedoman Perjenjangan Buku. Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi. https://buku.kemdikbud.go.id/
Bateman, John A. (2014). Text and Image: A critical introduction to the visual/verbal divide. Routledge.
Benjamin, R. G. (2011). Reconstructing readability: Recent developments and recommendations in the analysis of text difficulty. Educational Psychology Review, 24, 63–88. doi:10.1007/s10648-011-9181-8.
Callow, J. (2020). Visual and verbal intersections in picture books–multimodal assessment for middle years students. Language and Education, 34(2), 115–134. https://doi.org/10.1080/09500782.2019.1689996
Cresswell, John W., & Poth, Cheryl N. (2018). Qualitative inquiry research design: Choosing among five approaches (fourth edition). SAGE.
Damayanti, I. L., Moecharam, N. Y., & Asyifa, F. (2021). Unfolding layers of meanings: Visual-verbal relations in Just Ask—a children’s picture book. Indonesian Journal of Applied Linguistics, 11(2), 372-381. https://doi.org/10.17509/ijal.v11i2.39195
Edmonds, W. Alex., & Kennedy, Thomas D. (2017). An applied guide to research designs: quantitative, qualitative, and mixed methods. SAGE.
Elmiana, D. S., & Shen, S. (2024). Let pictures explain the world: text–image narration in picture books. Journal of Research in Childhood Education. https://doi.org/10.1080/02568543.2024.2367406
Green, C. (2025). Multi-word expressions occur infrequently in picturebooks: implications for early vocabulary instruction. Literacy Research and Instruction, 1–18. https://remote-lib.ui.ac.id:2075/10.1080/19388071.2025.2469068
Green, C., Keogh, K., Sun, H., & O’Brien, B. (2024). The children’s picture books lexicon (CPB-LEX):A large-scale lexical database from children’s picture books. Behavior Research Methods, 56(5),4504–4521. https://doi.org/10.3758/s13428-023-02198-y
Guijarro, Arsenio Jesus Moya, & Sanz, Maria Jesus Pinar. (2008). Compositional, interpersonal and representational meanings in a children’s narrative: a multimodal discourse analysis. Journal of Pragmatics, 40(9), 1601–1619. https://doi.org/10.1016/j.pragma.2008.04.019
Halliday, M. A. K., Matthiessen, Christian M. I. M. (2014). Halliday’s introduction to functional grammar: fourth edition. Routledge.
Haris, N. A., Febrianti, Y., & Yannuar, N. (2023). Exploring augmentation of meaning through intersemiotic complementarity in children comic book series. Indonesian Journal of Applied Linguistics, 12(3), 739-751. https://doi.org/10.17509/ijal.v12i3.39951
Hu, Y., & Qiu, Q. (2020). A study on verbal and image relations in multimodal texts from the perspective of intersemiotic complementarity. Canadian Social Science, 16(10), 50–56. http://doi.org/10.3968/11938
Jian, Y. C. (2022). Using an eye tracker to examine the effect of prior knowledge on reading processes while reading a printed scientific text with multiple representations. International Journal of Science Education, 44(8), 1209–1229. https://doi.org/10.1080/09500693.2022.2072013
Karatza, Styliani. (2020). Multimodal literacy and language testing: Visual and intersemiotic literacy indicators of reading comprehension texts, Journal of Visual Literacy, 39:3-4, 220-255, DOI: 10.1080/1051144X.2020.1826222
Kelly, Laura Beth, & Kachorsky, Dani. (2022). Text complexity and picturebooks: learning from multimodal analysis and children’s discussion. Reading & Writing Quarterly, 38:1, 33-50, DOI: 10.1080/10573569.2021.1907636
Koutsikou, M., Christidou, V., Papadopoulou, M., & Bonoti, F. (2021). Interpersonal meaning: Verbal text–image relations in multimodal science texts for young children. Education Sciences. 11, 245. https://doi.org/10.3390/educsci11050245
Kress, Gunther, & Leeuwen, van Theo. (2021). Reading images: the grammar of visual design. Routledge.
Li, Linhui, Martin, Cortazzi, Liu, Bagoen, & Gao, Xiaomei. (2020). Moving eyes on pictures following visual grammar benefits meaning making: evidence from the independent reading of Chinese preschool children. Journal of Chinese Writing Systems, 4(1), 57–70. https://doi.org/10.1177/2513850219886109
Lewis, D. (2006) Reading contemporary picture books: Picturing text. Routledge.
Lepola, J., Kajamies, A., Laakkonen, E., & Niemi, P. (2020). Vocabulary, metacognitive knowledge and task orientation as predictors of narrative picture book comprehension: from preschool to grade 3. Reading and Writing. 33, 1351–1373, https://doi.org/10.1007/s11145-019-10010-7
Mikkelsen, Monica. (2016). Picture books as crossover literature: A study of how readers of different ages perceive iconotext and themes in picture books. [Unpublished Master thesis]. Western Norway University of Applied Sciences.
Morris, D., Trathen, W., Frye, E. M., Kucan, L., Ward, D., Schlagal, R., & Hendrix, M. (2013). The role of reading rate in the informal assessment of reading ability. Literacy Research and Instruction, 52, 52–64. doi: 10.1080/19388071.2012.702188.
Nikolajeva, Maria, & Scott, Carole. (2006). How picture books work. Routledge.
Pemerintah Indonesia. (2014). Undang-undang (UU) Nomor 35 Tahun 2014 tentang Perubahan atas Undang-Undang Nomor 23 Tahun 2002 Tentang Perlindungan Anak. https://peraturan.bpk.go.id/Details/38723/uu-no-35-tahun-2014
Pike, M. M., Barnes, M. A., & Barron, R. W. (2010). The role of illustrations in children’s inferential comprehension. Journal of Experimental Child Psychology, 105(3), 243–255. https://doi.org/10.1016/j.jecp.2009.10.006
Ramos, Ana Margarida. (2020). Picture book format: Beyond the relationship between words and pictures an overview of the Portuguese editorial panorama. In Libri et Liberi (Vol. 9, Issue 1, pp. 61–74). Croatian Association of Researchers in Children’s Literature, 9(1), 61-74. https://doi.org/10.21066/CARCL.LIBRI.2020.1.4
Serafini, F. (2024). The complex relationship of words and images in picture books. Journal of Visual Literacy, 43(3), 233–249. https://doi.org/10.1080/1051144X.2024.2394338
Serafini, F. (2023). How multimodality matters in children’s literature scholarship. Australian Journal of Language and Literacy, 46(3), 245-256. https://doi.org/10.1007/s44020-023-00046-2
Serafini, F., & Reid, S. F. (2022). Analyzing picturebooks: semiotic, literary, and artistic frameworks. Visual Communication, 23(2), 330-350. https://doi.org/10.1177/14703572211069623
Shi, Dan. (2023). Intermodality and visual literacy: exploring visual-verbal instantiation in children’s picture books on coronavirus, Journal of Visual Literacy, 42:3, 183-209, DOI: 10.1080/1051144X.2023.2258741
Shimek, C. (2021). Recursive readings and reckonings: kindergarteners’ multimodal transactions with a nonfiction picturebook. English Teaching: Practice & Critique, Vol. 20 No. 2, pp. 149-162. https://doi.org/10.1108/ETPC-07-2020-0068
Yefymenko, Victoria. (2024). Multimodal text-image synergy in representing interpersonal relations in picture books. Cognition, Communication, Discourse, 28, 102–109. https://doi.org/10.26565/2218-2926-2024-28-07
Zhao, Jing, Zhu, Meifang, Ruiter, de Laura, & Chen, Si. (2022). Linguistic indicators for text complexity in picture books for young chinese children learning English as a foreign language. Frontiers in Education. DOI: 10.3389/feduc.2022.758736
Data Sources
Fauzia, Sarah, & Nufus, Alimah. (2022). Naik-naik ke puncak bukit. Pusat Perbukuan. Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi Republik Indonesia.
Widjati, Tyas, & Faza. (2022). Gambar lucu Mika. Pusat Perbukuan. Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi Republik Indonesia.
DOI: https://doi.org/10.21831/diksi.v33i1.82907
Refbacks
- There are currently no refbacks.
Copyright (c) 2025 Muhamad Fanni Al Farisi, Bernadette Kushartanti, Dwi Purwanto
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Jurnal Diksi is published by Faculty of Languages, Arts, and Culture, Universitas Negeri Yogyakarta. It is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. Based on a work at http://journal.uny.ac.id/index.php/diksi
Our Journal has been Indexed by:
Diksi Journal is published by the Faculty of Languages, Arts, and Culture Universitas Negeri Yogyakarta in collaboration with Himpunan Sarjana Kesusasteraan Indonesia (HISKI)
Supervised by:
