“Should you read the text?” Semiotic Focus in Multimodal Meaning-Making
Submitted by:
Sylvana Sofkova Hashemi
Abstract:
The socio-technological changes in the communication and representation of meaning provide opportunities for more hybrid, intertextual and creative texts that go beyond traditional modes, conventions and genres. This paper explores what the presence of digital and mediating resources means in the literacy classroom. The current study in particular focuses on the semiotic choices that eight-year-old students make when interpreting digital instructional text composed by peers and how they make meaning from the text: where the students direct their semiotic attention and how does the text design influence their reading.
The cross-class work was followed on two separate occasions using ethnographic techniques for the collection of data that include video-observations of students’ work, field notes, photographs and semi-structured follow-up interviews. The study is informed by the theoretical perspectives of social semiotics assuming that the relation of form and meaning is motivated by the interest of the sign-maker and the social context (Kress, 2010; Bezemer & Kress, 2016). The study uses the multiliteracies (New London Group, 1996) and grammar of visual design (Kress & van Leeuwen, 1996/2006) frameworks as meta-tools to examine the overt purposes and intentions (i.e. semiotic functions) of the digital texts the students interpret and what in the meaning of the text captures their attention.
The study demonstrates the complexity of reading digitally mediated multimodal texts in regard to the hybridity and blend of semiotic resources that affords the students a wide range of available designs as a mix of genres. These young students predominantly interacted with the visual modes influenced by their prior experience as novice readers as well as the text design. The findings imply a need for a holistic approach to literacy teaching that involves an understanding of the multimodal design and hybridity of texts as well as semiotics of technology.
Bezemer, J. & Kress, G. (2016). Multimodality, Learning and Communication: a social semiotic frame. London: Routledge.
Kress, G. (2010). Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.
Kress, G. & van Leeuwen, T. (1996/2006). Reading images: The grammar of visual design. London: Routledge.
New London Group (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66, 60-92.