Multimodal is all interactions, meaning multimodal stresses that all means of  communication  playan  important  role  both  verbally  and  visually  because language  contains  meaning,  content  or  informative  content.  According  to O'Halloran  (2008)  states  multimodal  analysis  includes  all  types  of communication  that  have  text  interactions  and  interactions  of  two  or  more semiotic  sources  or  means  of  communication  to  achieve  the  communicative function of the text. 

Multimodality  is  defined  as  "the  use  of  several  semioticmodes  in  the design  of  semiotic  product  or  event,  together  with  a  particular  way  in  which these  modes  are  combined".Modality"  in  multimodal  discourse  refers  to  how people  use  semiotic  resources  to  create  truth.  It's  a  very  complicated  concept and has different modality markers in different semiotic resources. It does not simply  equal  to  modal  verbs  or  auxiliary  verbs,  for  example  can,  may,  and must.

Theories of Multimodal

1.      Kress and Leewen

Kress and Leewen’s opinion, the three metafunctions of linguistics can be extended  to  visual  communication.  In  Reading  image,  they  see  image  as  a resource  for  representation  and  thus  will  display  culturally  produced regularities. The meanings expressed by people are the first and foremost social meanings, so Halliday’s three metafunctions for language can also be used a starting  point  for  their  account  of  images  because  they  assume  the  three metafunctions  model  works  well  as  a  source  for  thinking  about  all  modes  of representation.  Kress  and  Van  Leeuwen  assume  that  images,  color,  music, typography  and  other  visual  modes  are  similar  to  language  and  they  can simultaneously fulfill and realize the three broad communicative metafunctions as  language  does.  In  their  view,  image  and  other  visual  modes  can  represent objects  and  their  relations  in  a  world  outside  the  representational  system,  so there  are  many  ideational  choices  available  for  visual  sign-making  in  visual communication.  They  also  think  that  image  and  other  visual  modes  have  the capacity  to  form  texts,  complexes  of  signs  which  internally  cohere  with  each other  and  extremely  with  the  context  in  and  for  which  they  were  produced.

Besides, image and other visual modes are able to represent a particular social relation  between  the  produce,  the  viewer  and  the  object  represented.  And  all semiotic systems are social semiotic system which allow us to negotiate social and power relationships. They  see images of whatever kinds as means for the articulation of ideological position. That’s why in their Reading Images, they draw  examples  from  many  domains,  such  as  textbooks,  websites, advertisements, magazines articles and so on to express their meaning making in  their  social  practices.  The  key  notion  in  any  semiotics  is  the  sign  or  sign making,  so  Kress  and  Van  Leeuwen  discuss  forms  (signifier)  such  as  vector, modality, gaze, composition, perspective, line and color, as well as the way in which these forms are used to realize meanigs in the making of signs.

Based on  Halliday’s theory,  Kress and Van  Leeuwen use a slightly different  terminology  in  discussing  the  meaning  of  image  in  visual communication:  representational  instead  of  ideational;  interactive  instead  of interpersonal; and compositional instead of textual. As for the representational meaning,  they  have  distinguished  two  kinds  of  image  in  the  light  of  the different  characteristics  of  image:  one  is  narrative  images  which  involve  four processes,  those  are  action  process,  reactional  process,  speech  and  mental process, and conversation process, another is conceptual images which include three  kinds,  namely  classificational  process,  anlytical  process  and  symbolic process. 

Kress  and  Van  Leeuwen  have  suggested  three  ways  to  examine  the interactive meaning of images from three aspects: contact, social distance and attitude.  The  compositional  meaning  of  images  is  realized  through  :  three interrelated systems: information value, salience and framing. From Kress and Van Leeuwen’s visual  grammar,  we  cam  see  that  images  are  made  up  of elements that can be decomposed when we analyzed the meaning of them. Just as  language,  images  have  meanings  only  when  they  are  integrated  together. That is, the meaning of visualimages comes from the arrangement of different visual elements. The three metafunctions put forth by Kress and Van Leeuwen are  not  direct  relationships  between  semiotic  resources  and  meaning.  The concept such as power, interaction, detachment and involvement and so on are not meanings hidden in the images, rather, they are meaning potentials, that is they  are  the  possible  meanings  which  will  be  activated  by  the  producers  and viewers of images.

2.      Bull and Anstey

In  multimodal  analysis,  according  to  Geoff  Bull  and  Michelle  Anstey (2010),  state  that  a  text  is  called  multimodal  if  the  text  is  realized  from  a combination  of  two  or  more  semiotic  systems.  There  are  five  semiotic  (sign, symbol and or code) systems that the multiliterate needs to have knowledge of when exploring and examining multimodal texts.

a.       Linguistic analysis

Comprising  aspects  such  as  vocabulary,  alphabets,  generic  structure  and the grammar of oral and written language.

b.      Visual analysis 

Comparising  aspects  such  as  colour,  layout,  page  or  screen  and  still  and moving  images.  Here  visual  images  may  involved  body  laguage,  motion, dance,  two  and  three  dimensional  works  of  art,  photographs  and  clipart, films  and  video,  museum  exhibits  and  dioramas,  advertisements, illustrated written or verbal discourse, architecture, hypermedia and visual relity experiences, and so on.

c.       Audio analysis

Comparising  aspects  such  as  volume,  pitch  and  rhythm  of  music,  sound effects and silence.

d.      Gestural analysis

Comparising  aspects  such  as  movements,  speed  and  stillness  in  facial expression and body language and posture.

e.       Spatial analysis

Comparising  aspects  such  as  proximity,  direction,  position  of  layout  and praganisation of object in space.





Postingan terkait: