Multidimensional Approaches to the Study of Written Communication
Ph.D. in Philology,
Assistant Teacher in the Applied Linguistics Department of the Institute of Foreign Philology
Lesya Ukrainka Eastern European National University in Volyn
Keywords: communication, visual, multimodal, verbal, non-verbal, paralinguistics.
Макарук Л. Л. Полиаспектное исследование письменной коммуникации
Аннотация. В работе предложено новый подход к рассмотрению письменной коммуникации, которая одновременно является предметом нескольких лингвистических направлений. Такой подход обусловлен тем, что в современном комуникативно-ориентированом пространстве появляется все больше каналов для обмена информацией, и, следовательно, средств, которые осуществляют этот процесс. Поэтому, возникает необходимость подробнее проанализировать те научные направления, в центре внимания которых остается коммуникация. В связи с этим в работе проанализировано лингвистические отрасли, которые в большей или меньшей мере имеют отношение к рассматриваемой проблеме. Речь идет об графической лингвистике, паралингвистике, визуальной коммуникации и мультимодальности. Обусловлено, что каждое из этих направлений имеет свой предмет, объект, задания и методы исследования, но в связи с современным развитием технологий появляются новые графические единицы, статус которых пока не изучен. Следовательно, некоторые отрасли, которые оставались, долгое время в тени, но были созданы и развивались благодаря трудам отдельных ученых. Из этого следует, что их нужно возродить, пересмотреть терминологический аппарат, инструментарий и его составляющие в целом. Это касается графической лингвистики и паралингвистики, в центре внимания которой в большей степени находились паралингвистические компоненты, сопровождающие устную речь. Новые единицы, которые появляются, требуют от исследователей новых подходов, которые и рассматриваются в рамках визуализации и мультимодальности. В работе также перечислены некоторые единицы, которые появились сравнительно недавно и, соответственно, входят в спектр отдельных исследований, составляя, таким образом, его терминологический аппарат. Обусловлено также, что лингвисты сегодня немного отошли от традиционных исследований и обратили значительное внимание на те сферы, которые в предыдущие годы оставались на периферии. Это создало новые возможности в исследованиях и позволило открыть несколько, на первый взгляд, нетрадиционных параметров, которые еще предстоит рассмотреть и подробно изучить.
Ключевые слова: коммуникация, визуальный, графическая лингвистика, паралингвистика, мультимодальность.
The fact that communication is a multidimensional and multifunctional process, and one which is quite complicated in its nature, its structure and its constituents, is seldom considered by people in general. Though they may be aware that the ability to communicate is inborn, many people fail to realize how they influence the independent development of that communication, whether intentionally or unintentionally.
Communication itself undergoes a variety of changes because individuals interfere with it, trying to modify it ostensibly for the sake of simplicity, but instead introducing contradictory elements that complicate the process rather than simplying it.
In the last decade technological advances have had a massive influence on communication, affecting the participants in the speech act―the sender and receiver―and its components―the channel and the code―to a greater or a lesser degree. The resources which facilitate successful interaction have increased significantly and need to be subjected to argumentational analysis by linguistic science.
These and other factors have altered traditional scientific paradigms and approaches, and have brought about the formation of some new and unusual ones which are considered to be important for further studies.
Communication as a whole [5; 9] has customarily been divided into verbal and non-verbal communication, although the latter has received less research attention since it wasn’t considered to be of primary importance. The situation has evolved gradually and has caused a change from traditional linguistic viewpoints, in terms of both verbal and non-verbal modes of expressing thoughts in combination for the purpose of conveying information. Any form of communication, whether verbal or non-verbal, often involves the two codes mentioned above.
With respect to ‘written’ communication, in the technological age it has become more common to talk about those forms of printed expression which have become increasingly popular with the development of the Internet, opening up new possibilities for communication [4; 5] based on programming which facilitates information exchange, including social networking, electronic mail, forums and blogs, and interaction through Skype and ICQ .
It may be considered that communication has a history almost as long as mankind itself, to the extent that even cave men would have felt the necessity of interacting. It was naturally quite primitive to begin with, its first surviving manifestations being the cave paintings . No matter where they were, prehistoric men used similar symbols and signs to represent their thoughts. Later, they must have realized the limitations of their system in terms of potential ambiguity, in that one image could have multiple meanings. Even though they were then in a very primitive form, some of the visceral images which we use today have evolved from those early drawings and almost no one tries to state categorically how old they may be. However, we cannot help but be aware of their importance to human progress, in that they are intrinsically linked to communication.
The first stage of written communication was thus based on drawings, and is known as pictorial or pictographic writing . This picture-based process without words was the predecessor of other, more sophisticated systems of written communication which appeared later. It is evident that the modern signs which we call pictograms have nothing in common with those which were developed more than 50,000 years ago, and their objective is quite different – not just to represent an idea or a thought, but to be understandable to people all over the world.
Communication itself is one of the key aspects of several linguistics-related disciplines. One of the oldest of these is ‘graphic linguistics’, which was established by Ronald A. Crossland  in 1956. The necessity of creating a new branch of linguistics had actually become evident in the middle of the previous century when writing systems became of particular interest to scholars. Although Ignace Gelb  is generally credited with having realized the importance of a developing a new branch of linguistics, a German scholar named Friedrich Ballhorn  had mentioned about it a century earlier.
In their works, Russian scholars who were dealing with writing systems also described the importance of having a special subject which would be focused on the synchronical and diachronical aspects of writing itself, the stages of its development and the possible prospects for its future. New terminology naturally appeared in their works, but there was no unanimous answer regarding the naming of the new discipline, although many possibilities were proposed, such as grammatology  and graphistics . We are inclined to voice support for Crossland’s recommendation, which received further extensive study and development by the famous Russian linguist T. Amirova, that it should be called graphic linguistics.
In modern research papers this term is almost never used and the field is obviously not studied as a separate linguistic branch. In our opinion, it ought to be the object of research now, and its scope should be broadened significantly. The ‘information revolution’ mentioned above represented a new stage in its development, which involves not only letters which stand for words, but also various other signs and symbols which are used for a range of purposes in the communication process. Along with the increased number of different means and devices employed in the communication process, we are of the opinion that other characters which may be of a non-language nature but which can convey language information should be regarded as an inalienable part of its apparatus.
Within the linguistic discipline which has been discussed, there is another problem which remains to be solved, involving an in-depth study of and a prospective extension of its terminology. Some existing terms should be redefined, and its subject, object, tasks and methods need further investigation. In modern research papers a number of terms occur with varying degrees of frequency which we believe should be included within the terminology of graphic linguistics, such as “graphemic symbolism” , “graphitic contradictions” , “graphological freedom” , “graphological distinctiveness” , and “graphitic distinctiveness” . Some other terms which occur in various sources are: graphoderivate, code-graphicisation, graphic play, graphic and orphographic elements, graphic imagery.
Another discipline which devotes more attention to non-verbal elements rather than to the verbal is paralinguistics , which is sometimes called metalinguistics. There is some considerable disagreement regarding this discipline’s identity, and for a long time it languished in an inferior position in comparison with other linguistic disciplines which were developing quickly. Moreover, its main focus of attention involved non-verbal means which accompanied speaking, all of which were apt to be classed in with various other groups, such as kinesics, proxemics, and haptics. In previous studies, more attention had been devoted to those aspects, and a certain portion of it was focused on non-verbal written communication.
It seems evident now that more attention should be devoted to non-verbal elements which successfully perform a number of functions allotted to them. The research and the papers analysed previously make it possible to conclude that when more ways and types come to light with respect to the process of interpersonal communication, they merit a correspondingly greater amount of attention.
Paralinguistic graphic means have only been studied carefully in a few research papers, and have been divided into a number of respective groups. Generally the spectrum of paralinguistic devices is considered to have a connection with the following aspects: the graphic segmentation of a text and its location on the page; the line spacing, font, colour, underlining, and italicization of textual material; the presence of typographical signs, graphic symbols, figures (numbers) and some subsidiary signs (§, №, %, +, -); iconic language symbols (pictures, photos, tables, diagrams, and drawings), unusual patterns of writing words or punctuation marks; and other characteristics such as page orientation, margins, and text width. Obviously it must be understood that this list is not comprehensive, and that it may be improved―and, in fact, will be improved as changes in language occur.
Another point which seems to require description in our opinion is that these non-verbal means need to be studied within the two branches of linguistics mentioned above, graphics and paralinguistics, but when they are treated as separate subjects, it is necessary to determine the difference between the various approaches involved in the study of the graphic devices which are associated with written communication (discourse).
In our opinion, graphic linguistic should deal only with the spectrum of graphic signs which are used within a word or a sentence, which according to modern typology they are termed ‘ideograms’. Although of a non-imagery nature, they convey a certain informational content. Moreover, within graphic linguistics there is no need to define the functions and the main usage objectives of the devices: the discipline should only be employed to determine whether or not there has been an increase in the existing elements which make up the totality of its devices; whether it is be included as a character and takes on a fixed identity; and whether or not it should be viewed as a constituent part of the range of graphic devices utilized within the English language. In the meantime any question regarding their importance or essentiality will vanish as they are codified and become generally accepted. Apart from this, one of the most important questions which needs to be considered within the graphic linguistics is orthography, which evidently does not belong within the purview of paralinguistics.
However, paralinguistics does include non-verbal modes used in both oral and written communication. In addition to the devices or features which have already been enumerated above and which are a part of graphic linguistics, there are others which fall into the category of imagery, and which cannot be fixed or limited in the sense that it is impossible to place limits on an individual’s capacity for imagery. However, they form a part of the general typology of non-linguistic devices and convey linguistic information of some kind. Among them we can distinguish different types of images, including pictures, photos, diagrams, tables and graphic means which are based on images, such as pictograms made up of different combined logos. Of course, logos can also consist of uniquely graphic means which are lack the characteristics of an image. Within paralinguistics, other aspects can also be investigated, including the distinctive features and the functions of the various devices.
One new research field which has aroused particular interest among scholars is visual communication. We have termed it a ‘new’ field, but in actuality the historical development of civilization shows that its foundations were laid more than 50,000 years ago as has been mentioned above. This can be regarded as another positive indication that modern linguists are contemplating not only the traditional approaches which have been used with regard to the problems under consideration, but that they are also confidently promoting new ones and foreseeing excellent prospects for success and future development within them. The full range of available paralinguistic elements may differ from one language to another; for example, in English there are almost no signs which are a constituent part of a letter such as those which exist in French (é, è, ê, for instance), or in German (such as ä and ö), we mean accent and diacritical marks.
Visual communication [2; 11] constitutes an area of study which can be simultaneously viewed as new and old. It can naturally be regarded as a subsidiary part of communication in general, but considering its current importance and the great interest it has evoked in those who are involved in studying, developing and applying it, it needs to be stated that it is a separate discipline which is undergoing very rapid development. It is based on semiotics because of its involvement with image-characters and signs. Sometimes the term ‘iconic’  is applied to it, and in many research papers the terms are used synonymously. Although they are the same in character, they can be studied separately because of certain differences.
The contemporary state of this branch of study shows that scholars are particularly interested in it, as it is the focus of research in many discrete research fields. In terms of the linguistic aspect it may seem that to describe images which are non-verbal in nature is quite complicated, if not impossible, but it could be argued that this is not the case. Since it has been studied a great deal and involves results from other branches, it appears to be one of the most promising and interesting branches, with its objectives, tasks and special methods.
It is clear that it has created an ‘apparatus’ of terms inherent within itself which are still being developed and expanded. The concepts which have been formulated and which are now in use are the following: visemics (the combination of semiotics and visual communication), visual language, visual perception, visual art, visual culture, visual artifacts, visual aids, and visual mental imagery. In addition to these, it has become evident that there is a need to introduce this new area linguistics that is referred to as visual linguistics.
This need not be regarded as something strange, because when we study elements which are visual in nature it is possible to observe paradigmatic relationships which are common in traditional language investigations. Moreover, visual syntactics and visual morphology can also be found and included within visual grammar. Visual perception and recognition can be considered within cognitive linguistics and the functions of visual elements within functional linguistics. The elements which are of linguistic interest clearly exist in combination, so two codes are used, verbal and non-verbal. It is evident that visual linguistics (communication), because of its essentially visual mode, deals only with written messages.
Another aspect which is a problematic one (although all those mentioned above likewise require further discussion and investigation) is the problem of visual literacy [6; 17] and the choice of reading rules which can be applied and used. It has to be considered whether there is any rule which can be applied in order to grasp the idea behind an image that is presented image. It is evident that not all of these approaches need further investigation.
In Russian linguistics it is more common to refer to these as creolized texts, although there is no term which is unanimously used and accepted (in Ukrainian and Russian linguistics); the terms ‘intersemiotic’ and ‘polycode’ are also sometimes used as well.
Multimodal discourse analysis  presupposes an investigation of verbal and non-verbal means of communication in a single context, in both written and oral communication. It is also based on semiotics, in the sense that signs and symbols are the integral elements of information. The variety of directions and problems [14; 17] involved in the study of multimodal discourse analysis has enabled scholars to study its peculiarities within the context of systemic-functional multimodal discourse analysis.
This latter area takes in all the approaches mentioned above and, in comparison to the linguistic branches already referred to, along with visual communication, opens up new possibilities for further study. These approaches are very much contemporary and will be developed further as technologies develop more and more. Thanks to this factor and to the changes in traditional linguistic views, all of us in the field of linguistics are looking towards a new stage of language development which enables us to bring an aspect of reality and current relevance to things which previously seemed unreal. New approaches in general linguistics (such as visual linguistics) have appeared and are being developed rapidly although they could scarcely have been imagined a few years ago. Visual grammar , visual literacy , visual methodology  and visual perception lead scholars into branches of study which they would never have foreseen earlier.
However, graphic linguistics and paralinguistics (metalanguage) are also important in their own right. Textual linguistics is also important for the study of the distinctive features of devices related to the nature of imagery. No doubt new branches will appear, just as ‘netlinguistics’ or ‘journal linguistics’  are being introduced and developed now, for example.
It should also be mentioned that there can be a ‘peaceful coexistence’ between the previously existing and the newly-developed linguistic branches, which makes it possible to conduct a general linguistic analysis and to provide scholars and any other interested parties with the detailed and in-depth results. New and as yet unknown possibilities may be discovered in the future, although it isn’t possible to be certain that the existing ones are as easy to solve as it might seem at first glance. We must also bear in mind that even as Internet resources and programming achievements develop, they will never be able to deal with all the possible tasks which could be studied. No one can predict with certainty what technologies and devices may be invented even in the next five years, or what effects they may have on the recipients.
All the branches of linguistics that have been mentioned above, as well as multimodal discourse itself, face the same problems for which solutions need to be devised in the time to come―the codification of their devices, the identification and structuring of their terminology, and the tasks and methodology which make in-depth analysis possible and improve the prospects for identifying and cataloguing their peculiarities.
1. Ballhorn F. Grammatography. A Manual of Reference to the Alphabets of Ancient and Modern Languages / Friedrich Ballhorn ; translated from German. – London, 1861. – 176 p. [Electronic resource]. – Access mode : http://archive.org/details/cu319240264519
2. Bergstrom B. Essentials of Visual Communication / B. Bergstrom. – Published by Laurence King ; Publishing Ltd. ; Printed in China. – 2008. – 240 p.
3. Crossland R. A. Graphic Linguistics and its Terminology / R. A. Crossland // Mechanical Translation. – Vol. 3, no. 1, July. – Massachusetts Institute of Technology, 1956. – P. 8–11.
4. Crystal D. The Cambridge Encyclopedia of the English Language [Paperback] / D. Crystal. – [2 edition]. – Cambridge University Press, 2003. – 506 p.
5. Dominick J. P. The Dynamics of Mass Communication. // Joseph P. Dominick 4th ed. Published Created New York Mc Graw-Hill. 1993. – 616 p.
6. Elkins J. Visual literacy / James Elkins. – NY : Routledge, 2008. – 217 p.
7. Gelb. J. Grammatology and Graphemics / I. J. Gelb // Papers from the Fourth Regional Meeting of Chicago Linguistic Society. – Chicago, 1968. – Р. 197–198.
8. Helfman S. E. Signs and Symbols Around the World / Elizabeth S. Helfman. – Printed in the United States of America, 1967. – 192 p.
9. Holden S. Macmillan Topics / Susan Holden. – Macmillan Publishers Limited, 2006. – 24 p.
10. Leborg C. Visual Grammar / Christian Leborg. – NY : Princeton Architectural Press, 2006. – 97 p.
11. Lester P. M. Visual Communication Images with Messages / P. M. Lester 4th edition. Belmont, CA : Wadsowrth Publishing Company. – 2006. – 464 p.
12. O’Halloran K. L. Multimodal discourse analysis / K. L. O’Halloran ; K. Hyland and B. Paltridge (eds). – London & New York : Continuum, 2011. – 33 p.
13. Rennagel W. R. Graphistics / W. R. Rennagel // Studies in Linguistics in Honor of George L. Trager. – The Hague ; Paris, 1972. – Р. 58–65.
14. Snyder J. Applying multimodal discourse analysis to study image-enabled communication [Electronic resource] / Jaime Snyder ; Syracuse University, School of Information Studies. – NY & Syracuse, 2010. – Access mode : https://www.ideals.illinois.edu
15. Rose J. Visual Methodologies. An Introduction to Researching with Visual Materials / Gillian Rose. – [2nd Edition]. – London&Thousant Oks&New Delhi : Sage Publications, 2007. – 386 p.
16. Trager G. L. Paralanguage: A First Approach // Studies in Linguistics. – 1958. – № 13. – P. 1-12.
17. Walsh M. Reading visual multimodal texts: how is ‘reading’ different? [Електронний ресурс] / Maureen Walsh ; Australian Catholic University // Australian Journal of Language and Literacy. – 2006. – Vol. 29, № 1. – P. 24–37
18.Yazdani M. Communication through Icons / Masoud Yazdani // Iconic Communication / ed. by Masoud Yazdani, Philip Barker. – Bristol, UKPortlandOR, USA : Intellect Books, 2000. – P. 63–68.