Value-added text: Where graphic design meets paralinguistics

Expressive typography is the sine qua non of the graphic designer-font styles and parameters such as size and color are selected to lend additional interpretive potential to a plain text message. When applied by a designer the process is intuitive and is hardwired to a particular text. Value-added text (VAT) is an attempt to visually extend the semantic potential of a message still further in a computer-based environment and to render the process both algorithmic and dynamic, its principles being applicable to typographic (and to iconic) text. This paper emphasizes the exploration of potential paralinguistic mappings which exploit and extend the traditional vocabulary of typography. Much that can be communicated in human-to-human language is lost in its transfer to text but paralinguistics-which studies the features of communication that accompany, or substitute for, the bare words used-offers a gateway to an enriched presentation of text. VAT therefore proposes automated graphic proxies which communicate more in a typographic message than the literal semantics of the user's native language and also offers potential assistance in cross-language communication.

INTRODUCTION

When I first visited France I spoke expressionless schoolboy French and was understood by nobody. My wife on the other hand spoke only English but accompanied it with appropriate gestures and expressions and was universally comprehended. It was clear that the paralinguistic features of her language carried more meaning than the actual words of mine and had transcended the barriers of foreign language. In a similar manner the graphic designer will aim to communicate more in a typographic message than its literal semantics.

Most people have experienced problems responding to emails where the 'tone' of the message is ambiguous. While the word choice and syntax of formal written text gives controlled clues about whether the writer is being factual, cynical, angry or ironic for example, the closer stylistic proximity of email to spoken language does less to protect the recipient from the absence of contextualizing facial expression and intonation.

Elam subtitled her book on expressive typography 'the word as image' and it is the visual characteristics of a message-its image-that VAT employs. It replaces some of the meaning lost when rich layers of verbal communication are stripped off in the transposition to naked text. To some extent punctuation marks retain the flavor of spoken language, providing symbolic representation!?*!, and typographic options such as BLOCK CAPITALS can restore emphasis to the mental elocution that accompanies silently read text. Much that can be communicated in human-to-human language is, however, lost and paralinguistics studies those features of communication that are additional to the bare words used.

In direct communication between humans the words spoken are supplemented by, or on occasion replaced by, a range of paralinguistic features such as body language, expression, gesture, intonation, volume, etc. which serve to qualify the bare bones of the message. VAT provides visual mappings for these features, mappings that can then be combined with raw text to restore some of its lost semantic support. These mappings can be abstract, symbolic, representational or mimetic and either static or kinetic. Their application can readily borrow from the grammar, syntax and vocabularies of art, design, film, theater and existing sign languages (subject to cultural variations and familiarity). Their operation within a computer environment provides opportunities for dynamic presentation.

A semiotic division can be made of the major signs and signals of communication into verbal (in which the linguistic element of speech can be separated from the expressive, non-linguistic elements known as supra-segmentals) and non-verbal (which divides again into visual and tactile). Of these the visual elements potentially have a very direct relevance to VAT and the prosodic features of pitch, loudness and tempo (which together contribute to the rhythm of language) offer readily measurable input data that could be automatically mapped to visual characteristics.

Other voice qualities, such as timbre and intonation, can give clues about the emotional state and social group of the speaker as well as about the information structure of the utterance-for instance whispering can tell of secrecy or conspiracy. They are, unfortunately, prone to international variation in interpretation and therefore only useful within a limited language domain.

Although VAT was originally conceived in the context of iconic text (Mealing, 1993)-as a means of adding proxies for those features of human communication which could improve its international comprehension-its initial application to typographic text is perhaps more obvious. If consideration of cross-language issues is temporarily suspended then typographic text has some advantages as a starting point. Its features are comprehensively embodied in and easily accessed through a range of technologies, in particular computer applications, it is universally understood (by the literate) and, in normal practice, it enables clear understanding. Clear, that is, within the terms of reference of written language but without paralinguistic accessories.