Redundancy in the System
Whereas noise negatively impacts the successful transmission of a signal, information theory also identifies its opposite, the counterbalance to noise known as redundancy. Redundancy refers to the fact that in information systems, the message can be successfully transmitted even if some of the signal is lost, because redundant information patterns compensate for "holes" in the data of the signal.
In most communication systems, such as spoken language, much more information is transmitted in a single statement than is minimally required to convey the signal. As Weaver notes, "the redundancy of English is just about 50 per cent, so that about half of the letters or words we choose in writing or speaking are under our free choice, and about half (although we are not ordinarily aware of it) are really controlled by the statistical structure of the language." [10]
If someone shouts, "GET OUT OF THE ROOM RIGHT ___!" and you don't quite hear the final word, you will most likely be able to infer the missing "NOW" and exit the room. The sentence could lose even more words and still convey its intended signal. This kind of redundancy exists on many levels of language: it is also the case with letters. For example, you can s ill read th s sente ce even thou h som of th let ers are m ssing. Noise and redundancy together contribute to the ability of a system to transmit signals. In the case of games, redundancy is just as important a concept as noise. In a system filled with modular, interlocking informational elements, redundancy becomes important, because a system with a lot of redundancy is more flexible than one with less. Crossword puzzles, for example, take advantage of redundancy. Instead of the Twenty Questions-style single-answer guess, in a crossword puzzle there are many clues, each with its own answer. Yet the answers overlap with each other, and there is redundancy in the system that can "fill in the holes" and compensate for a clue that is too difficult to decipher. The name of an obscure capital city is much easier to figure out when previous answers have already determined some of the letters. In The Mathematical Theory of Communication, Warren Weaver explicitly draws the connection between crossword puzzles and redundancy: It is interesting to note that a language must have at least 50 per cent of real freedom (or negative entropy) in the choice of letters if one is to be able to construct satisfactory crossword puzzles. If it has complete freedom, then every array of letters is a crossword puzzle. If it has only 20 per cent of freedom, then it would be impossible to construct crossword puzzles in such complexity and number as would make the game popular.[11]
By 50 percent redundancy, Weaver means that half of the letters in a word or statement could be removed without a loss of understanding. This is because not every letter combination appears in English. There are no words, for example, in which "G" follows "T." On the other hand, letters such as "H," "R," and "E," very commonly follow "T." When we start to write a word with the letter "T," we have already cut out many possible letters that might follow it—about 50 percent of them, according to Weaver. As Weaver points out, if a language had 100 percent freedom, then every possible combination of letters would be a crossword puzzle. If this were the case, crossword puzzles would be far easier and less satisfying: when we figured out a clue and filled in the letters of a word, we would gain little or no benefit from all of the other blank words that it intersects. On the other hand, Warren points out that if the redundancy in English were too low, there simply would not be enough flexibility to properly design crossword puzzles. We could presumably construct some kind of word puzzles with 100 percent or 10 percent redundancy, but they simply would not be crossword puzzles as we know them. Crossword puzzles contain a delicate balance that keeps them just flexible enough to allow for numerous combinations, but still rigid enough so that one correctly answered clue leads to others, and yet others, until the crossword puzzle is solved. [10]Ibid. p. 13[11]Ibid. p. 14