Noise and Telecommunications Systems

Noise, as a research topic studied for its own sake, is an endlessly fascinating and richly rewarding intellectual treat; and as a topic of pragmatic concern, it is absolutely essential for comprehending and improving the operation of telecommunication systems, electrical circuits, semiconductor devices, computers and practically all systems in electrical engineering. As noted in [1], “communication is a process of transferring information from a source to a destination,” but during transmission through the channel, noise distorts the message. Consequently, understanding the properties of noise is key to accurately decoding the received message. Noise plays a central role in Information Theory, where its analysis spawned Shannon’s Entropy, and while Thermodynamic entropy was already birthed in the nineteenth century, it is the study of noise that reveals the intimate connection between the two concepts of entropy. In its most dramatic manifestation, microwave cosmic background radiation from the thermal noise bequeathed to us by the Big Bang informs us about events in the first trillionth of a trillionth of a trillionth of a second of the age of the universe [2].


Background
Noise, as a research topic studied for its own sake, is an endlessly fascinating and richly rewarding intellectual treat; and as a topic of pragmatic concern, it is absolutely essential for comprehending and improving the operation of telecommunication systems, electrical circuits, semiconductor devices, computers and practically all systems in electrical engineering. As noted in [1], "communication is a process of transferring information from a source to a destination," but during transmission through the channel, noise distorts the message. Consequently, understanding the properties of noise is key to accurately decoding the received message. Noise plays a central role in Information Theory, where its analysis spawned Shannon's Entropy, and while Thermodynamic entropy was already birthed in the nineteenth century, it is the study of noise that reveals the intimate connection between the two concepts of entropy. In its most dramatic manifestation, microwave cosmic background radiation from the thermal noise bequeathed to us by the Big Bang informs us about events in the first trillionth of a trillionth of a trillionth of a second of the age of the universe [2].

Historical Perspective
Noise has a venerable history, with origins in the field of Statistics which dates back to at least the 1650s [3]. However, the modern study of noise may be said to have begun after 1900, sparked by Einstein's brilliant analysis of the phenomenon of Brownian motion -the incessant random movements of pollen grains suspended in water, observed by the botanist Robert Brown. In a spectacular display of mathematical sleuthing, Albert Einstein proved the existence of atoms by predicting Avogadro's number (6×10 23 ), based on the properties of Brownian motion [4]. The methodology of stochastic differential equations, an indispensable tool in signal processing, signal detection and signal estimation, is grounded in Brownian motion. The birth of Quantum Mechanics in the early years of the twentieth century ushered in completely new perspectives on noise, introducing the concept of complex-valued probability amplitudes whose squared magnitudes produce probabilities of quantum events. Noise is inseparably implicated in the operation of vacuum tubes, radio telescopes, microwave ovens, lasers, computers, GPS systems and cell phones, all of which are quantum devices.

Implications of Noise for Telecommunications
Noise sets fundamental physical limits on the performance of all electronic and electrical systems. The amazing fact is that the performances of the vast majority of the devices we use operate extremely close to these limits [5]. No device exemplifies this more than Global Positioning System (GPS), which incorporates into its operations the fundamental physical limits imposed by the three great theories of the twentieth century that rocked the very foundations of physics -special relativity, general relativity, and quantum mechanics. GPS operates accurately only by applying corrections to the trajectories of electromagnetic signals arriving from geosynchronous satellites; these corrections stemming from time-dilation, a very real effect due to the universal upper bound on the speed of all material objects imposed by the speed of light, the space-time curvature caused by the mass of the earth, and the quantum effects that allow the electronics of GPS to succeed. Communication systems contain one or more of the following types of noise -white noise, shot noise or Schottky noise, thermal noise or Johnson noise, and quantum noise. Johnson noise is the most important because it is present in all communication systems -indeed in all physical systems -a fact due to its origin in the thermal motion of the molecules [6]. A communications system operating at very high frequencies encounters statistical fluctuations in the detection of individual photons due to the quantization of the electromagnetic field at those frequencies -these fluctuations are called quantum noise [6]. Consequently, quantum noise is the dominant type of noise at very high frequencies.
How reliably a bit can be stored and how fast it can be transmitted through a channel is essentially limited by noise [5]. The Bell Labs engineer Claude Shannon created Information Theory to analyze and find guidelines to improve the performance of telecommunication systems. Many of the theorems of Information Theory revolve around the concept of the Shannon Entropy of a probability distribution {p 1 , …, p n }, p i being the probability of choosing symbol 'i, from an alphabet used to code a message, often the binary source S = {0, 1}. The Shannon entropy is defined as: In the 1940s, the engineering community intuitively expected the error probability to increase as the transmission rate of a message increased [7]. Shannon made the surprising discovery that the error probability can be made arbitrarily small even as the transmission rate increases, as long as that rate remains below the channel capacity, a result enshrined in Shannon's Channel Coding Theorem [7]. Shannon showed that the optimal coding to maximize the channel capacity is such that C = Wlog (1 + Γ), where Γ is the signal to noise ratio, defined as the ratio of the average signal energy to the average noise energy, and W is the bandwidth of the channel. This result is known as Shannon's equation [6]. Thus for large Γ, C is approximately log (Γ), while for small Γ, C is approximately Γ. Shannon's equation reveals an unexpected fact even when the signal is much weaker than the noise (small Γ), the channel capacity remains above zero. From the law of large numbers, it can be shown that the sample entropy converges asymptotically to the population entropy from which follows the Asymptotic Equipartition Property (AEP) with implications for data compression. If the number of letters in the alphabet used for encoding the message is X, and the sample size is N, then the total number of possible strings is X N . The AEP states that the log-likelihood function is approximately 2 -NH(X) for large N, and consequently, that the effective number of strings will be 2 NH(X) . But the actual number of possible strings is larger, given by X N =2 Nlog 2 (X) ; and thus the difference between these two values enables data compression [5].
Noise interferes in the detection of a true signal. An often used example is that of a WWII radar operator who has to sound the alarm if enemy planes intrude into his territory. Increasing the sensitivity (or the hit rate) of the radar could ensure that there was no intrusion. But it would also result in numerous false alarms; say from the inability to distinguish between the feedback from a flock of geese and that of an enemy plane. Decreasing the sensitivity of the radar could reduce the number of false alarms but also increase the probability of an intrusion going undetected. Signal Detection Theory [8] was developed to study such problems and has since been extensively used in various domains such as signal processing [9], medical applications [10] and psychology [11]. At the heart of Signal Detection Theory is a 2×2 matrix described below Table 1.
A commonly used measure of sensitivity or discriminability in Signal Detection Theory is ' d (dprime) and calculated as ' d =Z (hit rate)-Z (false alarm rate), where hit rate is the ratio of total number of hits to total number of signal trials, false alarm rate is the ratio of total number of false alarms to total number of noise trials and ( ), is the inverse of the cumulative density function of a normal distribution.
Portable wireless devices have revolutionized the way individuals and institutions communicate with each other. However, noise in wireless channels presents a challenge to the performance of these devices. Mobile devices, sensors, smart sensors, laptops, RFID, GPS and cell phones have collectively created the concept of informationrich environments engaged in pervasive computation, and thus birthed the field known as ubiquitous computing or pervasive computing. High noise levels increase the uncertainty of an object's location and also result in lower performance of gadgets that have limited battery life. Given that location awareness is perhaps the most critical component of pervasive computing; the research community has proposed methodologies such as clustering [12], Bayes Filters [13] and fuzzy logic [14] to reduce the detrimental effects of noise so that the location of an object can be determined accurately. Interestingly, the widespread use of mobile devices has also provided individuals with the opportunity [15] to assess the level of noise in their environment and share it with others. Such participatory sensing GIS [16,17] activities have important implications for both individual and group behaviors as well as public policy.

Conclusions and Projections
Research into the implications of noise in information-rich environments will continue to be an important research effort, representing a natural extension of the results of information theory for older forms of communications systems such as land-based telephones, amplifiers, transmitters and receivers that are at the heart of all telecommunication systems. Pervasive computing presents new challenges to the ancient problem of coping with noise in all its forms-shot noise, Johnson noise, white noise, pink noise, 1/f noise, and quantum noise. Noise, and especially thermal noise, will present increasingly difficult challenges as the circuits in the devices of pervasive computing become denser and their sizes shrink to atomic dimensions. New results to advance the state of the art in pervasive computing and their implications for telecommunication systems await discovery by new generations of engineers, mathematicians and scientists.