Threshnomics: An Introduction to Threshold Logic in Algorithms and Circuits

Human intelligence is understood to be a result of collective decisions over a period of time with an objective to enforce a desired objective or activity. The neural understanding of the intelligence is reflective of a simplified behaviour of neurons and its networks. These neurons represent the building blocks of biological intelligence that is a collection of decision making units arranged in hierarchy at levels of functional and structural levels. These decision making units does the operation that we refer to as threshold processing. Because there are a huge variety of these decisions enabled by the threshold in any biological intelligence, it would be a natural progression in terminology to refer this large scale processing as threshnomics.


Introduction
Human intelligence is understood to be a result of collective decisions over a period of time with an objective to enforce a desired objective or activity. The neural understanding of the intelligence is reflective of a simplified behaviour of neurons and its networks. These neurons represent the building blocks of biological intelligence that is a collection of decision making units arranged in hierarchy at levels of functional and structural levels. These decision making units does the operation that we refer to as threshold processing. Because there are a huge variety of these decisions enabled by the threshold in any biological intelligence, it would be a natural progression in terminology to refer this large scale processing as threshnomics.
The seminal ideas on thresholds happening in human brain were exposed to a usable form when McCulloch and Pitts introduced the first mathematical model of a neuron: the threshold logic (TL) gate (TLG) [1] in 1943. The concept of threshold is not limited to biology but can be seen in everyday science.
A simple thresholding function is depicted in Figure 1. Here, any continuous signal in a given time can be converted to a digital signal using a threshold. The application of different threshold values result in different set of binary signal patterns. Theoretically, any analogue signal can be comprised of such infinite number of threshold and corresponding binary patterns. Obviously, nature has its own limits, that often reflects an analogue signal as a discrete signal that is confined within the measurement resolutions and uncertainty limits governing the physically perceptive world. Reverse engineering this aspect, any analogue signal can be formed from a series of binary signals, and thereby indicates that any decisions that seem analogue in nature can be comprised of a series of binary decisions. Such hierarchical characteristic and organization of the decisions is a primary architecture to development of intelligence systems.
Application of thresholds in solving various tasks involving decisions are very common, yet it is less acknowledged and understood. We state here the importance to look this aspect more carefully, in the sense, to understand the process of intelligence, and its application in practical applications. The applications where threshold are widely used vary from signal processing, image processing, and pattern recognition algorithms to mimicking brain logic in silicon. Through this paper, we point out the importance of thresholds and start the seminal discussion on the emerging science of threshnomics.

Threshold types
Thresholding techniques can be broadly classified as; one-sided, two-sided and multi-sided thesholding. In one-sided thresholding techniques, there will be only a single decision boundary in a plane whereas in two-sided thresholding, there will be two decision boundaries. Situation where more than two decision boundaries are involved falls into the category of multi-sided thresholding. Image segmentation can be considered as an example of a one-sided thresholding. Classifiers like SVM will fit as an example that uses twosided thresholding. Each of the broad classifications can be further divided into binary thresholds and continuous thresholding.

Threshold in algorithms
Signal processing: Thresholding is used to avoid the effects of noise [2,3] and for improving the signal to noise ratio. In multidimensional signal processing such as with images, it is used to keep useful information in the image while getting rid of unwanted things in the image. In [4], image classification accuracy has been improved by removing few pixels from inter-image similarity calculations. The decision of whether a pixel needs to be accommodated for similarity calculations was determined by thresholding the pixel-to-pixel distances. The presence of noise on images not only affects the quality but also affects the efficiency of other image processing operations done on the image. Removing noises is another useful application of thresholding.
The idea of thresholds is also used in clustering methods [11][12][13][14]. These methods set thresholds for indicating a boundary or limit on the degree of closeness. In density-based clustering methods [15][16][17], they follow growing a given cluster until the density in the neighborhood exceeds some predefined threshold. Traditional clustering approaches generate partitions which are disjoint because of the hard clustering done based on thresholds. There is another form of clustering called fuzzy clustering [18][19][20][21] in which a pattern is associated with every cluster based on a membership function. From this kind of fuzzy or soft clustering, a hard clustering can be obtained by using a threshold on the membership value.
In pattern recognition algorithms, selection of features is considered important as they are very crucial in determining the accuracy. The right selection of features is also a major concern when trying to improve accuracy of algorithms working on large volume data. Feature selection can also be made efficient by the use of thresholding techniques [22][23][24].
Image processing: In edge detection algorithms, the input images are usually smoothed with a low-pass filter to reduce the effects of noise. However, for determining which pixels should be discarded as noise and which should be retained, threshold based criterions are set. Segmentation of images [25][26][27] are also essentially supported by thresholds. While some segmentation methods use fixed threshold values to detect abrupt changes in pixels, some methods use adaptive thresholding techniques. Otsu's thresholding [28] is one such method that uses an adaptive method for choosing a threshold value.

Threshold in circuits
As opposed to algorithmic applications, the idea of thresholds in circuits has the primary purpose of modelling the intelligent systems by mimicking the electrical functionality of neurons. The ability to design electronic circuits that that mimic the human intelligence is a forefront research questions in this century. The use of threshold logic is leads this research by relation the neuronal firing as an important event in the existence and origins of intelligence. It has to be noted that there do exist another line of thinking where intelligence is related to the connectivity between the neurons, while this is definitely an important idea, without the firing mechanism it would be practically impossible to develop a system that would mimic the functionalities of intelligence as understood through the science and observation of human brain functions.
Threshold logic unit: The output of a Threshold Logic Unit is calculated in two stages. Firstly, based on the input pattern, activation is calculated. Secondly, the activation is supplied to a threshold function to get the final output. The activation a is the weighted sum of its inputs as: where w i is the i-th element of the weight vector and x i is the i-th element of the input vector. The output obtained, y when the activation is supplied to the threshold function is given as: where h is the threshold value. If the value of the activation a is greater than or equal to h then the output of the TLU will be 1 otherwise it will be 0. The input given to a TLU at the same time is called an input pattern. A decision boundary is a boundary where the activation value becomes equal to the threshold value.
Neuron Model: Neurons are the basic processing elements in the human brain. There are about 100 billion neurons in a human brain. The structure of a neuron is shown in Figure 2. It has got three main parts in it namely; cell body, dendrites and axon. The cell body controls all activities of the neuron. It functions like a thresholding unit inside a neuron. Dendrites extend out of the cell body and are responsible for getting input from other neurons. Messages from the cell body are transmitted to other neurons through a long fiber called as axon. The neurons are also covered by a protective covering called as myelin sheath. Messages travel as electrical impulses within a neuron, while but between neurons messages travel different functional mechanisms. The transfer of message between two neurons is done through the release of neurotransmitters into the space between the axon and the dendrites. Basically, the neurotransmitters are chemical substances and the communication phenomenon between two neurons is electrochemical. The space between the axon and dendrite is called as a synapse. The message receiving neurons have receptors located at the dendrites. The receptors are responsible for receiving and processing the message.
Neuron networks: Artificial Neural Networks (ANNs) [29] is an information processing paradigm inspired from the functioning of human brain. In other words ANNs can be considered as an attempt to mimic the human brain functionality. However, we can only implement limited functionalities in a neural network and can be configured only for some specific application such as data classification or pattern recognition. Learning in biological systems includes adjustments in synaptic connections between neurons. The same concept is implemented in a neural network as well. Figure 3 show how a biological neuron can be mapped to an artificial neuron. An artificial neuron can have many inputs and can have only a single output. In the training phase, the neuron can be works as a comparator and determines which bank of capacitor is having higher number of logic 1 inputs. Two CMOS inverter type amplifiers are connected with OTA. The inverters are used in this circuit to obtain output and its compliment at the same time higher gain and speed is achieved with these outputs. Analog switches connect digital inputs with capacitor banks. A threshold logic function is implemented with the inputs having positive weights connected to one bank, and the inputs having negative weights are connected to the other bank. In case of both banks have an identical number of high level inputs, one additional half capacitor Ca 0, Cb 0 unbalances the voltage levels at the amplifier inputs. BCTL works from one clock cycle and switches the gate into two phases Reset Phase and Evaluation phase.

Neuron MOSFET:
The idea behind Neuron MOS (νMOS) was introduced in 1966 [32]. It has a buried floating polysilicon gate and a number of input gates capacitive coupled to the floating gate. The floating gate voltage becomes a weighted sum of the voltages on the input gates and controls the current in the transistor channel [33,34]. νMOS simulates the function of neuron and can be used to construct artificial neural networks. If the floating gate potential is smaller than the threshold voltage of the transistor, then transistor is off. When this Floating gate potential exceeds threshold voltage of the transistor it will turn on. This is the working principle of νMOS transistor ( Figure 6).

Self-time threshold logic:
The main feature of this circuit is the use of shared capacitors among all threshold gates of a single network, for computing weighted sum of all inputs. This capacitor sharing reduces the area consumption. This self-timing idea is derived from asynchronous circuits. A cross coupled transistor pair is the main building block of this logic gate circuit (Figure 7). trained to fire (or not to fire) for specific input patterns. This can be considered as a setting of proper threshold of the neuron. When the neuron is in use mode, the input pattern which does not belong to the taught list of patterns can be isolated and be identified as a different input pattern. Here also, the concept of thresholding comes to a great use. It may be noted that there exist a wide range of literature in the area of neuromorphic circuits and there has been substantial attempts in understanding and realising neuron networks as electrically and functionality close as biological neurons. However, the scalability of such neuron networks in silicon is a challenging problem and there does not seem to have a concrete solution that would match the complete network abilities of human brain.

Threshold logic circuits
Capacitive Threshold-Logic (CTL) Gate: This CTL architecture consists of a poly-to-poly capacitor array and an inverter chain ( Figure  4). These Poly-to-poly capacitors are available in all standard mixedmode CMOS technologies [30]. This gate operates within two phase non-overlapping clocks consists of a reset phase and an evaluation phase. At the reset phase the row voltage reset the threshold voltage v th . During the evaluation phase binary input signals are enter in to the columns and row voltage is perturbed from the reset level v th . Boolean functions performed by the capacitive threshold logic gate is soft programmable. This ability is achieved by adjusting the threshold with a DC voltage. In the asynchronous mode of operations capacitive threshold logic gates are directly cascadable. CTL concept can be applied in the parallel counter and serial/parallel multiplier design.
Balanced capacitive threshold logic gate: BCTL gate is based on CTL and latch type comparator logic [31]. The requirement of reference voltage is eliminated by implementing a zero threshold ( Figure 5). Any logic function can be converted into an equivalent threshold logic function with threshold equal to zero by inverting certain inputs and changing the sign of their associated weights. Circuit consists of two banks of capacitors and both of this capacitor banks are connected with an operational trans-conductance amplifier (OTCA). This amplifier    The transistor pair generates the output Q and its complement Q ' after buffering the two inverters [35]. This gate operates in two phases precharge and evaluate phase. These phases are specified by the dual enable signals E and its complement E b . When the first gate reaches the precharge phase, then the subsequent gates are also in the state of precharge phase. After this precharge phase outputs of the first stage are evaluated then all subsequent outputs are also evaluated in successive stages. In this self-timed fashion the outputs of each gate propagates through each stages.

CMOS capacitor coupling threshold logic:
Capacitor coupling technique and a current sense amplifier is the main part of this logic [34]. In the low and high clock stage transistors becomes in the ON and OFF stages. There is no offset cancellation mechanism and by using the differential configuration mechanism it can compensate the device parameter fluctuations.
Charge recycling threshold logic: It is developed on the basis of charge recycling asynchronous sense differential logic [36]. The applied inputs are capacitive coupled onto the floating gate of first transistor and the gate voltage of second transistor (Figure 8). The output y and its compliment y ' are generated by using the sense amplifiers. Precharge and recycling stages are determined by signal E and its compliment E ' [36]. The sense circuit is activated by the enable signal E. A charge recycling threshold logic gate has two operational phases evaluate and equalize. When enable signal compliment E i becomes high the output voltages are equalized.

Current Mode Threshold logic (CMTL):
The main advantage of using this circuit is its high performance and low power consumption [37]. For resulting high performance selecting transistor configuration which can sense a small difference in current and also set the differential outputs to the correct values. Reduced power dissipation is achieved by limiting the voltage swing on the interconnects and the internal nodes of the CMTL gates. PMOS based CMTL gate received low swing inputs, which are sensed by CMTL gate and performs the logic computations. The full-swing outputs are used as inputs to the nMOS based interconnect driver. This basic circuit face a problem of static short-circuit power dissipation in the equalize phase and at the end of evaluate phase. This can be overcome by adding two additional PMOS transistors on the path from Vdd to the output nodes. This will result in the generation of two different transistor configurations discharge current mode threshold logic and equalized current mode threshold logic.
Latch type threshold logic gate (LCTL): CMOS current controlled latch is the main component of this circuit. This latch provides both the output Y and its complement Y ' . Circuit also consists of two input arrays having an equal number of parallel input transistors. Two transistor pairs specify the precharge and evaluate phase, and transistors M 4n+1 / M 9n+1 guarantee correct operation. Precharging occurs when the reset signal Φ R =0 and evaluation begins when Φ R =1. The speed performance of LCTL can be increased by substituting additional NMOS banks external to the latch [38]. This will avoid the long feedback chain .This method of circuit implementation is called as Cross-couple Inverters with Asymmetrical Loads Threshold Logic.

CMOS solutions:
The early forms of CMOS solutions were developed in 1975 [39,40]. nMOS and pMOS are structurally similar, so it can produce similar switching delay. CMOS solutions for majority functions specified the representation of logic function as a majority function. This is achieved by complimenting or repeating its inputs. There is a tradeoff between large fanin and speed. Another CMOS solution is static null convention logic. This CMOS solution assigns two states for each input asserted state and null state. The inputs are in asserted state and null state based on the threshold value.

Split-Precharge Differential Noise-Immune Threshold Logic Gate (SPD-NTL):
It is a low-power differential threshold logic gate [41]. This TL combines mainly two ideas split-level precharge differential logic, and noise suppression logic. Also this circuit uses two threshold logic banks f and f_bar. This f and f_bar always have transitions in opposite directions. f represents one threshold bank and f-bar represents another threshold bank with inverted inputs and changed threshold.

Memristive Logic
The idea of mimicking brain computing involves a challenging task of circuit and logic circuits to have the special ability to connect large number of devices to it inputs. Conventional logic circuits such as using the MOS devices have several limitations when it comes to device scaling, such a leakage current and second order behaviours. As opposed to these devices, a realistically new theory and a device that has growing interest in intelligent circuits in the memristor. A memristor is a two-terminal device with a variable resistance that can be used as memory for many applications like neuromorphic systems,  logic systems etc [41][42][43][44]. An example is the resistive threshold logic has been proposed to develop a universal logic cell in [44]. They have used a resistive divider circuit and a threshold logic circuit in it which is basically inspired by the established neuron modeling techniques. A memristor array and its co-existence with conventional CMOS logic gates can lead to new computing solution that require fewer areas on chip and higher speed of processing. In addition, this would also pave way to the design of large scale neuron networks. However, as opposed of CMOS logic memristive systems are expected to high higher power dissipation, and would require the development of industry standard processes of large scale manufacturing.

Discussion
A wide range of applications exists that use thresholds in different forms. The applicability of a threshold value is tightly coupled to several factors such as the nature of the problem, practical aspects of the system under consideration, and level of understanding about the decision space. It is easy understood that the arbitrary selection of threshold can lead to wrong decisions.
A true analytic general solution to threshold identification is a practically difficult task due to the wide spread nature of the problems. However, it's important to note that threshold selection is an emerging science that can lead to better automation and decision systems. The threshold can be applied different decision levels, such as local and global. The local decision space often is abstract in nature, such as, as reflected by thresholds at neurons in neuron networks. The global decision space is strict in nature, as it indicates the final outcome of the threshold operation. Nonetheless, both these levels of thresholds are useful in the development of automated systems, in as intelligent algorithms and for intelligent circuits.
To conclude, although the threshold operations itself seems trivial, the applicability and identification of optimal threshold values are not a trivial task. The broad range of science surrounds around threshold and it's of paramount interest in several applications.