alexa Design and Simulation of a Linear Prolate Filter for a Baseband Receiver | Open Access Journals
ISSN: 2165- 7866
Journal of Information Technology & Software Engineering
Make the best use of Scientific Research and information from our 700+ peer reviewed, Open Access Journals that operates with the help of 50,000+ Editorial Board Members and esteemed reviewers and 1000+ Scientific associations in Medical, Clinical, Pharmaceutical, Engineering, Technology and Management Fields.
Meet Inspiring Speakers and Experts at our 3000+ Global Conferenceseries Events with over 600+ Conferences, 1200+ Symposiums and 1200+ Workshops on
Medical, Pharma, Engineering, Science, Technology and Business

Design and Simulation of a Linear Prolate Filter for a Baseband Receiver

Sagar Soman* and Michael Cada

Department of Electrical and Computer Engineering, Dalhousie University, Halifax, NS, Canada

*Corresponding Author:
Sagar Soman
Department of Electrical and Computer Engineering
Dalhousie University, Halifax, NS, Canada
Tel: +1-902-210-7788
E-mail: [email protected]

Received Date: January 27, 2017; Accepted Date: February 07, 2017; Published Date: February 13, 2017

Citation: Soman S, Cada M (2017) Design and Simulation of a Linear Prolate Filter for a Baseband Receiver. J Inform Tech Softw Eng 7: 197. doi: 10.4172/2165-7866.1000197

Copyright: © 2017 Soman S, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Visit for more related articles at Journal of Information Technology & Software Engineering


Digital signals transmitted over a communication channel are mostly affected by noise. To reduce the detrimental effects of noise, a band-limited filter is used at the receiver, which results a phenomenon known as Inter-Symbol Interference. To avoid Inter-Symbol Interference, filters with greater bandwidth can be used. However, this causes high frequency noise to interfere with the transmitted information signal. This paper illustrates an innovative way to reduce Inter-Symbol Interference in the received baseband signal. This is achieved by making use of the processing bandwidth of a special filter designed by using Linear Prolate Functions. The result of the signal reconstruction capabilities of a prolate filter are compared with those of an ideal low pass filter in this paper.


Inter-symbol interference; Linear prolate functions; Prolate filter; Processing bandwidth; Baseband receiver


The transmission of low frequency digital data pulses with sizeable power is known as Baseband transmission [1]. Common examples of channels used for baseband communication are optical fibers, twisted pair cables and co-axial cables [2]. Noise present in a communication channel causes distortion in the system, which can be detrimental to the information content. These undesirable effects can be reduced by using a band-limited low pass filter at the receiver. The band-limited frequency response of the filter at the receiver causes digital pulses to broaden outward leading to a phenomenon known as Inter-Symbol Interference (ISI). This affects the capability of the receiver to reproduce the original signal correctly after filtering.

The aim of this paper is to reduce ISI caused by a band-limited frequency response of the filter at the receiver. Raised cosine filters [2] can be used to optimise the response of the ideal low-pass filter by fluctuating the slope of the filter’s roll off. This filter allows an excess bandwidth in the frequency domain to pass, keeping the amplitude of the side lobes of the signal in the time domain as small as possible. The smaller side lobe amplitude leads to reduced ISI interference after filtering. But, the excess amount of bandwidth transmitted with the information pulses increases the cost of the telecommunication system. This higher cost is undesirable for any telecommunication service provider. To remediate this problem, there is a need for a filtering technique that can provide an interference reducing capability with less bandwidth consumption. This paper intends to fulfill this need for a filtering technique by studying, designing and simulating a special filter termed as “prolate filter” using linear prolate functions.

In the past, researchers were not able to achieve a high precision of linear prolate functions due to limited processing power, inefficient algorithm techniques, and insufficient numerical precision. Therefore a practical exploitation of intriguing properties of linear prolate functions (see Sections II and III) in, for example, optical image and/ or signal processing applications was not feasible. We have developed a robust algorithm to evaluate linear prolate functions accurately, quickly and for all orders and frequency parameters [3]. A desirable feature of the prolate filter is that the bandwidth consumed by a prolate filter is same as the bandwidth used by a low pass filter while the interference reducing capability is similar to a raised cosine filter.

In this paper, section I introduces the problem of ISI and details the motivation behind using a prolate filter to solve this problem. Section II introduces the theory of linear prolate functions. Section III reviews various mathematical notations and properties of linear prolate functions. Section IV explains the mathematical concepts of a prolate filter. Section V describes some illustrative results of using a prolate filter as a receiver’s filter in a baseband receiver, and compares the results with those obtained with using an ideal low-pass filter. It specifies some alterations that can be done to a prolate filter in order to change its response. Section VI offers concluding remarks that summarize the advantages and outcomes of the research.

Linear Prolate Functions

There is a reciprocal relationship between time and frequency. The Fourier transform of a small instant in time is equal to an infinitely wide band of frequencies in the frequency domain, and vice versa. Every physical device has a limited bandwidth response, which causes its output to undergo severe attenuation, if a high frequency signal is applied to its input. This band limiting nature of all devices prevents a successful reconstruction of any high frequency signals. For a single instant in both time and frequency domain, if a signal is confined to a specific interval or bandwidth, it can be exactly reconstructed at the receiver. However, there are no signals that can be confined in both time and frequency domain. This problem of simultaneously confining the signal and its amplitude spectrum has been present for a long time in digital communication systems [4].

Mathematically, a measure of concentration for a signal r (t) is a fraction of the signal’s energy that lies in a particular time slot, T, which can be written as:

equation (1)

where r (t) is a bandlimited signal with unit energy such that [4]:

equation (2)

If r (t) is time-limited to an interval (-T/2 to T/2), then a2 (t) will have a maximum value of unity. Because of the inverse relationship between time and frequency, any band-limited signals cannot be time-limited. The issue of obtaining a unity value for a2(t) while keeping r (t) band-limited, was undertaken by three scientists from Bell Laboratories, namely H Pollak, H Landau, and D Slepian. The issue was resolved by developing a set of band-limited functions that were maximally concentrated in a given time interval [5]. These functions are known as Prolate Spheroidal Wave Functions [6].

By solving the Helmholtz wave equation in spherical co-ordinates, two different kinds of solutions, i.e., angular and radial, can be obtained. Out of these solutions, the prolate spheroidal angular functions of the first order, which are denoted as equation , can be used to derive a set of functions known as Linear Prolate Functions designated by equation . These functions have many advantageous properties over trigonometric functions [7]. Similarly, the prolate spheroidal radial functions of the first order equation can be used to determine the corresponding linear prolate eigenvalues designated by equation Mathematically, linear prolate functions and their corresponding eigenvalues are expressed as follows [5,8]:

equation (3)

equation (4)

An application of linear prolate functions with high precision to numerical analysis and synthesis of signals was studied for the first time in [9].

Notations and Properties of Linear Prolate Functions

In this paper, the mathematical notations and normalizations used for the prolate spheroidal wave functions are similar to those used by Flammer [6] and Slepian [5] Linear prolate functions equation are dependent on four factors [10]: the time parameter t, the time-limited interval t0, the order of the function n, and the space bandwidth product parameter c with equation , where equation is the finite bandwidth. Figure 1 illustrates some of these functions.


Figure 1: Linear prolate functions of different orders.

The properties of prolate functions were first studied by Slepian and Pollak in [5] and were analytically discussed in detail by Frieden in [8]. With regards to this paper, some of the important properties of linear prolate functions are as follows:

Invariance to fourier transform

Linear prolate functions are invariant to finite and infinite Fourier transform:

equation (5)

equation (6)

One can observe that the Fourier transform of linear prolate functions produces a scaled version of the same prolate function except for a scaling multiplier.

Eigenvalues λn of linear prolate functions

By performing a finite Fourier transform on (5) we obtain:

equation (7)

The general equation for eigenvalues and eigenfunctions is satisfied by (7), thus the eigenvalues for linear prolate functions are the same as eigenvalues of a sinc kernel function.

Orthogonality and orthonormality property

Linear prolate functions have many advantages over trignometric functions, one of which is that the prolate functions are not only orthonormal on an infinite interval but they are also orthogonal on a finite interval:

equation (8)

equation (9)


In optical imaging systems, diffraction affects the resolving capability of a system. This diffraction problem is quite similar to the ISI problem present in digital communication systems. Hence, a solution for the diffraction issue in optics, can also be applied to resolve the interference problem in digital communication systems. Several researchers such as G. Toraldo Di Francia in [11], H. Osterberg and J. Wilkins in [12], J. Harris in [13] and C. Barnes in [14] have proposed various methodologies with different shortcomings, to overcome the diffraction problem in imaging systems. B. Frieden [8] proposed a theory which overcame the shortcomings of the methodologies proposed by these previous researchers. To solve the diffraction problem, Frieden used linear prolate functions to construct a point amplitude response whose side lobes do not increase in size even when the order ‘n’ of the prolate function is increased.

It is known that the Fourier transform of a unity function is a Dirac Delta function; but this statement is not true if a finite Fourier transform is considered, as there no functions available today whose finite Fourier transform is a Delta function. However, with the help of linear prolate functions one can obtain a function whose finite Fourier transform is a Dirac delta function over a finite extent. Mathematically, one desires a function as follows:

equation (10)

Since U (λ) is only defined over a finite interval equation (10) can also be written as:

equation (11)

Substituting (11) back into (10) one finds that coefficients an are needed to satisfy a condition:

equation (12)

The completeness and orthogonality property of linear prolate functions as seen in (9) can also be written as follows:

equation (13)

By comparing (12) and (13) one can find coefficients an :

equation (14)

Substituting anfrom (14) back into (11) yields:

equation (15)

In order to evaluate (15) in practice, the upper limit of the summation is changed from 8 to M where M indicates the maximum number of terms required to obtain the required response. In this paper, M will be referred to as the threshold value. Solving (15) in Mathematical software and employing our original proprietary algorithm [3], the frequency response of a prolate filter is as seen in Figure 2.


Figure 2: Frequency response of prolate filter with M=60.

The inverse Fourier transform of (15) provides the required Dirac delta function as seen in (16) below. Employing our proprietary algorithm [3], one obtains the time domain response for the prolate filter – illustrated in Figure 3.


Figure 3: Time domain response of prolate filter with M=60.

equation (16)

The width of the main lobe for the function as seen in Figure 3 is expressed analytically as [8].

equation (17)

It should be noted that the function in Figure 3 is a true delta impulse on a finite interval. The fact that it shows a finite width is a result of the numerical limit of M=60 in this case; an infinite M does produce a true delta function.

Figure 4 compares the time domain response of a prolate filter with the time domain response obtained from an ideal low-pass filter with both filters having the same physical bandwidth:


Figure 4: Comparison of time domain response of prolate filter with ideal lowpass filter.

As mentioned earlier there are no functions available today which can provide a delta response in the time domain for a finite interval. However, by increasing the threshold value M in (17), the width of the main lobe of the sinc signal can be reduced considerably such that a delta function is obtained in the time domain. Furthermore, if a Fourier transform is performed on such a function, then a filter response can be obtained in the frequency domain which has theoretically infinite bandwidth.

The most advantageous feature of a prolate filter is the concept of a virtual bandwidth which can be extended to an infinite value even when the physical bandwidth has only a certain finite value. In this paper, this virtually or synthetically extended bandwidth is termed as ‘processing bandwidth’.

Results and Discussion

In digital communication systems, the baseband transmission of information pulses is severely affected by ISI. Generally, low-pass filters are used at the receiver to filter out the noise from the received signal. The band-limited frequency response of these filters causes the interference to occur. This problem can be avoided by using a prolate filter. Consider a digital ted in Figure 5.


Figure 5: Transmitted input signal.

In signal processing, for calculating communication system that transmits digital pulses 1001 to the receiver through a communication channel, as indicating the output of any system, the time domain analysis using convolution can be extremely complex and time consuming. Hence, in order to simplify the calculations, the frequency domain analysis is employed by performing a discrete Fourier transform operation on the input signal, which is shown in Figure 6.


Figure 6: Fourier transform of transmitted signal.

We assume for the illustration purposes that this signal is transmitted through a channel with a bandwidth of ±2π and is noiseless. At the receiver the signal is filtered by the receiver’s filter. For a better understanding of the advantages of a prolate filter, initially consider the received signal be filtered by an ideal low-pass filter having a frequency response as shown in Figure 7.


Figure 7: Frequency response of ideal low-pass filter.

In order to obtain the transmitted signal at the receiver the signal is converted back into the time domain by performing an inverse Fourier transform operation. Due to the band-limited nature of the ideal lowpass filter, valuable information needed for signal reconstruction is lost. The reconstruction of the transmitted signal at the receiver is severally affected by this problem, as seen in Figure 8.


Figure 8: Reconstructed signal using ideal low-pass filter.

The reconstruction of the transmitted signal is significantly improved by increasing the bandwidth of the ideal low-pass filter as seen in Figure 9.


Figure 9: Reconstructed signal using ideal low-pass filter with increased bandwidth.

This increase in bandwidth introduces high frequency noise into the system which defeats the major purpose of a filter and also increases the cost of hardware. Hence, in order to overcome the limitation of physically increasing the bandwidth of a low-pass filter, one can instead increase the bandwidth virtually by making use of a prolate filter’s processing bandwidth. The equation for obtaining the frequency response of the prolate filter is in (15) and the response it generates is shown in Figure 10. The physical bandwidth the prolate filter in this case is from -2π to +2π which is the same as the one used in the ideal-low pass filter.


Figure 10: Frequency response of prolate filter.

The reconstructed signal obtained by using a prolate filter is illustrated in Figure 11. One observes that it is the same as the one obtained by physically increasing the bandwidth of an ideal low-pass filter in Figure 9.


Figure 11: Reconstructed signal using prolate filter with M=40.

The response in Figure 12 can be improved further by increasing the threshold value M.


Figure 12: Reconstructed signal using prolate filter with M=60.

The prolate filter operates efficiently even with the change in transmitted signal, as shown in Figure 13 with input signal 10101.


Figure 13: Reconstructed signal using prolate filter with input signal 10101 with M=60.

For the prolate filter to operate efficiently, the value of spacebandwidth parameter c must be equal to or greater than the bandwidth of the channel under consideration. The major advantage in using prolate filters, in addition to their larger processing bandwidths, in place of low-pass filters is exactly this parameter c because it is a free parameter one can choose as necessary. In order to illustrate this, consider for comparison the use of prolate filters with c =2π, c= 10π and c 20π along with a channel bandwidth of ±20π .The time domain response obtained by using these variations of a prolate filter is illustrated in Figure 14.


Figure 14: Reconstructed signal using prolate filter with varying parameter c.

The prolate filters with c=2π and c=10π both fail to operate properly and their signal reconstruction is unsatisfactory. On the other hand, the prolate filter with c=20π operates flawlessly.


This paper provides an innovative filtering technique, which can used to reduce ISI in a baseband communication system. Linear prolate functions numerically evaluated with a high precision for large orders were used in this technique for designing the prolate filter. The major advantage of using the prolate filter is that it can theoretically provide an infinitely large processing bandwidth and is only limited by the processing algorithm as it can only include a finite number of linear prolate functions in its calculations. By extending the bandwidth of a low pass filter, its interference reducing capabilities can be improved further. However, the excess bandwidth is undesirable. Hence, the prolate filter consumes a smaller bandwidth while utilizing its processing bandwidth, to provide an interference reducing capability similar to the low pass filter with an extended bandwidth. The interference reducing capability of the prolate filter can be further improved by increasing the threshold value M of the filter.

The interference reducing capability of the prolate filter will be same as an ideal low-pass filter when the threshold value M is equal to ncritical given by equation. It needs to be noted that the actual usable processing bandwidth of the prolate filter is obtained only when the threshold value ‘M’ is well above ncritical. This fact has caused insurmountable numerical problems in the past that in fact prevented a wider use of linear prolate functions in both, image and signal processing applications. The robust algorithms developed for the first time in is now available and thus overcomes these problems, as illustrated in this paper. It can also be noted that in order to obtain the best interference reduction capability of the prolate filter, the value of the space-bandwidth parameter ‘c’ should be equal to the bandwidth of the communication channel.


Select your language of interest to view the total content in your interested language
Post your comment

Share This Article

Recommended Conferences

Article Usage

  • Total views: 812
  • [From(publication date):
    February-2017 - Sep 20, 2017]
  • Breakdown by view type
  • HTML page views : 742
  • PDF downloads :70

Post your comment

captcha   Reload  Can't read the image? click here to refresh

Peer Reviewed Journals
Make the best use of Scientific Research and information from our 700 + peer reviewed, Open Access Journals
International Conferences 2017-18
Meet Inspiring Speakers and Experts at our 3000+ Global Annual Meetings

Contact Us

Agri, Food, Aqua and Veterinary Science Journals

Dr. Krish

[email protected]

1-702-714-7001 Extn: 9040

Clinical and Biochemistry Journals

Datta A

[email protected]

1-702-714-7001Extn: 9037

Business & Management Journals


[email protected]

1-702-714-7001Extn: 9042

Chemical Engineering and Chemistry Journals

Gabriel Shaw

[email protected]

1-702-714-7001 Extn: 9040

Earth & Environmental Sciences

Katie Wilson

[email protected]

1-702-714-7001Extn: 9042

Engineering Journals

James Franklin

[email protected]

1-702-714-7001Extn: 9042

General Science and Health care Journals

Andrea Jason

[email protected]

1-702-714-7001Extn: 9043

Genetics and Molecular Biology Journals

Anna Melissa

[email protected]

1-702-714-7001 Extn: 9006

Immunology & Microbiology Journals

David Gorantl

[email protected]

1-702-714-7001Extn: 9014

Informatics Journals

Stephanie Skinner

[email protected]

1-702-714-7001Extn: 9039

Material Sciences Journals

Rachle Green

[email protected]

1-702-714-7001Extn: 9039

Mathematics and Physics Journals

Jim Willison

[email protected]

1-702-714-7001 Extn: 9042

Medical Journals

Nimmi Anna

[email protected]

1-702-714-7001 Extn: 9038

Neuroscience & Psychology Journals

Nathan T

[email protected]

1-702-714-7001Extn: 9041

Pharmaceutical Sciences Journals

John Behannon

[email protected]

1-702-714-7001Extn: 9007

Social & Political Science Journals

Steve Harry

[email protected]

1-702-714-7001 Extn: 9042

© 2008-2017 OMICS International - Open Access Publisher. Best viewed in Mozilla Firefox | Google Chrome | Above IE 7.0 version