Chapter9 Adaptive Filters Introduction The term adaptive filter implies changing the characteristic of a filter in some automated fashion to obtain the best possible signal quality in spite of changing signal/system conditions. Adaptive filters are usually associated with the broader topic of statistical signal processing. The operation of signal filtering by definition implies extracting something desired from a signal containing both desired and undesired components. With linear FIR and IIR filters the filter output is obtained as a linear function of the observation (signal applied) to the input. An optimum linear filter in the minimum mean square sense can be designed to extract a signal from noise by minimizing the error signal formed by subtracting the filtered signal from the desired signal. For noisy signals with time varying statistics, this minimization process is often done using an adaptive filter. For statistically stationary inputs this solution is known as a Wiener filter.1 + d[n] e [ n ] Error Σ Desired Signal x[n] Observation Signal Wiener Filter y[ n] MMSE Estimate of d[n] 1. Simon Haykin, Adaptive Filter Theory, fourth edition, Prentice Hall, 2002. ECE 5655/4655 Real-Time DSP 9–1 Chapter 9 • Adaptive Filters Wiener Filter • An M tap discrete-time Wiener filter is of the form y[n] = M–1 ∑ wm x [ n – m ] (9.1) m=0 where the w m are referred to as the filter weights – Note: (9.1) tells us that the Wiener filter is just an M-tap FIR filter • The quality of the filtered or estimated signal y [ n ] is determined from the error sequence e [ n ] = d [ n ] – y [ n ] • The weights w m , m = 0, 1, …, M – 1 are chosen such that 2 2 E{e [n]} = E{(d[n] – y[n]) } (9.2) is minimized, that is we obtain the minimum mean-squared error (MMSE) • The optimal weights are found by setting 2 ∂ E { e [ n ] } = 0, m = 0, 1, …, M – 1 ∂ wm (9.3) • From the orthogonality principle1 we choose the weights such that the error e [ n ] is orthogonal to the observations (data), i.e., E { x [ n – k ] ( d [ n ] – y [ n ] ) } = 0, k = 0, 1, …, M – 1 (9.4) 1. A. Papoulis, Probability, Random Variables, and Stochastic Processes, third edition, McGraw-Hill, 1991. 9–2 ECE 5655/4655 Real-Time DSP . φ xx [ 0 ] (9. φ xx [ – M + 1 ] . ..6) for k = 0... . M – 1 are known as the Wiener-Hopf or normal equations – Note: φ xx [ k ] is the autocorrelation function of x [ n ] and φ xd [ k ] is the cross-correlation function between x [ n ] and d[n] • In matrix form we can write R xx w o = p xd (9. = . .5) or M–1 w m φ xx [ m – k ] = ∑ m=0 φ xd [ – k ] (9. φ xx [ M – 1 ] . .7) where R xx is the M × M correlation matrix associated with x[ n] .. …. .8) w o is the optimum weight vector given by ECE 5655/4655 Real-Time DSP 9–3 . 1.Wiener Filter – This results in a filter that is optimum in the sense of minimum mean-square error • The resulting system of equations M–1 w m E { x [ n – k ]x [ n – m ] } ∑ m=0 = E{x[n – k](d[n])} (9. R xx φ xx [ 0 ] . 13) where N is the sample block size Adaptive Wiener Filter • In an adaptive Wiener filter the error signal e [ n ] is fed back to the filter weights to adjust them using a steepest-descent algorithm 9–4 ECE 5655/4655 Real-Time DSP . that is we replace the true statistical auto.10) • The optimal weight vector is given by –1 w o = R xx p xd (9.12) (9.9) and p xd is the cross-correlation vector given by p xd = φ xd [ 0 ] φ xd [ – 1 ] … φ xd [ 1 – M ] T (9.and crosscorrelation functions with time averages of the form N–1 1 φ xx [ k ] ≅ ---N x [ n + k ]x [ n ] ∑ n=0 1 φ xd [ k ] ≅ ---N N–1 x [ n + k ]d [ n ] ∑ n=0 (9.11) • As a matter of practice (9.Chapter 9 • Adaptive Filters w o = w o0 w o1 … w oM – 1 T (9.11) can be solved using sample statistics. the error e [ n ] can be viewed as an M dimensional error surface.1] w0 w1 Error Surface for M = 2 • The filter decorrelates the output error e [ n ] so that signals in common to both d [ n ] and x [ n ] in a correlation sense are cancelled ECE 5655/4655 Real-Time DSP 9–5 . that due to the squared error criterion. is convex cup (a bowl shape) Mean Square Error Optimum at [1.Adaptive Wiener Filter • With respect to the weight vector w . we use a stochastic gradient.14) pˆ xd = x [ n ]d [ n ] (9. which is simply the instantaneous estimate of R xx and p xd from the available data. but since the true gradient cannot be determined. e.15) where x[n] = x[n] x[n – 1] … x[n – M + 1] T (9..Chapter 9 • Adaptive Filters • A block diagram of this adaptive Wiener (FIR) filter is shown below + d[n] Desired Signal x[n] Observation Σ e[ n] Error Signal y[ n] MMSE Estimate of d[n] Wiener Filter Adaptive Algorithm Adaptive Wiener FIlter Least-Mean-Square Adaptation • Ideally the optimal weight solution can be obtained by applying the steepest descent method to the error surface.g. ˆ xx = x [ n ]x T [ n ] R (9.16) • A practical implementation involves estimating the gradient from the available data using the least-mean-square (LMS) algorithm 9–6 ECE 5655/4655 Real-Time DSP . Adaptive Wiener Filter • The steps to the LMS algorithm. the step-size μ must be chosen such that 2 0 < μ < -------------------------------------tap-input power (9.21) k=0 • In theory.20) is equivalent to saying 2 0 < μ < ----------λ max (9.17) m=0 – Form the estimation error: e[n] = d[n] – y[n] (9.18) – Update the weight vector using step-size parameter μ : w ˆ [n + 1] = w ˆ [ n ] + μx [ n ]e [ n ] (9.20) where M–1 tap-input power = ∑ 2 E{ x[n – k] } (9. (9. for each new sample at time n.22) where λ max is the maximum eigenvalue of R xx ECE 5655/4655 Real-Time DSP 9–7 . are: – Filter x [ n ] to produce: y[n] = M–1 ∑ T w ˆ [ n ]x [ n ] ˆ m [ n ]x [ n – m ] = w (9.19) • For algorithm stability. Prentice Hall. Adaptive Signal Processing. 1985. B. New Jersey.Chapter 9 • Adaptive Filters Adaptive Filter Variations1 • Prediction s[n] d[n] Delay x[n] Adaptive Filter - + Σ y[n] e[n] • System Identification s[n] d[n] Plant Adaptive Filter x[n] y[n] + Σ e[n] • Equalization s[n] Training Pattern Delay Plant/ Channel Adaptive Σ + Filter x [ n ] + Noise y[n] d[n] + Σ e[n] 1. Widrow and S. 9–8 ECE 5655/4655 Real-Time DSP . Stearns. Adaptive Line Enhancement • Interference Canceling d[n] Signal + Interference Interference’ x[n] Adaptive Filter - + Σ y[n] e[n] Adaptive Line Enhancement • A relative of the interference canceling scheme shown above. is the adaptive line enhancer (ALE) • Here we assume we have a narrowband signal (say a sinusoid) buried in broadband additive noise x [ n ] = NB [ n ] + BB [ n ] + Σ - z –Δ x[n – Δ] Adaptive Filter e[n] y[n] • The filter adapts in such a way that a narrow passband forms around the sinusoid frequency. thereby suppressing much of the noise and improving the signal-to-noise ratio (SNR) in y[ n] ECE 5655/4655 Real-Time DSP 9–9 . Wo] = lms_ale(SNR.y.Chapter 9 • Adaptive Filters Example: MATLAB ALE Simulation • A simple MATLAB simulation is constructed using a single sinusoid at normalized frequency f o = 1 ⁄ 20 plus additive white Gaussian noise x [ n ] = A cos [ 2πf o n ] + w [ n ] (9.M.Wo] = lms_ale(SNR.y. 4/27/98 % % n x x % Sinusoid SNR = (A^2/2)/noise_var = 0:N.F.mu) % %*******LMS ALE Simulation************ % SNR = Sinusoid SNR in dB % N = Number of simulation samples % M = FIR Filter length % mu = LMS step-size % % n = Index vector % x = Noisy input % y = Filtered output % e = Error sequence % wo = Final value of weight vector % F = Frequency response axis vector % Wo = Frequency response of filter %************************************** % Mark Wickert. Fo/Fs = 1/20 = x + sqrt(1/2/(10^(SNR/10)))*randn(1.e.F.x. so delay x by one sample 9–10 ECE 5655/4655 Real-Time DSP .wo.e.24) function [n.N+1).N.wo. % actually length N+1 = 1*cos(2*pi*1/20*n). % Here A = 1.mu) % [n.N.23) • The SNR is defined as 2 A SNR = ---------22σ w (9. White Noise -> Delta = 1.M.x. end %end loop on time index k % Create filter frequency response [Wo.1.5]) subplot(212) plot(n.M).Adaptive Line Enhancement xd = filter([0 1].z] = filter(wo. % Update the weight vector wo = wo + 2*mu*e(k)*xdm.1. % Initialize a vector for holding xd of length M xdm = zeros(1. % Initialize error vector e to zero e = zeros(1.N+1).e..01/64).x(k).^2) subplot(211) plot(n. % Update vector used for correlation with e(k) xdm = [xd(k) xdm(1:M-1)].x) axis([600 1000 -1.N+1).e.64. and μ = 0. % Initialize output vector y to zero y = zeros(1. plot(n.z).y.F. SNR = 10 dB.y(k).x.x).1.Wo] = lms_ale(10.M-1).1000.01 ⁄ 64 » » » » » » » » [n.F] = freqz(wo.wo. % Form the error sequence e(k) = x(k) . % Initialize filter memory to zero z = zeros(1.Wo) ECE 5655/4655 Real-Time DSP 9–11 .y) plot(F. for k=1:N+1 % Filter one sample at a time [y(k). % Initialize weight vector to zero wo = zeros(1. M = 64. • A simulation is run using 1000 samples.512. Wo = 20*log10(abs(Wo)).1).M).5 1. 5 600 650 700 750 650 700 750 800 850 900 950 1000 800 850 900 950 1000 1 y[n] 0.01/64 1. mu = 0.5 0 -0.5 1 x[n] 0.5 0 -0.5 -1 -1.5 -1 600 Index n 9–12 ECE 5655/4655 Real-Time DSP .Chapter 9 • Adaptive Filters Convergence Occurs in Here (~275 samples) ALE x[n] & y[n] for SNR = 10 dB. 2 0.1 0.4 0.3 0.15 0.01/64 5 0 -5 Sinusoid Frequency -10 |Wo(f)|dB -15 -20 -25 -30 -35 -40 -45 0 0. for SNR = 10 dB.5 Normalized Frequency f • A C version of the above MATLAB code would be very similar except all of the vector operations would have to be replaced by for loops • TBD C6x Code Examples A Two Channel Input Signal + Signal or Signal + Noise Canceller • In this first example a modified version of Chassaing’s Adaptnoise. Resp.25 0.05 0. mu = 0.35 0.c program is considered ECE 5655/4655 Real-Time DSP 9–13 .C6x Code Examples ALE Freq.45 0. in a real application the input x [ n ] would be similar to signal2 as seen in d [ n ] . The union allows the data to be accessed as a single entity when transferring to and from the serial port.Chapter 9 • Adaptive Filters • In this program the following system is implemented d[n] Signal1 + Signal2 (or noise) Signal2 x[n] Adaptive Filter y[n] + Σ e[n] • Notice that signal2 is present at both inputs in the exact same form. & Morrow. // Real-time Digital Signal Processing. but not exactly the same • Floating point C code is shown below: // Welch. but still be able to manipulate the left and right channels independently. // // // // Data is received as 2 16-bit words (left/right) packed into one 32-bit word. 2011 // Modified by Mark Wickert February 2012 to include GPIO ISR start/stop postings /////////////////////////////////////////////////////////////////////// // Filename: Adaptive_ISRs.c // // Synopsis: Interrupt service routine for codec data transmit/receive // /////////////////////////////////////////////////////////////////////// #include "DSP_Config. Wright.h" // Function Prototypes long int rand_int(void). #define LEFT 0 #define RIGHT 1 9–14 ECE 5655/4655 Real-Time DSP . float beta = 1E-11. } CodecDataIn.//alpha from slider interrupt void Codec_ISR() /////////////////////////////////////////////////////////////////////// // Purpose: Codec interface interrupt service routine // // Input: None // // Returns: Nothing // // Calls: CheckForOverrun. // so serial port is reset to recover CodecDataIn. noise=0. E=0. //weights for adapt filter float delay[N]. dplusn=0.// get input data samples /* add your code starting here */ desired = (float) CodecDataIn. //input buffer to adapt filter short output. //overall output short out_type = 1. // Write to GPIO J15.Channel[ LEFT].//rate of convergence #define N 64 //# of weights (coefficients) #define LEFT 0 //left channel #define RIGHT 1 //right channel float w[N]. ReadCodecData.e. CodecDataOut. desired=0. float yn=0.Channel[ RIGHT].//scale beta by slider alpha value if(CheckForOverrun())// overrun error occurred (i. beta *= alpha.C6x Code Examples volatile union { Uint32 UINT.//output type for slider float alpha = 10. pin 6. //input right chan ECE 5655/4655 Real-Time DSP 9–15 . halted DSP) return. WriteCodecData // // Notes: None /////////////////////////////////////////////////////////////////////// { /* add any local variables here */ WriteDigitalOutputs(1). begin ISR timing pulse short i. Int16 Channel[2]. /* add any global variables here */ //#define beta 1E-10. //input left chan noise = (float) CodecDataIn.UINT = ReadCodecData(). // Real-time Digital Signal Processing.// send output data to port WriteDigitalOutputs(0). //update delay samples } if (out_type == 1) //if slider in position 1 { output = (short) E. } // Welch. end ISR timing pulse } //White noise generator for filter noise testing long int rand_int(void) { static long int a = 100001. i >= 0. i < N.UINT).Channel[ LEFT] = output. //update weights delay[i] = delay[i-1].//overall output result CodecDataOut. // Write to GPIO J15. a = (a*125) % 2796203. //noise as input to adapt FIR for (i = 0. } //to calculate output of adaptive FIR E = (desired + noise) . return a. Wright.//desired+noise } CodecDataOut. i++) { yn += (w[i] * delay[i]).yn. & Morrow. //"error" signal=(d+n)-yn //output of adaptive filter for (i = N-1.Chapter 9 • Adaptive Filters dplusn = desired + noise.Channel[RIGHT] = 0. i--) //to update weights and delays { w[i] = w[i] + beta*E*delay[i]. 2011 /////////////////////////////////////////////////////////////////////// // Filename: main.c // 9–16 ECE 5655/4655 Real-Time DSP . //error signal as overall output } else if (out_type == 2) { output = (short) dplusn. /* end your code here */ WriteCodecData(CodecDataOut. pin 6. //desired+noise delay[0] = noise. } } // Welch. T++) { w[T] = 0. } //init variables //init weights of adaptive FIR //init buffer for delay sampless } ECE 5655/4655 Real-Time DSP 9–17 .h" #define N 64 //# of weights (coefficients) extern float w[N]. delay[T] = 0. & Morrow. // Real-time Digital Signal Processing. Wright. T < N.c // // Synopsis: Adaptive filter initialize // /////////////////////////////////////////////////////////////////////// #include "DSP_Config. // initialize filter weights and delay states DSP_Init(). interrupts drive operation while(1) { . int T = 0.0. 2011 /////////////////////////////////////////////////////////////////////// // Filename: StartUp.//input buffer to adapt filter void StartUp() { InitDigitalOutputs(). // main stalls here.0.h" int main() { // call StartUp for application specific code // defined in each application directory StartUp().//weights for adapt filter extern float delay[N].C6x Code Examples // Synopsis: Main program file for demonstration code // /////////////////////////////////////////////////////////////////////// #include "DSP_Config. for (T = 0. one at 1 kHz as the desired signal and one at 2. the filter history in delay[n] is also updated in this loop GEL Slider to toggle between dplusn and the error E GEL slider to set the value of coefficient beta between 1E-12 and 1E-10 • A second GEL slider controls which filter output goes to the left codec channel • This adaptive filter was tested with two sinusoid inputs.Chapter 9 • Adaptive Filters • The left channel input (desired) is added to the right channel input (noise) and forms the LMS input d [ n ] (dplusn) • The right channel input by itself forms the input x [ n ] (noise) • A direct form FIR filter is implemented using floats in the first for loop • In the second for loop the filter tap weights w[n] are updated using coefficient beta which can be adjusted using a GEL file.5 kHz the noise signal • Analog outputs were captured using the PC sound card and 9–18 ECE 5655/4655 Real-Time DSP . 5 kHz Noise Power Spectrum Magnitude (dB) 30 20 10 0 −10 −20 −30 −40 −50 −60 0 500 1000 1500 2000 2500 Frequency 3000 3500 4000 Error Signal Output: Noise Removed Power Spectrum Magnitude (dB) 30 20 10 0 −10 −20 −30 −40 −50 −60 0 500 1000 ECE 5655/4655 Real-Time DSP 1500 2000 2500 Frequency 3000 3500 4000 9–19 .C6x Code Examples spectrum analyzed using MATLAB Desired 1.0 kHz Sinusoid + 2. 24.1.96} Uint32 fs=DSK6713_AIC23_FREQ_8KHZ.c program is considered • In this program the following system is implemented x [ n ] = NB [ n ] + BB [ n ] + Σ - z –Δ x[n – Δ] Adaptive Filter e[n] y[n] • Floating point C code is shown below: //Adaptpredict_AIC23. short channel[2].48.c Adaptive predictor to cancel interference //using interrupts // Enhanced version of Chassaing program // by Mark Wickert Spring 2009 //********************************************************** #include "DSK6713_AIC23. #define DSK6713_AIC23_INPUT_MIC 0x0015 #define DSK6713_AIC23_INPUT_LINE 0x0011 Uint16 inputsource=DSK6713_AIC23_INPUT_LINE.16.44. // select input //*********************************************************** // For left and right channel processing // left and right help in packed 32 bit word #define LEFT 0 #define RIGHT 1 union {Uint32 combo. 9–20 ECE 5655/4655 Real-Time DSP . // select input //Uint16 inputsource=DSK6713_AIC23_INPUT_MIC.32.} AIC23_data.Chapter 9 • Adaptive Filters Adaptive Line Enhancer • In this first example a modified version of Chassaing’s Adaptpredict.h" //set sampling rate {8. 1*((short) rand_int()). i < N. i >= 0. //desired on l //noise = (float) AIC23_data.//rate of convergence beta *= alpha. //output of adapt FIR filter //(wb+noise)-out adapt FIR for (i = N-1. //update buf corr wb } ECE 5655/4655 Real-Time DSP 9–21 .//wideband desired signal float noise. E. //delayed input to adapt FIR yn = 0.C6x Code Examples // Program globals #define N 64 //# of coefficients of adapt FIR #define Ns 10 //length splusn buffer float splusn[Ns]. //external interference short output.combo = input_sample(). //buffer for weights of adapt FIR float delay[N]. //buffer wideband signal+interference float w[N]. i++) yn += (w[i] * delay[i]).//update wgts of FIR delay[i+1] = delay[i].yn.//yn=out adapt FIR.//intern noise splusn[0] = wb_signal + noise.channel[LEFT]. //update buf delay samps if (i < Ns-1) { splusn[i+1] = splusn[i]. //wb signal+interference delay[0] = splusn[3]. i--) { w[i] = w[i]+(beta*E*delay[i]). //buffer for input to adapt FIR Uint32 out_type = 1.//alpha from slider interrupt void c_int11() //interrupt service routine { short i.//output to codec float beta = 1E-13. float yn. E = splusn[0] .//scale beta by slider alpha value AIC23_data. error signal float wb_signal.channel[RIGHT].//output type for slider float alpha = 10. //init output of adapt FIR for (i = 0. //in l&r as 32-bit wb_signal = (float) AIC23_data. //noise on r noise = 0. 0. /* if (out_type == 1)//if slider in position 1 { output = ((short)E). //init DSK.//WB signal } else if (out_type == 2) { output =((short)yn).//NB signal //output =((short)wb_signal). return. McBSP while(1). T < N.0. //init wideband+interference } } comm_intr(). codec.Chapter 9 • Adaptive Filters } //out_type = 2. output_sample(output). } void main() //main function { int T = 0. for (T = 0. T++) //init variables { w[T] = 0. //init buffer for delay samples if (T < Ns) { splusn[T] = 0.//output resultto left chan.//NB signal }*/ output =((short)yn). //init weights of adaptive FIR delay[T] = 0. //infinite loop } //16 bit word random number generator int rand_int(void) { static int a = 100001. a = (a*125) % 2796203.//WB signal //output = ((short)wb_signal). 9–22 ECE 5655/4655 Real-Time DSP . . so here Δ = 3 • A GEL file allows the left channel of the codec to output either the wideband signal at setting 1 or the narrowband signal at setting two • The adaptation coefficient is set to be 10 ECE 5655/4655 Real-Time DSP – 12 9–23 . e. a sinusoid • The inputs arrive at the left and right codec inputs and are summed in code as wb_signal + noise • An input delay buffer of Ns samples is created in the array splusn[]. } • The inputs are assumed to be a wideband signal. and a narrowband signal. here Ns = 10 • splusn[3] serves as x [ n – Δ ] . noise.g.g. e.C6x Code Examples return a.. the frequency response of the converged filter weights w[n] is displayed in CCS • The corresponding spectra of y [ n ] and e [ n ] are captured via the PC sound card The Wideband Signal Output Power Spectrum Magnitude (dB) 30 20 10 0 −10 −20 −30 −40 −50 −60 9–24 0 500 1000 1500 2000 2500 Frequency 3000 3500 4000 ECE 5655/4655 Real-Time DSP .Chapter 9 • Adaptive Filters • A 1.8 kHz sinusoid is input from a function generator and wideband noise is input from another function generator. C6x Code Examples The Narrowband Signal Output Power Spectrum Magnitude (dB) 30 20 10 0 −10 −20 −30 −40 −50 −60 0 500 1000 ECE 5655/4655 Real-Time DSP 1500 2000 2500 Frequency 3000 3500 4000 9–25 . Chapter 9 • Adaptive Filters 9–26 ECE 5655/4655 Real-Time DSP .