Add FMCW_radar_data_compression

FMCW_radar_data_compression/README.md

@@ -0,0 +1,25 @@
+# FMCW radar data compression
+
+Our project focuses on compressing FMCW radar data by exploiting the stationary nature of the analyzed signal. Indeed, the output of such systems is a linear combination of multiple sinusoidal signals and therefore it is by its nature a strongly periodic signal. For this reason, we first apply **Linear Predictive Coding** to reduce the dynamic of the signal and then, we use **Huffman coding** to further shrink the output file and improve the overall compression ratio. This folder contains the slideshow `.pdf` and the corresponding `.tex` source. Inside the code folder are stored all the files used to generate the presented results.
+
+ ## References
+
+[1] N. Levinson. “The Wiener (Root Mean Square) Error Criterion in Filter Design and
+Prediction”. In: Journal of Mathematics and Physics 25.1-4 (1946), pp. 261–278. DOI:
+https://doi.org/10.1002/sapm1946251261. \
+\
+[2] David A. Huffman. “A Method for the Construction of Minimum-Redundancy Codes”. In:
+Proceedings of the IRE 40.9 (1952), pp. 1098–1101. DOI: https://doi.org/10.1109/JRPROC.1952.273898. \
+\
+[3] J. Makhoul. “Linear prediction: A tutorial review”. In: Proceedings of the IEEE 63.4 (1975),
+pp. 561–580. DOI: https://doi.org/10.1109/PROC.1975.9792. \
+\
+[4] S.M. Kay and S.L. Marple. “Spectrum analysis—A modern perspective”. In: Proceedings of
+the IEEE 69.11 (1981), pp. 1380–1419. DOI: https://doi.org/10.1109/PROC.1981.12184. \
+\
+[5] D. O’Shaughnessy. “Linear predictive coding”. In: IEEE Potentials 7.1 (1988), pp. 29–32. DOI:
+https://doi.org/10.1109/45.1890.
+
+ ## Authors 
+ Francesco Mancuso ([mandugo](https://github.com/mandugo))\
+ Giulio Meucci ([Rustafun](https://github.com/Rustafun))

FMCW_radar_data_compression/code/FMCW_LPC_analysis.m

@@ -0,0 +1,16 @@
+function [FIRcoeffs,residual,residual_variance] = FMCW_LPC_analysis(data,FIRLEN)
+    %
+    % Linear Predictive Coding analysis for FMCW radar signal
+    %
+    % USAGE: [FIRcoeffs,residual,residual_variance] = FMCW_LPC_analysis(data,FIRLEN)
+    %
+    % Exam's project: FMCW radar data compression
+    % Course: A Crash Course on Data Compression
+    % Authors: Giulio Meucci, Francesco Mancuso
+
+    signal = double(data);
+    [FIRcoeffs,residual_variance] = lpc(signal,FIRLEN);
+    est_s = int16(filter([0 -FIRcoeffs(2:end)],1,signal));
+    residual = data - est_s;
+
+end

FMCW_radar_data_compression/code/FMCW_radar_data_compression.m

@@ -0,0 +1,50 @@
+% Exam's project: FMCW radar data compression
+% Course: A Crash Course on Data Compression
+% Authors: Giulio Meucci, Francesco Mancuso
+% 
+% This MATLAB script compresses a .wav file using Linear Predictive Coding
+% and Huffman Coding and compares the input and output file sizes by 
+% evaluating the compression ratio.
+
+
+clear all
+close all
+clc
+
+%--------LOAD FILE--------%
+[data,~] = audioread('cleanCorsa2_CUT.wav','native');
+audioinfo('cleanCorsa2_CUT.wav')
+
+tic
+
+%--------LINEAR PREDICTIVE CODING--------%
+FIRLEN = 100;
+[FIRcoeffs,residual,~] = FMCW_LPC_analysis(data.',FIRLEN);
+
+%--------SYMBOLS PROBABILITY ESTIMATION--------%
+[probabilities, edges] = histcounts(residual,'Normalization','probability','BinMethod','integers');
+nonZeroInd = find(probabilities > 0);
+p = probabilities(nonZeroInd);
+symbols = ceil(edges);
+s = symbols(nonZeroInd);
+
+%--------RESIDUAL HUFFMAN ENCODING--------%
+dict = huffmandict(s,p);
+bitstream = huffmanenco(residual,dict);
+
+toc
+
+%--------AVERAGE CODEWORD LENGTH EVALUATION--------%
+totaL = 0;
+meanL = 0;
+for ind2 = 1:size(dict,1)
+    totaL = totaL + length(dict{ind2,2});
+    meanL = meanL + p(ind2)*length(dict{ind2,2});
+end
+bitstream = dec2bin(bitstream);
+
+%--------COMPARE FILES SIZE--------%
+compressed_data_size = length(bitstream) + 64*FIRLEN + 16*size(dict,1) + totaL
+original_data_size = 16*length(data)
+
+compression_ratio = original_data_size/compressed_data_size

FMCW_radar_data_compression/code/README.md

@@ -0,0 +1,15 @@
+# FMCW radar data compression code
+
+<a href="https://imgbb.com/"><img src="https://i.ibb.co/w0hXx1q/made-with-matlab.png" alt="made-with-matlab" border="0"></a> 
+
+- `FMCW_radar_data_compression.m`: this script evaluates the compression ratio achievable using [**Linear Predictive Coding**](https://en.wikipedia.org/wiki/Linear_predictive_coding) and [**Huffman Coding**](https://en.wikipedia.org/wiki/Huffman_coding), as explained in the slideshow, onto a `.wav` file. The file used is `cleanCorsa2_CUT.wav`. The length of the LPC FIR filter is 100. After the LPC filtering the histogram of the residuals is evaluated by specifying: `'BinMethod'` as `'integers'` to set the unit-width bins centered on integers, and `'Normalization'` as `'probability'` to obtain the relative frequencies of the symbols (i.e. each bin count represents the probability that an observation falls within that bin). All the probabilities equal to zero are discarded, the Huffman dictionary is built and the symbols are encoded. The last lines of the script calculate the **average codeword length** and the **compression ratio**;
+- `average_codeword_length_opt.m`: this script has been used to analyse the compression performance by varying the length of the LPC FIR filter. The analysis has been carried out using a number of coeffients that goes from 1 to 150. The core of the script is the same as the previous one;
+  > :warning: **The execution takes a lot** :warning: 
+- `FMCW_LPC_analysis.m`: this function is used in the above scripts. It does the estimation of the LPC FIR filter coefficients, it filters the input signal and gives in output the filter coefficients, the residual signal and the variance of the residual signal.
+
+  > We used the variance of the residual signal as a quality metric during the analysis phase to understand how the LPC was affecting the amplitude of the signal.
+
+## Runtime measurements :clock130:
+The code has been run on Ubuntu 21.10, the machine has an Intel® Core™ i7-860 @ 2.80GHz and 24GB of RAM. The execution time of `FMCW_radar_data_compression.m` is about 2.15 seconds, while the execution time of `average_codeword_length_opt.m` is about 639.5 seconds.
+## 
+> Written with [StackEdit](https://stackedit.io/) and [ForTheBadge](https://forthebadge.com/).

FMCW_radar_data_compression/code/average_codeword_length_opt.m

@@ -0,0 +1,57 @@
+% Exam's project: FMCW radar data compression
+% Course: A Crash Course on Data Compression
+% Authors: Giulio Meucci, Francesco Mancuso
+% 
+% This MATLAB script compresses a .wav file using Linear Predictive Coding
+% and Huffman Coding. It changes the length of the FIR filter iteration by
+% iteration in order to find the optimum value that minimize the average
+% codeword length.
+%
+% WARNING: it has a quite long execution time.
+
+clear all
+close all
+clc
+
+%--------LOAD FILE--------%
+[data,~] = audioread('cleanCorsa2_CUT.wav','native');
+audioinfo('cleanCorsa2_CUT.wav')
+
+%--------AVERAGE CODEWORD LENGTH EVALUATION--------%
+maxLength = 150;
+FIRLEN = 1:maxLength;
+
+meanL = zeros(1,maxLength);
+totaL = zeros(1,maxLength);
+
+f = waitbar(0, 'Starting');
+
+for ind = 1:maxLength
+    [~,residual,~] = FMCW_LPC_analysis(data.',ind);
+
+    [probabilities, edges] = histcounts(residual,'Normalization','probability','BinMethod','integers');
+    nonZeroInd = find(probabilities > 0);
+    p = probabilities(nonZeroInd);
+    symbols = ceil(edges);
+    s = symbols(nonZeroInd);
+
+    dict = huffmandict(s,p);
+    bitstream = huffmanenco(residual,dict);
+
+    totaL(ind) = 0;
+    meanL(ind) = 0;
+    for ind2 = 1:size(dict,1)
+        totaL(ind) = totaL(ind) + length(dict{ind2,2});
+        meanL(ind) = meanL(ind) + p(ind2)*length(dict{ind2,2});
+    end
+
+    waitbar(ind/maxLength, f, sprintf('Progress: %d %%', floor(ind/maxLength*100)));
+end
+
+figure(1)
+plot(FIRLEN,meanL,'LineWidth',2)
+ylabel('bits')
+xlabel('FIR length')
+title('Average Codeword Length')
+grid on
+