# How to approach Feature Extraction and Feature Selection part in machone learning in python?

I am a bit new to machine learning and I have the following questions:

Question 1:

When dealing with feature extraction with signals from sensors, what is the typical approach to extract features from different domains, for example:

• Time-Domain
• Frequency Domain
• Time-Frequency Domain

Because when I look at my approach, I feel that what I am doing is wrong. Can someone inform me about the standard way of extracting features from a signal? I don't mind if you can provide a code with sample data that represents a signal. I just need to fully understand how a person can approach this section in machine learning.

For example:

Say I have around 20 files that contain data collected from 3 sensors. Every 5 files represent a certain case, say, hand posture as a class.

My approach:

for fp in DataPathList:
print('Opened file number: {}'.format(fp))
print('Size of dataset matrix:', dataset.shape)
print('Number of sensors in the file:',dataset.shape[1])
file_number += 1
for column in range(0, dataset.shape[1]):
# Getting the time period and frequency:
S_F = # Sampling Frequency value
file_Sensor_number += 1

N = dataset.shape[0]
S_T = 1 / S_F
t_n = S_T * N  # seconds of sampling

# Obtaining data in order to plot the graph:
y = dataset[:,column]
x = np.linspace(0, t_n, N)
dt = x[1] - x[0]
print('Sample Frequency = ', S_F)
print('Sampling Period = ', dt)
print('Number of Samples = ', N)

SNR = signaltonoise(y)
print('Signal-to-Noise Ratio (SNR): ', SNR, 'dB')

## Signal Processing ##
#Implement any Signal Processing needed

## Feature Extraction ##
time_domain_features = get_time_domain_features_final(y_norm)
time_domain_features_np_shape = time_domain_features.shape
print('Time Domain Features Extracted from sensor {0}: '.format(column_no), time_domain_features)
print('Time Domain Features Shape Extracted from sensor {0}: '.format(file_Sensor_number), time_domain_features_np_shape)


Feature Extraction Part

# Definitions for time signal feature statistics:
def get_time_domain_tsfresh_features(raw_signal, signal, next_signal, normalize, signal_length, filter_lvl, lag):
## Time Domain FE (Separately)
energy = stEnergy(frame=signal)
abs_energy = tsfresh.feature_extraction.feature_calculators.abs_energy(x=signal)
power = (1 / (2 * signal_length) + 1) * energy
abs_sum_of_chg = tsfresh.feature_extraction.feature_calculators.absolute_sum_of_changes(x=signal)
cid_ce = tsfresh.feature_extraction.feature_calculators.cid_ce(x=signal, normalize=normalize)
time_rev_asym_stat = tsfresh.feature_extraction.feature_calculators.time_reversal_asymmetry_statistic(x=signal,lag=lag)
c3 = tsfresh.feature_extraction.feature_calculators.c3(x=signal, lag=lag)
zc = tsfresh.feature_extraction.feature_calculators.number_crossing_m(x=signal, m=0)
mean = tsfresh.feature_extraction.feature_calculators.mean(x=signal)
mean_abs_change = tsfresh.feature_extraction.feature_calculators.mean_abs_change(x=signal)
mean_change = tsfresh.feature_extraction.feature_calculators.mean_change(x=signal)
mean_second_derivative_central = tsfresh.feature_extraction.feature_calculators.mean_second_derivative_central(x=signal)
median = tsfresh.feature_extraction.feature_calculators.median(x=signal)
std = tsfresh.feature_extraction.feature_calculators.standard_deviation(x=signal)
variance = tsfresh.feature_extraction.feature_calculators.variance(x=signal)
skew = tsfresh.feature_extraction.feature_calculators.skewness(x=signal)
kurt = tsfresh.feature_extraction.feature_calculators.kurtosis(x=signal)
sample_entropy = tsfresh.feature_extraction.feature_calculators.approximate_entropy(x=signal, m=signal_length,r=filter_lvl)
approx_entropy = tsfresh.feature_extraction.feature_calculators.approximate_entropy(x=signal, m=signal_length,r=filter_lvl)
energy_entropy = stEnergyEntropy(frame=signal, n_short_blocks=signal_length)
mav = np.mean(np.abs(signal))
iav = np.sum(np.abs(signal))
median_abs_change = scipy.stats.median_absolute_deviation(x=signal)
std_abs_change = np.std(np.abs(np.diff(signal)))
var_abs_change = np.var(np.abs(np.diff(signal)))
pcc = scipy.stats.pearsonr(x=signal, y=next_signal)
n5 = np.nanpercentile(signal, 5)
n25 = np.nanpercentile(signal, 25)
n75 = np.nanpercentile(signal, 75)
n95 = np.nanpercentile(signal, 95)
rms = np.nanmean(np.sqrt(signal ** 2))
p2p = np.ptp(signal)
maxp = np.nanmax(signal)
minp = np.nanmin(signal)
cf = maxp / rms
kf = maxp * rms
pf = (maxp - minp) / np.mean(np.abs(signal))
mf = (maxp - minp) / (np.mean(np.sqrt(np.abs(signal)))) ** 2
mmt_1st = scipy.stats.moment(a=signal, moment=1)
mmt_2nd = scipy.stats.moment(a=signal, moment=2)
mmt_3rd = scipy.stats.moment(a=signal, moment=3)
mmt_4th = scipy.stats.moment(a=signal, moment=4)
mmt_5th = scipy.stats.moment(a=signal, moment=5)
ucl = mean + 3 * (std / np.sqrt(signal_length))
lcl = mean - 3 * (std / np.sqrt(signal_length))

## Time Domain FE (Combined)
time_domain_features = [energy, abs_energy, power, abs_sum_of_chg, cid_ce, time_rev_asym_stat, c3, zc, mean,
mean_abs_change, mean_change, mean_second_derivative_central, median, std, variance, skew,
kurt, sample_entropy, approx_entropy, energy_entropy, mav, iav,
median_abs_change, std_abs_change, var_abs_change, pcc, n5, n25, n75, n95, rms, p2p, maxp, minp, cf, kf,
pf, mf, mmt_1st, mmt_2nd, mmt_3rd, mmt_4th, mmt_5th, ucl, lcl]
##
return time_domain_features


Question 2:

After getting the features from the feature extraction part, how to approach feature selection and how to decide which method to use in order to be able to use them in pattern recognition part like classification cases? I know that there are three main methods:

• Filter Method
• Wrapper Method
• Embedded Method

And after reading several feature selection articles, I found out about a new method which is Hybrid method. So, My question here also, how to implement Hybrid method? I know that it needs to involve using both filter and wrapper method but I can't get how to implement or what to use first (Wrapper first then Filter or Filter first then Wrapper).

Question 3:

Also, when it comes to feature selection, I know before going to classification models you would need to tune models by manipulating with parameters in the model. How would you tune the models if you would use them in wrapper or hybrid method of feature selection part?

Question 4:

How can a person approach pattern recognition in order to be able to classify, for example, hand posture?

Question 5:

When dealing with feature extraction with signals from sensors, how can a person implement PCA into signals? From what I understand, PCA is used to reduce the dimensionality of a matrix and I read some articles of implementing PCA into signals from sensors, but I can't get my head around how to input the signal data into PCA as it will be 1D data and output is going to be only one principal component.