FUSION OF BAGGING BASED ENSEMBLE FRAMEWORK FOR EPILEPTIC SEIZURE CLASSIFICATION

The ensemble learning approach, especially in classification, has been widely carried out and is successful in many scopes, but unfortunately not many ensemble approaches are used for the detection and classification of epilepsy in biomedical terms. Compared to using a simple bagging ensemble framework, we propose a fusion bagging-based ensemble framework (FBEF) that uses 3 weak learners in each oracle, using fusion rules, a weak learner will give results as predictors of the oracle. All oracle predictors will be included in the trust factor to get a better prediction and classification. Compared to traditional Ensemble bagging and single learner type Ensemble bagging, our framework outperforms similar research in relation to the epileptic seizure classification as 98.11±0.68 and several real-world datasets


Introduction
Fisher et.al [1] explain that epilepsy is a group of neurological disorder that can be characterized by epileptic seizures which result by recurrent, spontaneous and abnormal cortical nerve cell activity in the brain. Shoeb [2] also mentions that Electroencephalogram (EEG) is widely used in epileptic seizure detection because epilepsy diagnosis can be performed by identifying the abnormalities from EEG. Since manual detection by expert neurologist is time consuming, expensive and concerns about accuracy caused by fatigue, using computer aided diagnosis for epilepsy diagnosis using EEG time series data analysis can be useful and efficient solution. Epileptic seizure EEG dataset usually contains several groups of subjects such as 1) epileptic subject during seizure (ictal), 2) epileptic subject during seizure free interval (inter-ictal) and 3) healthy subject [2].
Due to EEG signals have non-stationary nature, wavelet analysis is used by many researchers because wavelet transform can give precise time information at high frequencies and frequency information at low frequencies [3], [4] and [5]. Wavelet analysis using DWT can be used to preprocess the EEG signals by decomposing it into sub-bands level signals.
By using feature selection, Alzami et.al [6] explain that there are many features that can be extracted from EEG signal decomposition, but not all features contribute to the improvement of epileptic seizure classification performance. Alotaiby et.al [7] also provide comprehensive survey that most epileptic seizure detection and classification research focus on EEG signal preprocessing such as artifact removal, signal decomposition techniques and using single classifier rather than multi classifier such as ensemble learning. Ensemble approach especially for classification is primary used for real world dataset problem rather than epileptic seizure detection and classification, some example is Ho [8] that using random subspace method for constructing decision forest (RSCE), Yu et.al [9] by using Hybrid Adaptive Classifier Ensemble (HAEL) that compare RSCE method with their research such as HAEL and singly adaptive ensemble learning (SAEL).
In order to improve the accuracy of epileptic seizure classification and to validate that ensemble approach is more robust and suitable for epileptic seizure classification, we design a fusion of bagging-based ensemble framework for epileptic seizure classification. Our contribution is two-fold: First, rather than using traditional ensemble bagging, we use fusion rules that calculate three different classifiers (weak learner) in every oracle bootstrap which can be used to improve the ensemble accuracy. Second, propose the most useful classifier fusion that can optimize and improve the bagging classifier performance for epileptic seizure classification.
The remainder of the paper is organized as follows. Section 2 describes the fusion of bagging-based ensemble framework (FBEF) methods. Section 3 experimentally investigates the performance of the FBEF methods and section 4 presents the conclusion and future works Figure 1 provides the overview of the epileptic seizure classification approach. First, we set a band-pass FIR filter from 0.5 Hz to 60 Hz to preprocess the EEG raw data and extract 5 sub-band signals such as delta, theta, alpha, beta, gamma and band limited (0-60 Hz) using DWT db4 level 4. Then, features are extracted from each sub-band signal such as: embedding-dimension, fractal-dimension, correlation dimension, kurtosis, approximate-entropy, sample-entropy, standard-deviation, max, min, median, mean, skewness, time-lag, Largest-Lyapunov-Exponent and energy. These features then implemented into several feature selection such as MDEFS [6], mRMR [10] and genetic algorithm. Finally, all those features that been selected will be used with fusion of bagging-based ensemble framework.  Figure 2 illustrates the framework of FBEF approach, specifically, after clean dataset is obtained from Figure 1, we split dataset into two parts, training set data (D) and test set data. Then, FBEF generates a set of datasets = { 1 , 2 , … , } (where B is the total number of datasets) based on original training set using bagging technique.

Methods
Then, every dataset is feed into 3 different classifiers to obtains a set of classifiers Ω{ 1 , 1 , 1 , … , , , }. Next the set of classifiers Ω are evaluated through test set using decision profile (fusion) in same oracle respectively. Prediction results in every oracle is denoted as { 1 , 2 , … , }. Finally, FBEF combines those prediction results based on a set of confidence factors Φ to obtain the final result. In here, classifier is denoted as weakLearn.

Figure 2. Fusion of Bagging-Based Ensemble Framework
We use min-max as scaling options to a fixed range because it can make standard deviations become smaller which the result is suppress the effect of outliers.
Using explanation from Xu.et.al [11], in every member of oracle Ω, every weakLearn , , will produce several Type of Information (TOI) [12], hypothesis that can be group into three levels:  Abstract level (predicted class): each weakLearn outputs the class label for each input pattern.  Measurement level (decision profile): each weakLearn outputs a posterior probability, score or confidence level for each input pattern  Rank level: each weakLearn outputs a ranking list of possible classes for each input pattern (simply said, we sort the measurement level by descend sorting).
Fusion rule based on Kittler et.al [13] is used when all weakLearn in respective oracle already produce the TOI. Let's assume we have as possible classes ( 1 , … , ), as weak learner, as measurement vector by ith weak learner, class in the measurement space can be modeled by probability density function ( | ), as predicted class and ( | ) as measurement level output. After we have all those notations, we can put into fusion rule which is:  Maximum: finds the maximum score of each class between the weakLearn using measurement level and assigns the input pattern to the class with the maximum score among the maximum scores.
 Minimum: finds the minimum of each class between the weakLearn using measurement level and assign the input pattern to the class with the maximum score among the maximum scores  Average(sum): finds the mean of the scores of each class between the weakLearn using measurement level and assigns the input pattern to the class with the maximum score among the means.
 Product: multiplies the score provided by each base weakLearn using measurement level and assigns the class label with the maximum score to given input pattern  Majority vote: using abstract level from TOI, voting method finds what is the class output of each weakLearn and count it output as a vote for a class and assigns the input pattern to the class with the majority vote After all oracle is produced the prediction result, confidence factor Φ is used to get the final result. This confidence factor can also use fusion rule or just simply use majority vote.

Experiment and Results
We used Bonn dataset from Andrzejak [14] which is referred as SZN. This SZN means we use three classes of epileptic seizure subjects: S denoted as subject during epileptic seizure (ictal), Z denoted as normal data and N denoted as during seizure free interval (interictal). We chose three classes due to it represent real world situation, which is: healthy, patient when seizure and when the patient did not get seizure. These Dataset contains subband signals as follows: delta, theta, alpha and beta. Those four sub-band signals then extracted to obtain features such as: embedding-dimension, fractal-dimension, correlation dimension, kurtosis, approximate-entropy, sampleentropy, standard-deviation, max, min, median, mean, skewness, time-lag, Largest-Lyapunov-Exponent and energy. Those 15 features are fed into MDFS feature selection, mRMR feature selection and genetic algorithm feature selection. The number of obtained features is determined from the inner operation of respective feature selection mechanisms; thus, we did not set how many features that we will use.
The proposed fusion of bagging-based ensemble framework is measured by the average accuracy on the datasets. five-fold crossover validation is used to reduce the randomness effect. The number iteration of bagging T, number bagging Ω, scaling dataset is set, respectively. We also consider to not use the scaling into dataset to understand the effect of fusion classifier The classifier combination in the experiment include k nearest neighbor classifier (kNN), the Naive Bayes classifier (NB), the decision tree classifier (DT), Discriminant analysis which use linear (LDA) and pseudoQuadratic (QDA), the support vector machine (libSVM), the levenberg-Marquardt back propagation neural network (LMBPNN).
In the following experiments, we first explore the effect of FBEF using fusion rules and 3 classifiers. Then, FBEF is compared with another epileptic seizure single classification. FBEF also compared with some ensemble framework which used the real datasets.

The Effect of FBEF Using Fusion Rules and Classifiers
In order to explore the effect of fusion rules, we compare 5 fusion rules into SZN dataset using 3 feature selection. Table 1 shows that (1) majority vote have tendency get better result because it will predict the label if at least half the number weaklearner plus 1 weaklearner give same answer.
(2) average fusion rules can also be considered because once the individual weaklearn are trained, without any further training, their output can be fused to produce ensemble decision.  Table 2 shows our method in comparison to other ensemble learning with different learning model (such as ensemble LMBPNN, ensemble SVM, etc.) can give us some insight, when dataset is scaled and using voting fusion rule, SVM can gain more better result but KNN will have slightly bad result. Good combination fusion 3 classifier is LMBPNN-SVM-LDA with scaling dataset. In here, we only used voting, max, avg, min, product. Another fusion rule should be considered and tested.

Comparison with Other Epileptic Seizure Classification
From Table 3, it is observed that our proposed approach achieved 98.11±0.68. Using standard deviation, our method is comparable with the Alzami method. By using 5 features, FBEF with 3 sub-classifiers for bagging can improve the accuracy rather than alzami method by using single classifier. The processing time of FBEF is faster than alzami method because alzami is using pattern recognition neural network and we using levenberg-Marquardt BackPropagation Feed Forward Neural network

Comparison with other Method for Real Dataset
From Table 4, it is observed that our proposed approach, is can also be used into another dataset. For the remainder, some real-world dataset cannot use QDA but Linear Discriminant Analysis, this can be happened because we didn't put any feature selection or dimension reduction into real world dataset. Thus, the features characteristic is not changed in real datasets. RSCE and SAEL is focused in random subspace. RSCE have tendency ignore the importance degree of different subspaces and classifier. RSCE also generate different random subspace set which contains set of random subspaces that will produce different combinations of classifier which will affect the final prediction result. SAEL is improved RSCE by using adaptively adjust the weight of base classifier and using weighted voting to combine predicted labels from different base classifier and obtain final result. The reason FBEF is outperformed RSCE and SAEL is 1) FBEF treated the base classifier equally, 2) FBEF do not need tuning anything and weighting but SAEL need tuning the parameters that prespecified by the user to get better accuracy and need iterations to obtain precise weight.

Conclusion and future works
This paper has proposed fusion of bagging-based ensemble framework for epileptic seizure classification. The main contribution is using multi weak learner in oracle to improve the accuracy, robustness and suitable for epileptic seizure classification. The Experiment show that epileptic seizure classification based on FBEF with weak learner LMBPNN, SVM and LDA using voting and MDEFS as feature selection get 98.11%±0.68. FBEF also suitable to be used in real world dataset because it outperforms SAEL and RSCE. From the performance on epileptic seizure dataset and real-world dataset, it can be concluded that FBEF is better than using traditional Ensemble Bagging Framework. Moreover, the combination of weak learner (classifier) take major effect in accuracy, such as worst weaklearner have tendency to lower the accuracy and cannot put the weaklearner that sensitive into scaling data together such as SVM with KNN. Furthermore, LMBPNN-SVM-LDA is the best fusion weaklearner using scale data.