The present study developed a feature selection (FS)-based decision support system using the electroencephalography (EEG) signals recorded from neonates with and without seizures. The study employed 10 different FS algorithms to reduce the classification cost by using fewer features and to improve the classification performance of the model by removing the irrelevant features. In doing so, the classification performance of each FS algorithm on each EEG channel difference was also evaluated. The dataset used in the study included EEG measurements and visual EEG annotations that were recorded from 79 term neonates. Multiple features were extracted from each channel difference using the Feature extraction (FE). Subsequently, a novel feature subset was generated for the classification using FS algorithms. The classification performance of each selected feature was assessed based on multiple criteria. The use of features extracted by the combined use of FS algorithms showed higher performance compared to the use of all features. In this study, 18 channel differences were analyzed. Better performance was achieved by using 3 of the selected 14 features or 2 of the selected features. The C4-P4 channel difference showed the highest classification performance (98.8%) among all channel differences. In the literature, FE has already been performed for the classification of the dataset used in the present study. The primary aim of the present study was to perform the same classification with the minimum number of features. The results indicated that feature reduction reduced the cost and also improved the performance of the classification. These results seem to be highly promising and thus can be used in clinical practice and shed light for future studies.