TopIntroduction
Machine learning applications are a branch of science that has proven itself and its value both academically and in the business world. These areas include customer relationship management (Mizgier et al., 2018), credit card fraud detection (Maniraj et al., 2019; Thennakoon et al., 2019), insurance and banking (Doss, 2020; Paul et al., 2021), earthquake forecasting applications (Beroza et al., 2021; Johnson et al., 2021; Jordan & Jones, 2010), logistics and supply chain management (Hazen et al., 2014; Treiblmaier & Mair, 2021; Waller & Fawcett, 2013) or to model, forecast the trend of the pandemics (Alamo et al., 2020; Ray et al., 2020). Thanks to deep learning methods, high-level machine learning applications such as image & video processing and speech recognition have become possible. In deep learning applications, more parameters such as the number of neural network layers, regularization method, training parameters are used compared to classical machine learning applications. Models with different parameter values will have different performance metrics such as classification (accuracy, precision, recall, F1-score) and regression (mean squared error, mean absolute error). ML researcher's experience is essential for tuning the parameters, as well as many trial-and-error processes. Most of the time, the automatically adjusted parameters will work worse than the parameters tuned by machine learning experts. Again, the very first machine learning trial is tried with the default parameters. This stage is just the beginning phase and does not make sense in terms of performance metrics. A better option is needed by trying different parameter combinations instead of the default parameters. In other words, the processes of training a data set and establishing a model can be seen from a general point of view of machine learning studies. In short, the data enters the model, and the result of classification or clustering is revealed by the model. So, the process can be shown in a simple way as in Figure 1. As a result of a detailed examination of the ML, the complex structures of machine learning applications will be revealed.
Figure 1. Machine Learning as a Black Box
One of the most important goals of machine learning studies is to optimize specified performance metrics. Performance metrics vary depending on whether the problem is a classification, clustering, or regression problem. Some metrics can be given as; confusion matrix, precision, ROC-AUC curves, accuracy score for classification, mean squared error (MSE), mean absolute error (MAE), R2 score for regression analysis. Machine learning engineers work for the model to achieve the best learning (best performance metric). After the pre-preprocessing phase, the available data are divided into training and test sets. In the pre-processing data stage, statistical analyzes are made about the data, dirty data is cleaned, and feature selection is made. Learning is performed on the training set, and the test set shows how generalizable the learning can be. One of the challenges encountered at this stage is underfitting or overfitting. For learning to be accepted, such difficulties must be overcome by cross-validation, adding more relevant data, working with more feasible features, and regularization. These types of situations are complex parts of machine learning. The complexity in learning is explained in terms of bias and variance. Bias shows how well the algorithm fits the data. On the other hand, variance measures the consistency in the algorithm's performance in different data sets. The relevance between overfitting and underfitting and bias and variance is shown in Figure 2.
Figure 2. Overfitting and underfitting with low and high biases