Comparative Performance Evaluation of Five Machine Learning Models for Diagnosing Blood Diseases from Microscopic Images

Full text

Text extracted from the article PDF for reading and search. Tables, figures and equations are in the PDF, which remains the authoritative version.

1. Introduction

1.1. Background

Microscopic examination of stained blood smears remains the reference method for diagnosing leukemia and malaria in many hospitals. It is slow, depends on the experience of the microscopist and is hard to scale where specialists are few. Automated classification of single cells promises a fast second reader. Acute lymphoblastic leukemia (ALL), the most common childhood cancer, requires distinguishing immature lymphoblasts from normal B-lymphocytes that look very similar; malaria diagnosis requires detecting Plasmodium parasites inside red blood cells.

1.2. Research Gap

Recent studies (Table 1) report accuracies of 95–99% and AUC above 0.99 on the same two public datasets used here [1-4, 8-15]. Most do not state how images were split, and those that do split images rather than patients [5, 14]. Because cells of one patient share staining, illumination and morphology, an image-level split lets a model recognise the patient instead of the disease. Leakage of this kind inflates accuracy by up to 18 points in histopathology [16], ultrasound [17] and neuroimaging [18], but its size has not been measured on blood-smear benchmarks. Comparisons between handcrafted features, CNNs and vision transformers are also usually run with one seed and without significance tests, so small differences are reported as rankings.

Table 1: Recent related works (2024–2026). Split: train/test split level as stated by the authors; n/r = not reported. # Study Task / data Method Result Split 1 Mohan et al.

ALL vs normal B-cells; C-NMC, 10,661 images DenseNet-121 + ResNet-34 deep features fused, classified by SVM/KNN/RF/AdaBoost/BPN Acc 92.5%; F1 93.1%; AUC 0.975; t-test and Wilcoxon p<0.05 n/r 2 Jammal et al.

ALL vs benign; C-NMC 15,135 images / 118 patients; ALL Image 3,256 images / 89 patients CNN features + ViT contextual encoding + fuzzy reasoning; Grad-CAM, SHAP, IG, LIME Acc 0.9715 C-NMC (SGD), 1.0000 ALL Image; 0.5002 on C-NMC with Nadam n/r

3 Jammal

ALL vs normal; ALL Image; C-NMC Meta-Conformer-XAI: CNN + ViT dual-attention fusion, meta-learned path controller Acc 0.9636 C-NMC, 0.9924 ALL Image; AUC > 0.99 on both n/r

4 Hendrawan et

ALL vs healthy cells; C-NMC 2019 (size n/r)

Pretrained ResNet-50 features + ViT self-attention (hybrid)

Acc 98% (abstract seen via search index only; verify from full text) n/r

5 Ashikuzzaman

Leukemia vs normal; Kaggle leukemia dataset, 10,700 images 14 pretrained CNNs (EfficientNet-B1/B3/B5, ResNet, Xception, DenseNet, VGG); Grad-CAM, LIME; McNemar Acc up to 96%; F1 up to 97%; AUC up to 0.987 (mean of 10 seeds) image 6 Manalı et al.

Blood cancer (WBC) classification; Public Kaggle WBC dataset (size n/r)

Healthy vs lymphoblast vs myeloblast; ALL-IDB + Munich AML Morphology VGG16 / InceptionV3 / ResNet50 features + RF, SVM, XGBoost, MLP Best Acc 88% (InceptionV3 + SVM); ResNet50 overfitted n/r 8 Boit et al.

NIH malaria, 27,558 images EDRI hybrid multi-architecture deep model Acc 97.68% n/r 9 Mujahid et al.

Malaria: infected vs uninfected; RBC smear images (n/r in abstract)

EfficientNet vs other pretrained models; k-fold CV Acc 97.57% n/r

10 Benachour et

Five CNN variants (basic to augmented, VGG16 transfer learning)

Simple CNN Acc > 99%, beating deeper models n/r 11 Shahin et al.

Thin-smear cell images (n/r in abstract)

Malaria: infected vs uninfected; NIH malaria 27,558 images (+ NM-AIST 3,544)

Track A: CNN features + calibrated classic classifiers; Track B: end-to-end CNNs; weighted pairwise ensembles Acc 96.35% on held-out data (ensemble) n/r 14 Dehbi et al.

Blood smear images (n/r in abstract)

ViT-L/16 + EfficientNet-B0 feature fusion, SHAP selection, boosting ensemble Acc 95.61% n/r 16 Nogueira et al. (2026)

Data leakage (oral cancer histopathology); Public oral cancer histopathology dataset 11 CNN and Transformer models, leakage-free vs two leakage scenarios Sample-related leakage inflated Acc by up to 18% (mean +14.3%) image vs leakage-free 17 Wang (2026)

Patient-level validation (breast ultrasound); BUS-UCLM, 264 images / 36 patients 7 CNNs + ViT + ResNet18-ViT fusion; patient-level 5-fold CV vs image-level Patient-level best AUC 0.67; image-level split substantially inflated results patient 18 Young et al.

(2025) Data leakage (review); 44 studies PRISMA-ScR scoping review with a three-tier leakage-risk framework Subject-wise splits 66–90% Acc vs 95–99% for high-leakage studies patient vs leaky splits compared

1.3. Contributions

1. A controlled comparison of five model families (Random Forest, SVM, a custom CNN, transfer-learning CNNs and a vision transformer) on two diseases, with identical preprocessing, patient-disjoint splits, three seeds and significance tests.

2. A measurement of image-level leakage on C-NMC and NIH malaria, and a duplicate-image audit of C-NMC that found identical cells filed under different patient IDs. 3. A per-patient error analysis showing that the residual malaria errors are concentrated in a few patients, and an efficiency comparison on a 4 GB consumer GPU.

2. Related Work

2.1. Leukemia (C-NMC)

Mohan et al. [1] fused DenseNet-121 and ResNet-34 features and classified them with SVM, k-NN and Random Forest, reporting 92.5% accuracy and AUC 0.975 on the C-NMC training subset. Jammal et al. combined CNN features with a vision transformer and fuzzy reasoning (97.15% accuracy) [2], and later a meta-learned CNN–transformer hybrid with AUC above 0.99 [3]. Hendrawan et al. [4] joined a ResNet-50 backbone with a ViT. Ashikuzzaman et al. [5] compared 14 transfer-learning CNNs with ten seeds and McNemar tests on a Kaggle leukemia set, with an explicitly image-level split. For other leukemia data, Manalı et al. [6] compared CNNs, ViTs and hybrids under identical settings and found plain ViTs weakest, and Kasim et al. [7] reported a more modest 88% for deep features with SVM on small multiclass sets.

2.2. Malaria (NIH)

On the NIH set, reported accuracies range from 95.3% to above 99%: a multi-architecture hybrid [8], EfficientNet [9], a small custom CNN that matched deeper transfer-learning models [10], deep features with SVM and LSTM [11], CNN ensembles [12], two-track classic-ML and CNN ensembles [13], and ViT–EfficientNet-B0 fusion [15]. Dehbi et al. [14] tuned a sensitivity-oriented threshold on validation data only and stated that their image-level split could not guarantee patient independence.

2.3. Leakage in Medical Image Evaluation

Nogueira and Gomes [16] showed that sample-level leakage raised the accuracy of eleven CNNs and transformers on oral cancer images by 14 points on average; Wang [17] found that only patient-level validation gave reliable rankings in breast ultrasound; and a review of Alzheimer studies [18] reported 66–90% accuracy for subject-wise splits against 95–99% for leaky ones. Table 1 summarises the works.

3. Materials and Methods

3.1. Datasets and Leakage-Free Splits

C-NMC 2019 [19] contains segmented cells from bone-marrow smears labelled ALL or normal; we used the labelled training and preliminary-test parts (12,528 images, 101 patient IDs). The NIH malaria set [20] contains 27,558 segmented red cells from 200 patients (150 infected, 50 healthy); the patient is encoded in each file name. A SHA-1 hash of the decoded pixels of every image revealed 110 images in C-NMC that occur under two different patient IDs; 12 IDs share identical cells and were merged with a union– find step into fewer subjects (93 in total), keeping one copy of each image. No duplicates were found in the malaria set. Subjects were split 70/15/15 into training, validation and test parts, stratified by subject class, with seed 2026. An automatic check confirms that no subject and no pixel hash occurs in two parts (fingerprint of splits.json: 2c8efd575bae). Healthy malaria cells also come from infected patients' slides, so a patient-level split separates slides, not only diseases. Table 2 gives the sizes.

3.2. Preprocessing

Every image is resized to 224 × 224 pixels, enhanced once by contrast-limited adaptive histogram equalisation (CLAHE) on the L channel of the LAB colour space (clip limit 2.0, 8 × 8 tiles) [21] and smoothed with a 3 × 3 median filter. The same images feed all models. During network training, flips, 90° rotations and ±10% brightness and saturation jitter are applied on the GPU.

3.3. Handcrafted Features

For Random Forest and SVM, 135 features are extracted per image: 16-bin histograms of the H, S, V, L, a and b channels (96) and the mean, standard deviation and skewness of R, G and B (9); grey-level co-occurrence features [22] at distances 1 and 3 (12) and a uniform local-binary-pattern histogram [23] (10); and eight shape descriptors of the stained region segmented by Otsu's threshold on saturation. Features are mapped to a normal distribution by a rank-based quantile transform fitted on the training images; this scaler was chosen on validation AUC over standardisation and a robust scaler (Table 8) because several histogram bins have extreme values after standardisation (46 features in malaria and 40 in C-NMC with |z| > 10). Bins that are constant in the training part (17 and 27) have no effect.

3.4. Models

Random Forest [24] uses 500 trees with class-balanced weights and fixed settings. The SVM [25] uses an RBF kernel with C ∈ {1, 10, 100} and γ ∈ {scale, 0.01, 0.001} chosen on validation AUC; its score is the decision function. The custom CNN has a stride-2 stem followed by four convolution blocks (32–64–128–256 filters, two 3 × 3 convolutions with batch normalisation each) and global pooling. EfficientNet-B0 [26], ResNet-50 [27] and DeiT-Small/16 [28] are ImageNet-pretrained models from the timm library [29] with a new two-class head; all layers are fine-tuned. We count the two transfer-learning CNNs as one family, giving five families and six models (Table 3).

Table 3: Models compared. Parameters of the networks from the trained models. Model Family Input Parameters Configuration Random Forest Classic ML 135 features — 500 trees, balanced class weights SVM Classic ML 135 features — RBF, C × γ grid on validation Custom CNN CNN from scratch 224 × 224 RGB 1.2 M stride-2 stem + 4 conv blocks EfficientNet-B0 Transfer-learning CNN 224 × 224 RGB 4.0 M ImageNet, fine-tuned ResNet-50 Transfer-learning CNN 224 × 224 RGB 23.5 M ImageNet, fine-tuned DeiT-Small Vision transformer 224 × 224 RGB 21.7 M ImageNet, patch 16, fine-tuned

3.5. Training

Networks are trained with AdamW [30] (learning rate 1e-4, 1e-3 for the custom CNN, weight decay 0.05) and a cosine schedule for at most 20 epochs on C-NMC and 10 on malaria, with cross-entropy weighted by inverse class frequency and early stopping on validation AUC (patience 5). Batch sizes are 64 (custom CNN), 32 (EfficientNet-B0), 32 (DeiT-Small) and 16 (ResNet-50). All runs use full fp32 precision on an NVIDIA GTX 1650 SUPER (4 GB), because mixed precision was 2.5–5 times slower on this card without tensor cores. Each model is trained with seeds 0, 1 and 2; the SVM is deterministic and gives the same model for every seed. Training can stop and resume from a per-epoch checkpoint; resumed runs are not bit-identical to uninterrupted ones, so replication rests on the three seeds, and their wall-clock training time is excluded from the time table. Random Forest is fitted on all CPU cores, the SVM on one; handcrafted features are extracted on one core.

3.6. Evaluation

Disease is the positive class. We report accuracy, sensitivity, specificity, F1, balanced accuracy and AUC on the test part, as mean ± standard deviation over seeds, at two thresholds: 0.5, and the Youden threshold [31] chosen on the validation part of each run and then applied unchanged to the test part. Pairs of models are compared on seed 0 with the DeLong test for correlated AUCs [32] and the exact McNemar test [33] on the validation-threshold decisions, with Holm correction [34] over the 15 pairs of each task. 95% confidence intervals are obtained by resampling test patients (2000 resamples), because cells of one patient are correlated. Fig. 8 uses Grad-CAM [35] for ResNet-50 and attention rollout [36] for DeiT-Small.

4. Results

4.1. Overall Performance

On malaria (Table 5) every model exceeds AUC 0.98. DeiT-Small is highest (0.9922 ± 0.0008, accuracy 95.80%), followed closely by Random Forest (0.990, 95.41%) and ResNet-50 (0.986). Sensitivity is lower than specificity for every model (93.05% vs 98.87% for DeiT-Small); Section 4.6 shows where the missed cells come from. Fig. 2 shows the ROC curves and Fig. 3 the confusion matrices.

Fig 1: Example test cells of each class before and after CLAHE and median filtering.

Fig 2: ROC curves of the six models on the test sets (seed 0); dashed and dotted lines: handcrafted-feature models.

Fig 3: Confusion matrices on the test sets (seed 0, threshold 0.5); percentages per true class.

4.3. Handcrafted-Feature Scaling

Table 8: Feature scaler for the SVM, chosen on validation AUC (C and γ re-selected for each scaler). Test AUC is shown for transparency and was not used for the choice. Task Scaler C γ Val. AUC Test AUC C-NMC Standardisation 10 0.01 0.9115 0.8876 Robust (5–95%, clipped) 10 scale 0.9106 0.8926 Quantile (normal) 1 0.01 0.9181 0.8757 NIH malaria Standardisation 100 0.001 0.9803 0.9155 Robust (5–95%, clipped) 10 0.01 0.9829 0.9524 Quantile (normal) 10 scale 0.9858 0.9804 With standardised features the SVM reached only 0.9155 test AUC on malaria, far below Random Forest on the same features; the quantile transform, which had the best validation AUC in both tasks, raised it to 0.9804 (Table 8). On C-NMC the same choice lowered the test AUC from 0.8876 to 0.8757; we keep it because choosing a scaler per task on test results would itself be leakage.

4.4. Training and Inference Cost

Fitting Random Forest takes seconds, but extracting the handcrafted features takes 3.1 minutes for the C-NMC training set and 19.8 minutes for malaria on one core, which also limits their inference to 15–43 images per second (Table 9). The custom CNN is the fastest model at inference (662 images/s on malaria) and DeiT-Small the slowest network (142 images/s) and the longest to train (50.1 minutes on malaria). Fig. 5 relates AUC to throughput; Fig. 4 shows the training curves Table 9: Training time per run (mean over the seeds that were not resumed; ResNet-50 on C-NMC from 2 seeds, DeiT-Small on C-NMC from 2 seeds) and inference cost. Networks: GTX 1650 SUPER. RF and SVM: feature extraction on one CPU core (parallelisable) is included in the total and in the inference cost; the RF fit uses all cores, the SVM fit one. Task Model Feature extr. (min) Fit (min) Total (min) Images/s Latency (ms) C-NMC Random Forest 3.1 0.0 3.1 43 75.0 SVM 3.1 0.5 3.6 42 23.8 Custom CNN — 7.1 7.1 303 1.4 EfficientNet-B0 — 22.3 22.3 298 7.6 ResNet-50 — 25.2 25.2 239 6.5 DeiT-Small — 30.1 30.1 175 7.6 NIH malaria Random Forest 19.8 0.1 19.9 15 118.3 SVM 19.8 1.3 21.1 15 67.4 Custom CNN — 9.4 9.4 662 1.4 EfficientNet-B0 — 28.7 28.7 423 7.8 ResNet-50 — 43.0 43.0 224 6.5 DeiT-Small — 50.1 50.1 142 7.5

4.5. Image-Level Leakage and Ablation (Single Runs)

Splitting the same images by image instead of by patient raised the C-NMC AUC of EfficientNet-B0 from 0.880 to 0.974 and its accuracy from 76.20% to 92.16% (Table 10, Fig. 6). Random Forest gained +0.065 AUC, so leakage affects shallow models on handcrafted features as well as deep networks. The inflated values are close to those reported for C-NMC in recent studies [1]– [3]. On malaria the gain is smaller (0.983 → 0.995) because performance is already near the ceiling. Removing CLAHE lowered the C-NMC AUC of EfficientNet-B0 from 0.8805 to 0.8190 but had no adverse effect on malaria (0.9867 vs 0.9830); removing augmentation lowered the C-NMC AUC to 0.8550. These variants were run once and indicate direction, not precise effect sizes.

Fig 4: Training loss and validation AUC per epoch of the four networks (thick: seed 0; thin: seeds 1–2).

Fig 5: Test AUC (mean ± sd over seeds) against inference throughput (log scale).

4.6. Per-Patient Analysis of Malaria Errors

The 22 malaria test patients with parasitized cells contribute 2,313 positive cells. 95 of them are missed by all six models, and 75 of these come from one patient, C116 (Table 11, Fig. 7), whose 414 parasitized cells are detected with only 60– 75% sensitivity, against a median of 97% for the other patients (DeiT-Small). The three worst patients hold 83% of these cells. When five model families with different inductive biases agree on the same errors, the cause is more likely the labels or the staining of that slide than the models; the sensitivity ceiling on this dataset is a data ceiling. We kept the patient in the test set.

Table 11: Malaria test patients with the most parasitized cells missed by all six models (seed 0, threshold 0.5); sensitivity in %. Patient Cells RF SVM CNN EffB0 R50 DeiT Missed by all C70 229 99.1 99.1 99.1 99.1 99.1 99.1 2 Cells: parasitized cells of the patient in the test set; R50 = ResNet-50, EffB0 = EfficientNet-B0.

Fig 7: Sensitivity of each model for every malaria test patient with parasitized cells (seed 0).

4.7. Explainability

Fig. 8 shows Grad-CAM maps of ResNet-50 and attention rollout of DeiT-Small for test cells chosen by a fixed rule (for each class, the first two cells in file order that both networks classify correctly and the first cell that ResNet-50 misclassifies). For correctly detected parasitized cells both models concentrate on the stained parasite, and for correctly detected lymphoblasts on the interior of the cell, where chromatin texture differs. The errors are informative: the missed parasitized cell shows no visible parasite, consistent with a doubtful label, and the uninfected cell called parasitized carries a stained fragment at its edge, on which both models focus. Grad-CAM shows evidence for the disease class and is normalised per image, so on negative cells, where that evidence is weak, it highlights background noise; rollout maps are coarser (14 × 14 patches) and spread over the whole cell or its border.

Fig 8: Grad-CAM (ResNet-50, evidence for the disease class) and attention rollout (DeiT-Small) on test cells selected by a fixed rule.

5. Discussion

Interpretation: Under a patient-disjoint protocol, the gap between model families depends on the task. Detecting a parasite is a colour-and-shape problem that 135 handcrafted features solve almost as well as a vision transformer; Random Forest is within 0.003 AUC of DeiT-Small and makes as few errors at a validated threshold. Separating lymphoblasts from normal lymphocytes depends on subtle chromatin texture, and there the pretrained networks gain about 0.04 AUC over handcrafted features, while a CNN trained from scratch does not.

Leakage:Reported C-NMC results of 92.5–97% accuracy and AUC up to 0.99 [1]– [3] are approached in our experiments only when images are split instead of patients. We therefore read published C-NMC results without a stated patient-level split as optimistic by roughly 0.07–0.09 AUC. The duplicate cells filed under different patient IDs show that even a patient-level split needs a pixel-level audit.

Limitations: The C-NMC test part has only 14 subjects, so its confidence intervals are wide and the ResNet-50 versus DeiT-Small comparison cannot be settled. The Youden thresholds are estimated on validation parts of 14 and 30 subjects and transfer imperfectly. Statistical tests use seed 0 only. Ablation and leakage variants are single runs. The SVM scaler was changed after the first review of the results (Section 4.3), always on validation evidence. Healthy malaria cells come from the slides of infected patients, so the task is cell classification, not patient diagnosis. All timings are for one consumer GPU and a single desktop CPU, with feature extraction on one core.

6. Conclusion and Future Work

On two public blood-smear benchmarks evaluated with patient-disjoint splits, three seeds and significance tests, pretrained ResNet-50 and DeiT-Small lead on leukemia (AUC about 0.91) without a significant difference between them, while on malaria a Random Forest on handcrafted features is nearly as good as the best network at a fraction of the model size. Image-level splitting inflates leukemia AUC by up to 0.09, and the remaining malaria errors cluster in a few patients. Future work will add more patients and external test sites, multiclass leukemia subtypes and anemia morphology, calibration of decision thresholds, and whole-slide (patient-level) diagnosis.

Data and Code Availability Both datasets are public [19, 20]. The preprocessing, split, training and evaluation code, the patient-level splits (fingerprint 2c8efd575bae) and all result files are available from the author [TODO repository link].