Axl Ibiza presents · A Kaggle competition story
A love story. A learning curve. A completely avoidable iceberg.
TITANICMy Model Will Go On
I gave a small dataset a very large ego.
Then I gave it 23 chances to reconsider.
“It’s been 84 years.”
It was twelve months. There were 23 experiments.
Emotionally, she’s correct.
Based on my original Kaggle notebook, its saved code and outputs. Historical experiment labels and published version numbers are different systems. The website analysis is dated September 5, 2026; the illustrated Kaggle continuation and official scoring receipt were added September 6.
“I’m the king
of the world!”
— me, before evaluating on unseen data.
There are 891 labeled passengers in the training data and 418 passengers whose outcomes Kaggle scores. The task sounds modest: predict a zero or a one. Naturally, I responded by developing a feature-length relationship with the dataset.
The early work happened in R: inspect missingness, impute Age, transform Fare, encode categories, and try random forests. The retrospective begins at 0.52870, with a numeric target mistakenly treated as regression. Better preprocessing takes the recorded score to 0.76076.
The important romance is between a method and its evidence. Changing the algorithm’s outfit does not guarantee a better date: factor encoding and multinomial logistic regression both record 0.75358.
Draw me like one of
your decision trees.
Retrospective V4: a modest ensemble briefly becomes the love of my life.
My score will go on.
In several directions.
Retrospective records
score_history table. The low first score is included. The dashed line marks retrospective V4. Kaggle’s later 0.81100 badge is a separate observation.God himself could
not overfit
this ensemble.
The ensemble: hold my validation set.
V4’s retrospective three-model blend reaches 0.78947. The reasonable response would be to ask which parts are helping. The cinematic response is to add decks.
Deep learning. Two-level stacking. Pseudo-labeling. Twenty seeds. Surgical rules. Python ports. Support vectors. The whole cast assembles, and most of them proceed to underperform the simpler reference.
The Advanced Hybrid is the blockbuster: 39 features, eight models, optimized blending weights and a tuned decision threshold. Its reported CV exceeds 86%. Its recorded leaderboard score is 0.74401.
That is 4.546 percentage points below V4. The ship has acquired every luxury except generalization.
V4 · simple ensemble
Consensus · majority vote
Advanced Hybrid · impressive wreckage
These are three selected historical approaches, not a controlled experiment isolating model count. Architecture, features and tuning also changed. The evidence supports this failure story; it does not prove that more models always perform worse.
“I’ll never let go,
training data.”
Immediately lets go of the test set.
The door has a
regularization budget.
The retrospective rescue starts from V4’s predictions and reduces low-confidence positive calls. V4 records 154 predicted survivors. Strategy 1: 152. Strategy 2: 149. Final 2: 147.
The corresponding scores improve from 0.78947 → 0.79425 → 0.79665 → 0.80143. In that sequence, restraint helps. But predicting fewer survivors is not a general scoring trick: which passengers change matters.
The notebook proposes a lower hidden-test survival rate. That remains a hypothesis. A prediction count is not the true count, and repeated reactions to the leaderboard can themselves overfit.
Fewer survivors.
Better score?
The passenger
manifest.
Every one of the 23 recorded approaches, including the bad ideas, near misses and late rescues. Open a row for the method, result and what the evidence can actually tell us.
Dates and scores below reproduce the V16 retrospective. They are not a private Kaggle submission-log export. Download the experiment CSV ↗
23 experiments aboard
01v1Basic RF0.52870146
The maiden misclassification
A basic random forest opened the retrospective at 0.52870. The notebook diagnoses a numeric Survived target that sent caret into regression mode, followed by thresholding continuous predictions.
The archived first run does contain regression warnings and 146 positive predictions. But the published history also attaches 0.52870 to a later v2.1 title. The retrospective attribution is preserved, not silently rewritten. Also: 52.87% is above a 50% coin flip, despite the original prose saying otherwise.
The model packed for a regression cruise. The itinerary said classification.
Original score-history source ↗02v2.0Better preprocessing0.76076150
A brief, dangerous optimism
Better preprocessing lifted the retrospective score to 0.76076. Cleaning, missing-value handling and appropriate encodings mattered more than a new algorithm.
The saved version named Version-2.0_Good_Score-0.76315 contains a different score in its title. That title is not substituted for this table entry; the two records remain separately inspectable.
Jack has discovered preprocessing. Everyone on the bow is unbearable.
Original score-history source ↗03v2.2Factor encoding0.75358148
A more formal wardrobe
The factor-encoding attempt reports 0.75358 with 148 positive predictions. Categorical representation changes how an R model treats the features.
This falls 0.718 percentage points below the preceding 0.76076 entry. An encoding that looks cleaner is not automatically a better predictor.
First-class factors. Third-class generalization.
Original score-history source ↗04v2.3Multinomial LR0.75358148
The logistic love triangle
Multinomial logistic regression also reports 0.75358 and 148 positives. The linear classifier changes the model family without improving the recorded score.
Equal score and equal positive count do not prove identical passenger-level predictions. The score table cannot recover which individual decisions changed.
A new leading man. Exactly the same box office.
Original score-history source ↗05v3.0Rule-based overrides0.76076155
The captain knows a shortcut
The retrospective describes manual rules based on sex, class, age and family outcomes. Its table records 0.76076 and 155 positive predictions.
The surrounding prose instead says 0.76555 and calls that worse than 0.76076. Both the value and direction conflict with the table. This page uses the literal table for charts and exposes the conflict in the audit.
Domain knowledge has seized the wheel. The iceberg has declined to comment.
Original score-history source ↗06V4 (Champion)Simple 3-model ensemble0.78947154
Three models, one lifeboat
The retrospective champion combines shallow XGBoost, ranger random forest and glmnet elastic net. It describes title/family features, a surname-plus-fare family survival signal, simple probability averaging and a 0.5 threshold.
It records 0.78947 with 154 positives. The retrospective V4 is not the same object as published notebook Version 4 or the January archive titled v4.0, which contains a single random forest.
For one shining moment, the orchestra is playing in the right key.
Original score-history source ↗07V5Equal weights0.76555160
Democracy at sea
The V5 row is labeled Equal weights: 0.76555, with 160 positives. It is a recorded ensemble variation that underperformed the V4 reference.
Because the V4 narrative also describes an unweighted average, the available retrospective does not fully specify what distinguished V5. Lost calibration is the author’s proposed explanation, not an isolated experimental proof.
Every model gets a vote. Buoyancy remains unelected.
Original score-history source ↗08V6 (Deep Learning)Neural Network0.77511158
Deep learning. Deeper water.
The retrospective describes a 12-input multilayer perceptron with hidden layers 64 → 32 → 16, ReLU, sigmoid output, dropout 0.3 and L2 regularization. Its recorded score is 0.77511.
That is 1.436 percentage points below V4. Separately, the January archived nnet attempts use five hidden units and fail at package loading; those failed executions are not this scored retrospective experiment.
The network has hidden layers. The ship has flooded ones.
Original score-history source ↗09V9 (Stacking)2-level stacking0.77272162
Stacking the deck
V9 combines XGBoost, random forest, logistic regression, SVM and KNN beneath a logistic meta-learner. The narrative describes out-of-fold base predictions.
The score is 0.77272, below V4 by 1.675 percentage points. The result is real as a historical record; it does not establish that stacking universally fails on small datasets.
We put a model on the models. The extra deck did not improve flotation.
Original score-history source ↗10V10 (Pseudo-label)Semi-supervised0.75837165
Believe your own publicity
V10 uses confident predictions on unlabeled rows as pseudo-labels to expand training. It records 0.75837 with 165 positives.
If provisional labels are wrong, treating them as observations can reinforce the original mistake. That is a plausible failure mechanism here, although the retrospective does not provide a controlled ablation.
The ship asked itself whether it was sinking. The ship said no.
Original score-history source ↗11V11 (Seed Avg)20-seed averaging0.78708151
Twenty alternate timelines
V11 averages across 20 random seeds, seeking stability against randomness. It records 0.78708 and 151 positives.
It misses the V4 score by 0.239 percentage points—approximately one correctly classified row on a 418-row evaluation. That tiny gap is not strong evidence that seed averaging is harmful in general.
In nineteen universes the door is still too small.
Original score-history source ↗12V13 (Surgical)Surgical rules0.78468158
Surgical, in the nautical sense
V13 adds targeted rule adjustments to repair apparent errors. It records 0.78468 and 158 positives.
It stays close to V4 but does not beat it. Rules suggested by training mistakes or repeated leaderboard feedback need fresh validation before they can be called improvements.
We have repaired the hull by drawing a more flattering hull.
Original score-history source ↗13V15 (Python WCG)WCG post-processing0.76076155
Women, children, and group statistics
The Python WCG attempt applies family/ticket rules inspired by Chris Deotte’s Women-Children-Groups approach. The retrospective records 0.76076 and 155 positives.
Published Version 15 actually runs XGBoost with 2,000 estimators and applies nine WCG overrides. Its executable implementation and output are preserved in the archive.
The evacuation policy now has a groupby clause.
Original score-history source ↗14Python EnsembleSoft voting0.78229156
A soft-voting romance
The Python Ensemble row records soft voting: averaging model probabilities before deciding the class. Its score is 0.78229, with 156 positive predictions.
It trails retrospective V4 by 0.718 percentage points. A language migration also changes library implementations, encodings and defaults; it is not automatically an equivalent reproduction.
Same love story. Different runtime. Slightly colder ending.
Original score-history source ↗15Advanced Hybrid39 features, 8 models0.74401189
The unsinkable ensemble
Advanced Hybrid brings 39 features and eight models: XGBoost, LightGBM, CatBoost, random forest, RBF SVM, KNN, L1 logistic regression and extra trees. The narrative adds Optuna blending and threshold search.
The recorded 0.74401, with 189 positives, trails V4 by 4.546 percentage points. The narrative reports CV above 86% and a chosen threshold of 0.32; that gap warrants a leakage and selection audit rather than a claim that complexity alone caused it.
Eight models entered. None had read the iceberg’s terms of service.
Original score-history source ↗16Approach AV4 Python port0.72488165
The remake nobody ordered
Approach A tries to port V4 into Python. It records 0.72488 with 165 positives, a drop of 6.459 percentage points from the retrospective R reference.
The retrospective does not expose this attempt’s complete independent implementation. Numerical equivalence between the two languages cannot be assumed from the name V4.
The remake has the same ship and somehow a worse ending.
Original score-history source ↗17Approach BSVM ensemble0.73684164
A support vector to cling to
Approach B adds an SVM ensemble. It records 0.73684 and 164 positive predictions.
This is better than Approach A but still 5.263 percentage points below V4. The table does not specify its complete kernel configuration or blend weights.
The support vector is not rated as a flotation device.
Original score-history source ↗18Approach C10-seed Python0.77033166
Ten more chances to miss the boat
Approach C uses 10-seed averaging in Python. It reports 0.77033 and 166 positives.
Averaging recovers ground relative to Approaches A and B, while still missing V4. Its 93.3% match rate with V4 is reported in a separate eight-model comparison.
Reproducibly damp.
Original score-history source ↗19Approach DError analysis0.75598158
An autopsy with a submission button
Approach D follows error analysis, recording 0.75598 and 158 positives.
The surviving record gives the method label rather than a complete algorithm. Matching the positive count of stronger attempts does not guarantee matching their predictions or score.
Cause of death: attempting to fix the test set with a hunch.
Original score-history source ↗20Consensus5-way majority vote0.78468158
The committee goes down together
Consensus takes a five-way majority vote, scoring 0.78468 with 158 positives.
It outperforms the weaker Python variants but remains below V4. Voting can reduce some errors while preserving errors the models share. The exact member predictions are not in the historical table.
The band has formed a subcommittee.
Original score-history source ↗21Strategy 1V4 + fare filter0.79425152
Two fewer passengers on the prediction door
Strategy 1 is labeled V4 + fare filter. It records 152 positives and 0.79425, versus V4’s 154 and 0.78947.
That is a 0.478 percentage-point improvement, equivalent to about two additional correct rows. A smaller positive count alone does not identify the correct people.
The door’s capacity is now a hyperparameter.
Original score-history source ↗22Strategy 2Ultra-conservative0.79665149
The very selective rescue
Strategy 2 becomes ultra-conservative: 149 positives and a score of 0.79665.
Its 0.240 percentage-point gain over Strategy 1 is about one net correct row. Fewer positive predictions and higher score co-occur here, but this local sequence does not prove a universal relationship.
Rose would like to see your probability calibration before making room.
Original score-history source ↗23Final 2Maximum conservative0.80143147
The last recorded lifeboat
Final 2 reaches 0.80143 with 147 positives in the retrospective, up 1.196 percentage points from V4.
The later saved V16 execution starts at 144 and flips seven to reach 137. Kaggle’s V16 badge reads 0.81100. Those are three different observations, so this table stops where its own source stops.
The model will go on. The provenance will come with it.
Original score-history source ↗No experiments match. Even the wreck has its limits. Clear the search or change the language.
THE DELETED SCENESThe 16 earlier published notebook versions +
The actual saved versions tell another part of the story, including five failed executions, a Naive Bayes detour and chronology that differs from the retrospective. Numbering here is Kaggle’s published sequence. Times use America/Chicago.
Version 1
R random forest with modeled Age imputation, log Fare, HasCabin and FamilySize. Numeric Survived triggers regression warnings; saved output has 146 positive predictions.
Read code & saved output ↗Version-2.0_Good_Score-0.76315
A one-line change in the published diff. The title records 0.76315. That title conflicts with the retrospective v2.0 score of 0.76076.
Read code & saved output ↗Version-2.1_Poor-Performance_Score-0.52870
A quick-save revision titled “I broke it” in the linked GitHub filename, with 0.52870 in its Kaggle title. Quick-save outputs may be retained from an earlier execution.
Read code & saved output ↗Version 2.2
Categorical factor handling, title/deck work and random-forest preprocessing in R. Linked to the author’s original v2.2 GitHub commit.
Read code & saved output ↗v2.2
Another saved run of v2.2. The published diff is +0/−0 and the recovered code is identical to Version 4.
Read code & saved output ↗v2.3 - Multinomial Logistic Regression
Multinomial logistic regression via caret/multinom, with cross-validation and categorical predictors.
Read code & saved output ↗v3.0 - New Title Feature w Random Forest Model
Title extraction joins the random-forest feature set. The saved run completes in 140 seconds.
Read code & saved output ↗v3.1 Neural Network
An nnet model with five hidden units, decay 0.1 and 200 iterations is written, but execution fails loading nnet: the existing namespace is imported by ipred and Hmisc.
Read code & saved output ↗v3.3 Neural Network
A second neural-network run with a +0/−0 code diff. It encounters the same nnet namespace-unload failure.
Read code & saved output ↗v3.3 Neural Network
The next neural-network revision changes the submission-writing code. The saved run still fails at the package boundary.
Read code & saved output ↗v3.3 Neural Network
Another neural-network repair attempt. The version history reports failure after 44 seconds; the archived output retains the nnet namespace error.
Read code & saved output ↗v4.0
The January v4.0 adds Ticket GroupSize, FarePerPerson and ChildInFamily, fits a single random forest with 10-fold CV, and reports feature importance. This is not the retrospective three-model V4.
Read code & saved output ↗Version 13
A Naive Bayes model using e1071, with an 80/20 split. Saved holdout accuracy: 0.7486034. This is a local validation result, not a Kaggle score.
Read code & saved output ↗Version 14
The first archived Python XGBoost/WCG attempt stops immediately with ModuleNotFoundError: no module named pandas. A failed runtime has no model score.
Read code & saved output ↗Version 15
The Python run succeeds: 891 × 11 training features, 418 × 11 test features, 2,000-tree XGBoost, then nine WCG overrides based on surnames and tickets.
Read code & saved output ↗Ultimate Titanic Meta-Analysis
The full retrospective and 23-row score history, plus executable 11-feature Python ensemble. Saved output has 144 base and 137 final positives; Kaggle displays 0.81100.
Read code & saved output ↗81.1% is strong.
100% needs
an explanation.
My notebook’s current best public score is 0.81100. On 418 rows, that corresponds to 339 correct predictions and 79 errors. A single additional correct row moves accuracy by about 0.239 percentage points. Tiny leaderboard gaps are tiny numbers of people.
Titanic’s historical outcomes are publicly recoverable. Kaggle’s own rules say that looking up the answers defeats the purpose. A perfect CSV assembled from those outcomes demonstrates answer retrieval, not prediction on unseen cases.
Repeatedly changing a model based on the same leaderboard also spends its independence. Research on adaptive leaderboard overfitting explains why a higher score can become a less reliable account of generalization.
There is no demonstrated 81.1% statistical ceiling. A stronger valid model is possible. Nor does one number prove misconduct by a particular entrant. What deserves credit is a reproducible method and credible validation—not an unexplained perfect ending.
Illustrative 95% Wilson interval:
77.07%–84.56%
Computed from 339/418 under an independent Bernoulli sampling model. Related passengers and repeated model selection weaken that assumption. This interval is an illustration of uncertainty, not a ceiling, a rank estimate, or a paired test against another model.
The Heart of the Ocean was valuable.
The answer key was already on the internet.
The wreck has
excellent documentation.
80.143% in the story.
81.100% on Kaggle.
The retrospective ends at Final 2. The public V16 badge is higher. The saved run starts with 144 positives and flips seven, producing 137—not the story’s 147. We preserve all three records without inventing a missing submission.
V4 is not Version 4.
The archive’s January 28 v4.0 is a single R random forest. The retrospective calls its December three-model champion V4. Published Version 13 is Naive Bayes; retrospective V13 is surgical rules. These labels cannot be used interchangeably.
Family labels crossed
the validation boundary.
V16 builds FamilySurvived from the whole training set before cross-validation. Excluding a passenger’s own row still allows validation relatives’ outcomes to influence the features. Preprocessing and target-derived features must respect the split.
Correlation depends
on who stays aboard.
The complete 23-row history gives r = +0.070. Excluding the first attempt gives −0.491. The original −0.73 claim is not reproduced from that table. Positive prediction counts do not reveal hidden ground truth.
Send a rescue model.
With a validation plan.
A fresh evaluation rebuilt all imputations and the surname/fare feature inside each fold. Five prespecified candidates—logistic regression, random forest, XGBoost, extra trees and a simple three-model average—competed on the 712-row development partition. Selection was frozen before opening the 179-row holdout. The sex-only rule provides a baseline.
| Model | Labels before split | Fold-local features | No family feature | Groups kept together |
|---|---|---|---|---|
| Logistic regression | 83.39% | 83.16% | 81.26% | 80.25% |
| Random forest | 84.51% | 84.51% | 83.05% | 81.03% |
| XGBoost | 85.52% | 84.74% | 84.62% | 80.36% |
| Extra trees | 84.62% | 84.74% | 83.05% | 80.36% |
| Simple ensemble | 84.51% | 83.84% | 83.61% | 81.48% |
| Sex-only baseline | 78.68% | 78.68% | 78.68% | 78.68% |
The controlled XGBoost comparison drops from 85.52% to 84.74% when the family-label boundary is repaired. The effect varies by model; the random forest is unchanged. Keeping connected surnames and tickets together produces 80.36% for XGBoost and 81.48% for the simple ensemble. Learning about unseen families is a harder question than predicting another member of a known family.
These diagnostics were run after the contender was selected; they did not choose a new winner. The reserved holdout was separated for the website evaluation, then reused by the illustrated notebook. Its rows were historically available in the original training data; the rerun is not a newly untouched evaluation. The completed Kaggle submission is reported above, separately from these local diagnostics. It provides a clearer account of what the score means.
The scene
the trailer left out.
97 correct negatives · 46 correct positives.
13 false positives · 23 false negatives.
| Predicted 0 | Predicted 1 | |
|---|---|---|
| Actual 0 | 97 | 13 |
| Actual 1 | 23 | 46 |
Evaluation method, data sources & limitations
Five prespecified model candidates; development-only five-fold selection; locked 20% holdout (seed 804); separate full-data leakage, ablation and grouped diagnostics. Fixed >0.5 threshold. No leaderboard queries or hidden outcomes used. Holdout rows are freshly withheld for this run, not historically untouched by the original notebook.
Dataset: the agconti public mirror of the competition CSVs. The direct Kaggle data endpoint requires authentication. Counts, ID ranges, 342 training survivors and missingness were verified; byte-for-byte parity with a fresh authenticated Kaggle download remains unverified. No external test outcomes were retrieved or used.
The contender uses a fixed 0.5 threshold, 220 shallow XGBoost trees, learning rate 0.04 and L2 regularization 3. It retains the family feature only with fold-local labels. No passenger-specific manual overrides or leaderboard tuning were used.
- One holdout split; no claim of a statistical maximum or superiority on Kaggle.
- Family/ticket grouping is a conservative proxy and can group unrelated surnames.
- Leakage experiment isolates reference-label contamination in a new controlled implementation; it does not reproduce the original dependency environment.
- Repeated diagnostic folds overlap; fold scores are not independent replications.
Original notebook authorship: Axl Ibiza, published as Andrex Ibiza, MBA. Original WCG inspiration is credited to Chris Deotte. Original notebooks are preserved under their published Apache 2.0 license.
Evaluation source code ↓
My model will go on.
My certainty won’t.
A good score is a result. Knowing what it does—and doesn’t—prove is the work.
Original illustrations and cinematic parody. No affiliation with the film or Kaggle. All historical experiment values remain linked to their source.