Machine learning workflows often suffer from code duplication, data leakage, and hyperparameter tuning complexities. Scikit-Learn's Pipeline and ColumnTransformer modules simplify this by combining feature preprocessing and model estimation into an integrated, reproducible object. This guide demonstrates a complete predictive modeling workflow using the classic Titanic: Machine Learning from Disaster dataset, covering exploratory data analysis, pipeline-based feature engineering, model selection, and cross-validation. 1. Environment Setup & Data Loading Begin by importing the necessary libraries for data manipulation, visualization, preprocessing, modeling, and evaluation. import pandas as pd import numpy as np import seaborn as sns import matplotlib.pyplot as plt # Model selection & evaluation from sklearn.model_selection import train_test_split, StratifiedKFold, cross_val_score from sklearn.metrics import roc_auc_score, classification_report # Preprocessing & Pipelines from sklearn.pipeline import Pipeline from sklearn.impute import SimpleImputer from sklearn.preprocessing import StandardScaler, OneHotEncoder from sklearn.compose import ColumnTransformer # Classifiers from sklearn.linear_model import LogisticRegression from sklearn.neighbors import KNeighborsClassifier from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier from sklearn.svm import SVC from xgboost import XGBClassifier # Load dataset titanic_df = pd.read_csv("data/train.csv") 2. Exploratory Data Analysis & Data Sanitation A quick inspection of the dataset structure reveals the feature types, missing values, and potential columns to drop. def basic_checks(df): print("Shape:", df.shape) print("Missing Values:\n", df.isnull().sum()) print("Data Types:\n", df.dtypes) basic_checks(titanic_df) Missing Value Analysis Age: Missing 19.87% of data (177 missing values). Embarked: Missing 0.22% of data (2 missing values). Cabin: Missing 77.10% of data (687 missing values). Due to the extreme proportion of missing data, the Cabin column is dropped. titanic_df = titanic_df.drop(columns=['Cabin']) 3. Train-Test Split & Data Partitioning Identifiers such as PassengerId, Name, and Ticket carry no predictive power for survival and are excluded from the feature set $X$. We split the data using a 80/20 train-test split stratified by the target column Survived to maintain class proportions. # Separate features and target X = titanic_df.drop(columns=['PassengerId', 'Name', 'Ticket', 'Survived']) y = titanic_df['Survived'] # Categorize feature types numeric_columns = ['Age', 'SibSp', 'Parch', 'Fare'] categorical_columns = ['Pclass', 'Sex', 'Embarked'] # Stratified split X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42, stratify=y ) 4. Building Scikit-Learn Preprocessing Pipelines To avoid data leakage (such as calculating the mean or standard deviation on the entire dataset before splitting), transformations must be learned only on the training set. Using Pipeline and ColumnTransformer ensures these transformations are safely applied during cross-validation. # 1. Numerical Pipeline: Impute missing values with median, then scale numerical_pipeline = Pipeline(steps=[ ('imputer', SimpleImputer(strategy='median')), ('scaler', StandardScaler()) ]) # 2. Categorical Pipeline: Impute missing values with mode, then One-Hot Encode categorical_pipeline = Pipeline(steps=[ ('imputer', SimpleImputer(strategy='most_frequent')), ('onehot', OneHotEncoder(handle_unknown='ignore')) ]) # 3. Combine Preprocessing Steps preprocessor = ColumnTransformer(transformers=[ ('num', numerical_pipeline, numeric_columns), ('cat', categorical_pipeline, categorical_columns) ]) 5. Model Candidate Comparison & Stratified Cross-Validation Evaluate multiple classification algorithms using 5-Fold Stratified Cross-Validation evaluated on the $F_1$-score metric. # Define candidate models models = { 'Logistic Regression': LogisticRegression(max_iter=1000), 'K-Nearest Neighbors': KNeighborsClassifier(n_neighbors=7, metric='euclidean'), 'Random Forest': RandomForestClassifier(n_estimators=300, max_depth=15, random_state=42), 'Support Vector Machine': SVC(kernel='rbf', probability=True, random_state=42), 'Gradient Boosting': GradientBoostingClassifier(n_estimators=300, max_depth=15, random_state=42) } # Cross-Validation setup cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42) results = {} for name, model in models.items(): # Chain preprocessing and model estimation into a single pipeline full_pipeline = Pipeline(steps=[ ('preprocessor', preprocessor), ('model', model) ]) cv_scores = cross_val_score(full_pipeline, X_train, y_train, cv=cv, scoring='f1', n_jobs=-1) results[name] = { 'Mean F1 Score': cv_scores.mean(), 'Std Dev': cv_scores.std() } # Display results in a table results_df = pd.DataFrame(results).T.sort_values(by='Mean F1 Score', ascending=False) Model Performance Metrics Model Classifier Mean $F_1$ Score Standard Deviation Support Vector Machine (SVM) 0.7447 0.0311 Random Forest 0.7374 0.0200 Logistic Regression 0.7301 0.0217 K-Nearest Neighbors 0.7194 0.0229 Gradient Boosting 0.7002 0.0233 6. Key Takeaways & Best Practices Preventing Data Leakage: Wrapping feature engineering steps inside a Pipeline guarantees that statistics (e.g., mean/median for imputation, standard deviations for scaling) are calculated strictly on training folds during cross-validation. Handling Mixed Data Types: ColumnTransformer cleanly routes numerical features to continuous scaling modules and categorical variables to encoding blocks. Model Verdict: The Support Vector Machine (SVM) achieved the highest mean $F_1$-score ($0.7447$), with Random Forest displaying the highest stability across folds with the lowest variance ($\sigma = 0.0200$).