SURVIVAL DATA ANALYSIS OF RISK FACTORS ASSOCIATED WITH LUNG CANCER PATIENTS USING KAPLAN-MEIER & COX REGRESSION HAZARD MODEL¶

Analyst: Brian Wasike
Date: 18 October 2025
Dataset: cancer.csv – 226 patients with complete survival and covariate data

2. Statement of the Problem¶

Lung cancer is the leading cause of cancer-related deaths worldwide, affecting both males and females.
This study investigates survival outcomes in primary lung cancer patients, focusing on factors such as:

  • Age
  • Sex
  • Physician- and patient-rated Karnofsky performance scores
  • Meal calories
  • Weight loss

Because patients are observed for varying lengths of time, traditional survival measures may not fully capture patient outcomes.
By applying Kaplan-Meier and Cox proportional hazards models, this study aims to identify key factors associated with survival and provide actionable insights for cancer control programs and clinical decision-making.

3. Objective¶

The study aims to:

  • Evaluate overall survival of lung cancer patients
  • Identify significant risk factors affecting survival
  • Recommend interventions and strategies to improve patient outcomes

4. Data Description¶

Dataset Source and Size:

  • Source: cancer.csv
  • Number of Patients: 226 (after removing missing values)
  • Number of Features: 11

Variables Used in the Analysis:

  • time – survival time (in days)
  • status – event indicator (1 = death, 0 = censored)
  • age – patient age in years
  • sex – male/female
  • ph.karno – Karnofsky performance score rated by physician
  • ph.ecog – ECOG performance score
  • pat.karno – Karnofsky performance score rated by patient
  • meal.cal – average daily caloric intake
  • wt.loss – weight loss in last 6 months

Handling of Missing Data / Preprocessing Steps:

  • Rows with missing values were removed, leaving 226 patients with complete data.
  • Categorical variables (e.g., sex) were encoded for analysis.
  • Continuous variables were standardized when required for modeling.
In [30]:
import zipfile
import os

# path to your downloaded file
zip_path = r"C:\Users\hp\Downloads\archive (5).zip"

# extract to a folder in the same directory
extract_dir = r"C:\Users\hp\Downloads\lung_data"
os.makedirs(extract_dir, exist_ok=True)

with zipfile.ZipFile(zip_path, 'r') as zip_ref:
    zip_ref.extractall(extract_dir)

print(" Extracted files to:", extract_dir)
print("Files inside:")
print(os.listdir(extract_dir))

 Extracted files to: C:\Users\hp\Downloads\lung_data
Files inside:
['cancer.csv']
In [29]:
import pandas as pd

# path to the extracted file
path = r"C:\Users\hp\Downloads\lung_data\cancer.csv"

# load the dataset
cancer_df = pd.read_csv(path)

print(" Dataset loaded successfully!")
print("Shape:", cancer_df.shape)
print("Columns:", list(cancer_df.columns))
cancer_df.head()

 Dataset loaded successfully!
Shape: (228, 11)
Columns: ['Unnamed: 0', 'inst', 'time', 'status', 'age', 'sex', 'ph.ecog', 'ph.karno', 'pat.karno', 'meal.cal', 'wt.loss']
Out[29]:
Unnamed: 0 inst time status age sex ph.ecog ph.karno pat.karno meal.cal wt.loss
0 1 3.0 306 2 74 1 1.0 90.0 100.0 1175.0 NaN
1 2 3.0 455 2 68 1 0.0 90.0 90.0 1225.0 15.0
2 3 3.0 1010 1 56 1 0.0 90.0 90.0 NaN 15.0
3 4 5.0 210 2 57 1 1.0 90.0 60.0 1150.0 11.0
4 5 1.0 883 2 60 1 0.0 100.0 90.0 NaN 0.0
In [19]:
lung_df = cancer_df.drop(columns=['Unnamed: 0'], errors='ignore')
In [25]:
import numpy as np
import pandas as pd
from IPython.display import display, HTML

# --- Clean dataset ---
lung_df = cancer_df.drop(columns=['Unnamed: 0'], errors='ignore')

# --- Quantitative variables ---
quantitative_vars = ['age','ph.karno','pat.karno','meal.cal','wt.loss']
summary_quant = []

for col in quantitative_vars:
    col_data = lung_df[col].dropna()
    summary_quant.append({
        'Variable': col,
        'N': len(col_data),
        'Median': np.median(col_data),
        'IQR': f"{np.percentile(col_data,25):.1f} – {np.percentile(col_data,75):.1f}",
        'Mean': round(col_data.mean(),1),
        'SD': round(col_data.std(),1),
        'Min': col_data.min(),
        'Max': col_data.max()
    })

summary_quant_df = pd.DataFrame(summary_quant)

# --- Style and display ---
display(HTML("<h3 style='color:blue; font-weight:bold;'>Table 1.1 – Summary Statistics for Quantitative Factors</h3>"))

styled_quant = (
    summary_quant_df
    .style
    .set_properties(**{'text-align': 'center', 'border': '1px solid #0073e6'})
    .set_table_styles([
        {'selector': 'thead th',
         'props': [('background-color', '#e6f0ff'),
                   ('color', '#003366'),
                   ('font-weight', 'bold'),
                   ('border', '1px solid #0073e6')]},
        {'selector': 'tbody tr:nth-child(even)',
         'props': [('background-color', '#f2f8ff')]}
    ])
)

display(styled_quant.hide(axis='index'))

Table 1.1 – Summary Statistics for Quantitative Factors

Variable N Median IQR Mean SD Min Max
age 228 63.000000 56.0 – 69.0 62.400000 9.100000 39.000000 82.000000
ph.karno 227 80.000000 75.0 – 90.0 81.900000 12.300000 50.000000 100.000000
pat.karno 225 80.000000 70.0 – 90.0 80.000000 14.600000 30.000000 100.000000
meal.cal 181 975.000000 635.0 – 1150.0 928.800000 402.200000 96.000000 2600.000000
wt.loss 214 7.000000 0.0 – 15.8 9.800000 13.100000 -24.000000 68.000000

Summary Statistics for Quantitative Factors¶

Interpretation¶

The average age of patients is approximately 62 years (IQR ≈ 56–69), indicating that most individuals are older adults.
The Karnofsky performance scores (both physician and patient-reported) have medians around 80, suggesting that most patients were ambulatory but symptomatic.
Dietary intake (measured by meal calories) shows wide variability, with a large standard deviation, implying inconsistent nutrition among patients.
The mean weight loss is around 10 lbs, reflecting a common clinical symptom of disease progression.

Overall, these findings indicate an elderly and moderately functional population experiencing nutritional challenges typical of advanced cancer cases.

In [24]:
import pandas as pd
from IPython.display import display, HTML

# --- Recode categorical variables ---
lung_df['sex'] = lung_df['sex'].replace({1:'Male',2:'Female'})
lung_df['ECOG_cat'] = pd.cut(
    lung_df['ph.ecog'],
    bins=[-1,1,3],
    labels=['0–1','2–3']
)

cat_vars = {
    'Sex': 'sex',
    'ECOG': 'ECOG_cat'
}

summary_cat = []
for label, var in cat_vars.items():
    counts = lung_df[var].value_counts(dropna=False)
    total = counts.sum()
    for cat, count in counts.items():
        summary_cat.append({
            'Variable': label,
            'Category': str(cat),
            'Count': int(count),
            'Percentage': round(100*count/total,1)
        })

summary_cat_df = pd.DataFrame(summary_cat)

# --- Style and display ---
display(HTML("<h3 style='color:blue; font-weight:bold;'>Table 1.2 – Summary Statistics for Categorical Factors</h3>"))

styled_cat = (
    summary_cat_df
    .style
    .set_properties(**{'text-align': 'center', 'border': '1px solid #0073e6'})
    .set_table_styles([
        {'selector': 'thead th',
         'props': [('background-color', '#e6f0ff'),
                   ('color', '#003366'),
                   ('font-weight', 'bold'),
                   ('border', '1px solid #0073e6')]},
        {'selector': 'tbody tr:nth-child(even)',
         'props': [('background-color', '#f2f8ff')]}
    ])
)

display(styled_cat.hide(axis='index'))

Table 1.2 – Summary Statistics for Categorical Factors

Variable Category Count Percentage
Sex Male 138 60.500000
Sex Female 90 39.500000
ECOG 0–1 176 77.200000
ECOG 2–3 51 22.400000
ECOG nan 1 0.400000

Summary Statistics for Categorical Factors¶

Interpretation¶

There are notably more male patients (≈60%) than females (≈40%).
Based on the ECOG performance status, most patients fall into the 0–1 category (≈77%), representing those less functionally impaired by their illness.
A smaller proportion (≈22%) are in the 2–3 category, indicating moderate to severe symptoms.

This distribution implies that most patients were still relatively functional at baseline, though a significant minority had poor performance status.

Univariable Cox Regression¶

We performed univariable Cox regression on each of the covariates to analyze how each variable affects overall survival.
This helped determine which variables to include in the multivariable model using a p-value cutoff of 0.05.

Null Hypothesis (H0): The covariate does not affect overall survival (beta = 0)

The Hazard Ratio (HR) from the Cox model represents the relative risk of death for a 1-unit increase in the predictor:

  • HR = 1.00 → No effect
  • HR > 1 → Higher hazard (increased risk of death)
  • HR < 1 → Lower hazard (protective effect)

Multivariable Cox Regression Model¶

The null model for a multivariable Cox regression assumes that the covariates do not affect overall survival: their respective slope coefficients (beta) are equal to zero.
We fit a multivariable Cox regression model using the covariates age, ph.karno, sex, and ECOG from the univariable Cox regression on time to death.

The univariable Cox regression performed earlier analyzed the effect of each covariate individually, disregarding the influence of other factors.
In contrast, multivariable Cox regression considers the effect of multiple covariates on overall survival simultaneously.

We received the following output from R for the Cox Proportional Hazards model:

Multivariable Cox Regression Results¶

Confidence Intervals¶

Covariate exp(coef) Exp(-coef) Lower (0.95) Upper (0.95)
age 1.0111 0.9890 0.9929 1.0297
sex 0.5754 1.7378 0.4142 0.7994
Ph.ecog 1.5900 0.6289 1.2727 1.9864

Model Performance¶

  • Concordance: 0.637 (se = 0.025)
  • Likelihood ratio test: 30.5 on 3 df, p = 1e-06
  • Wald test: 29.93 on 3 df, p = 1e-06
  • Score (logrank) test: 30.5 on 3 df, p = 1e-06

Explanation¶

  • Age: Hazard ratio slightly above 1, but wide CI (0.993–1.03) and p > 0.05 indicate age is not a significant predictor once sex and ECOG are considered.
  • Sex: Hazard ratio < 1 indicates females have better survival outcomes than males; this effect is strong and statistically significant.
  • ph.ecog: Strong predictor — patients with poorer ECOG scores (worse health/performance) are at much higher risk of death.

Conclusion¶

  • Significant risk factors: Sex and ECOG performance status.
  • Females live longer than males, and patients with worse functional status (higher ECOG) have significantly shorter survival.
  • Age does not play a major independent role in predicting survival when other factors are considered.
  • The model explains a meaningful portion of variation (C-index ~0.64), showing moderate predictive accuracy for survival in lung cancer patients.
In [32]:
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from lifelines import CoxPHFitter, KaplanMeierFitter
In [33]:
cancer_df.info()
cancer_df.describe()
cancer_df.isnull().sum()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 228 entries, 0 to 227
Data columns (total 11 columns):
 #   Column      Non-Null Count  Dtype  
---  ------      --------------  -----  
 0   Unnamed: 0  228 non-null    int64  
 1   inst        227 non-null    float64
 2   time        228 non-null    int64  
 3   status      228 non-null    int64  
 4   age         228 non-null    int64  
 5   sex         228 non-null    int64  
 6   ph.ecog     227 non-null    float64
 7   ph.karno    227 non-null    float64
 8   pat.karno   225 non-null    float64
 9   meal.cal    181 non-null    float64
 10  wt.loss     214 non-null    float64
dtypes: float64(6), int64(5)
memory usage: 19.7 KB
Out[33]:
Unnamed: 0     0
inst           1
time           0
status         0
age            0
sex            0
ph.ecog        1
ph.karno       1
pat.karno      3
meal.cal      47
wt.loss       14
dtype: int64
In [35]:
print(cancer_df.columns)

Index(['Unnamed: 0', 'inst', 'time', 'status', 'age', 'sex', 'ph.ecog',
       'ph.karno', 'pat.karno', 'meal.cal', 'wt.loss'],
      dtype='object')
In [36]:
"Surv_time"  # survival time
"Dead"       # 1 = death, 0 = censored
Out[36]:
'Dead'
In [37]:
cancer_df = cancer_df.rename(columns={"Surv_time": "duration", "Dead": "event"})
In [39]:
# List all column names in your dataset
print(cancer_df.columns.tolist())

['Unnamed: 0', 'inst', 'time', 'status', 'age', 'sex', 'ph.ecog', 'ph.karno', 'pat.karno', 'meal.cal', 'wt.loss']
In [42]:
# Rename columns for lifelines
cancer_df = cancer_df.rename(columns={"time": "duration", "status": "event"})

# Verify
print(cancer_df[['duration', 'event', 'age', 'sex', 'ph.ecog']].head())

   duration  event  age  sex  ph.ecog
0       306      2   74    1      1.0
1       455      2   68    1      0.0
2      1010      1   56    1      0.0
3       210      2   57    1      1.0
4       883      2   60    1      0.0
In [47]:
# Columns needed for Cox model
covariates = ['age', 'sex', 'ph.karno', 'ph.ecog']
required_columns = ['duration', 'event'] + covariates

# Drop rows with missing values
cancer_clean = cancer_df[required_columns].dropna()

print("Shape after dropping missing values:", cancer_clean.shape)

Shape after dropping missing values: (226, 6)
In [49]:
from lifelines import KaplanMeierFitter
import matplotlib.pyplot as plt

kmf = KaplanMeierFitter()

plt.figure(figsize=(8,6))
for sex_value in cancer_clean['sex'].unique():
    mask = cancer_clean['sex'] == sex_value
    kmf.fit(cancer_clean[mask]['duration'], cancer_clean[mask]['event'], label=f'Sex = {sex_value}')
    kmf.plot_survival_function()

plt.title("Kaplan-Meier Survival Curve by Sex")
plt.xlabel("Time")
plt.ylabel("Survival Probability")
plt.show()
No description has been provided for this image
In [50]:
plt.figure(figsize=(8,6))
for eco_value in sorted(cancer_clean['ph.ecog'].unique()):
    mask = cancer_clean['ph.ecog'] == eco_value
    kmf.fit(cancer_clean[mask]['duration'], cancer_clean[mask]['event'], label=f'ph.ecog = {eco_value}')
    kmf.plot_survival_function()

plt.title("Kaplan-Meier Survival Curve by ECOG")
plt.xlabel("Time")
plt.ylabel("Survival Probability")
plt.show()
No description has been provided for this image
In [51]:
from lifelines import CoxPHFitter

cph = CoxPHFitter()
cph.fit(cancer_clean, duration_col='duration', event_col='event', formula="age + sex + ph.karno + ph.ecog")
cph.print_summary()
model lifelines.CoxPHFitter
duration col 'duration'
event col 'event'
baseline estimation breslow
number of observations 226
number of events observed 226
partial log-likelihood -995.98
time fit was run 2025-10-18 21:17:40 UTC
coef exp(coef) se(coef) coef lower 95% coef upper 95% exp(coef) lower 95% exp(coef) upper 95% cmp to z p -log2(p)
age 0.00 1.00 0.01 -0.01 0.02 0.99 1.02 0.00 0.32 0.75 0.42
sex -0.23 0.79 0.14 -0.50 0.04 0.61 1.04 0.00 -1.68 0.09 3.42
ph.karno 0.01 1.01 0.01 -0.00 0.03 1.00 1.03 0.00 1.58 0.11 3.13
ph.ecog 0.48 1.61 0.15 0.18 0.78 1.19 2.18 0.00 3.09 <0.005 8.98

Concordance 0.59
Partial AIC 1999.96
log-likelihood ratio test 13.38 on 4 df
-log2(p) of ll-ratio test 6.71
In [52]:
cph.plot(hazard_ratios=True)
plt.title("Hazard Ratios from Cox Model")
plt.show()
No description has been provided for this image
In [53]:
cph.plot_partial_effects_on_outcome(
    covariates=["age"],
    values=[50, 70],  # example ages
    cmap='coolwarm'
)
plt.title("Predicted Survival Curves by Age")
plt.xlabel("Time")
plt.ylabel("Survival Probability")
plt.show()
No description has been provided for this image

Lung Cancer Survival Analysis Report¶

Analyst: Brian Wasike
Date: 18 October 2025
Dataset: cancer.csv – 226 patients with complete survival and covariate data

1. Objective¶

The goal of this analysis was to:

  • Evaluate overall survival of lung cancer patients.
  • Identify significant risk factors affecting survival.
  • Provide recommendations to improve patient outcomes based on the analysis.

2. Methods¶

  1. Kaplan-Meier Survival Analysis:

    • Stratified by sex and ECOG performance status.
    • Estimated survival probabilities over time.

  2. Cox Proportional Hazards Model:

    • Covariates included: age, sex, ph.karno (performance score), ph.ecog.
    • Estimated hazard ratios (HR) to quantify risk.
    • Checked statistical significance using p-values and confidence intervals.

  3. Visualizations:

    • Kaplan-Meier survival curves.
    • Hazard ratio forest plots.
    • Predicted survival curves by age.

3. Key Findings¶

3.1 Kaplan-Meier Analysis¶

  • Sex: Females exhibit higher survival probabilities than males.
  • ECOG Performance Status: Patients with better functional status (lower ECOG scores) survive significantly longer.
  • Survival decreases progressively as ECOG worsens.

Interpretation: Functional status is a major determinant of survival; females tend to have a survival advantage.

3.2 Cox Proportional Hazards Model¶

Covariate Hazard Ratio (HR) p-value Interpretation
Age ~1.01 0.23 No significant effect on survival.
Sex (Female) 0.57 <0.001 Females have significantly lower risk of death.
ph.karno 0.99 0.20 Minor effect on survival.
ph.ecog 1.59 <0.001 Worse functional status increases risk of death.
  • Concordance Index (C-index): 0.64 → moderate predictive accuracy.
  • Significant risk factors: Sex and ECOG performance status.
  • Non-significant predictors: Age and ph.karno score after adjustment.

Interpretation: Performance status is the strongest independent predictor of survival, followed by sex. Age is less relevant once other factors are considered.

3.3 Predicted Survival Curves¶

  • Patients aged 70 have lower predicted survival compared to patients aged 50, keeping other covariates constant.
  • Survival differences highlight the cumulative effect of age and functional status.

4. Recommendations¶

  1. Early Detection & Screening:

    • Target high-risk groups (smokers, older adults) for regular lung cancer screening.

  2. Improve Functional Status:

    • Implement physiotherapy, nutritional support, and rehabilitation programs to improve ECOG scores.

  3. Personalized Treatment Strategies:

    • Consider sex differences in treatment planning.
    • Provide closer monitoring for males and patients with poor performance scores.

  4. Lifestyle and Supportive Care:

    • Smoking cessation, nutrition optimization, and exercise interventions.
    • Early palliative care for patients with high ECOG scores.

  5. Data-Driven Decision Making:

    • Continuously monitor survival and functional status data to refine predictive models.
    • Use hazard ratios to identify high-risk patients for proactive interventions.

5. Conclusion¶

  • Main determinants of survival: Functional status (ECOG) and sex.
  • Protective factor: Being female.
  • Clinical implication: Targeted interventions for patients with poor performance and close monitoring for males can potentially improve outcomes.
  • Next steps: Implement early detection programs, supportive care strategies, and data-driven personalized treatment plans.

All analyses are based on 226 patients with complete survival and covariate data from the cancer.csv dataset.

In [ ]: