Introduction
Major Depressive Disorder is characterized by depressed mood; suicidal thoughts; and diminished sleep, energy, appetite and physical activity for at least two weeks [1]. Worldwide, the point prevalence of depression is 5-9% in women and 2-3% in men and the lifetime prevalence rate is 16.6% [2, 3].
The presence of depression has been linked to negative health outcomes, including cardiovascular disease, diabetes mellitus and increased risk of mortality [4]. Depression also has been associated with productivity loss and disability [5]; individuals with depression are seven-fold more likely to develop disability than the general population [6].
We recently explored factors associated with long-term disability (LTD) claim duration using administrative data from a large, private Canadian insurer (Sun Life Canada) [7]. Our adjusted analysis found an unexpected association between province of residence and claim duration; specifically, LTD claimants residing in Quebec were more likely to resolve their claim faster than similar patients residing in Ontario who had depression (hazard ratio [HR] = 1.93; 99% confidence interval [CI] = 1.78 to 2.09) [7].
In this study, we used the administrative data of another Canadian disability insurer— SSQ Life Insurance Company Inc. (SSQ) to confirm our previous research findings of variations in claim closures in disability claimants with depression. The aim of this study is to develop and validate a prognostic model for predicting the possibility of long-term disability (LTD) claim closure in workers disabled by depression.
Material and Method
Standardized reporting
We followed the recommendations from the reporting statement and checklist¬—Reporting Recommendations for Tumour Marker Prognostic Studies (REMARK) and quality assessment tool—Quality in Prognosis Studies (QUIPS) [8].
Patients
We undertook a prognostic model study using a retrospective cohort design. We examined all claims that SSQ approved for LTD benefits from January 1, 2007 to March 31, 2014. Our study sample consisted of 22,205 LTD claims. We excluded claimants who received a General Expenses benefit plan, as well as those for whom the case managers did not record an initial decision to maintain data integrity (Figure 1).
LTD benefits provide wage replacement benefits for disabled workers up to age 65, typically as long as they remain disabled from their own occupation for the initial two years of the claim, and disabled from any and all occupations for which they are qualified by training or experience after they have been on claim for two years (the change of definition period). For our analyses we included all unique claims that were approved for LTD benefits with a primary diagnosis of Major Depressive Disorder (henceforth referred to as depression). We excluded all individuals whose claims were recorded as closed prior to contractual approval. The study was approved by the Research Ethics Board at McMaster University.
SSQ administrative database consists of demographic, administrative, and clinical information. All data is entered by the case manager(s) overseeing each claim. We screened all data to identify outliers, inconsistencies and missing data by calculating summary statistics and exploring distributions graphically. If clear outliers and inconsistencies were identified, we worked with SSQ to correct the data.
The principal prognostic variable was claimant’s province of residence (Ontario or Quebec). We also included age, gender, pre-disability salary, physical workplace demands and time to claim approval in our statistical model as candidate prognostic markers. Our primary outcome was time to LTD claim closure, defined as the duration from claim approval until claim closure. We performed a time-to-event analysis using flexible parametric models to assess the association between time to claim closure and prognostic markers [9, 10]. For LTD claims that were unresolved when the data was extracted, we used the date of extraction as our censoring point. We defined, a priori, the following five prognostic markers based on previous research, consultation with experts in the field, and availability in the administrative database: (1) province of residence, (2) age, (3) gender, (4) pre-disability salary, (5) physical workplace demands and (6) time to claim approval (Table 1) [7].
Statistical analysis
We present baseline demographics as frequency (percentage) for categorical variables and means and standard deviations for continuous variables. We tested the differences between LTD claimants who resided in Ontario and Quebec with the Pearson’s Chi Square test for categorical variables and two-sample t-tests for continuous variables.
Pair-wise correlations between predictor variables using the Pearson correlation coefficient (r) were tested. We planned to remove the variable with lesser importance if r is greater than 0.80 for 2 predictors. The linear regression model was used to generate collinearity statistics. Tolerance and variance inflation factor (VIF) were used to test the assumption. Values less than 10 for VIF, and more than 0.1 for tolerance were considered violations. All the values were below the limits for r and VIF.
We used spline-based survival models (Royston-Parmar) which allow more flexibility for the shape of the survival distribution and employ restricted cubic spline (RCS) of log-time. We started fitting a model by incorporating candidate prognostic markers—province of residence, age, gender, pre-disability salary, time to claim approval, and physical workplace demands—into the models for estimating survival. For variable selection, we employed p≤0.1 as the significance threshold to be retained in the model with a backward elimination approach [11]. We employed parsimonious prediction models by retaining the most significant factors for our final model [12]. We used a log-likelihood ratio test, Akaike Information criterion (AIC) and Bayesian Information criterion (BIC) to guide decisions regarding the number of optimal nodes and best fitted model in terms of variable selection and time-dependency of variables [13]. A risk factor can be fixed (i.e., race) or change over the follow-up time (i.e., weight) [14] and may need to be included as time-varying covariate. Province of residence interacted with time and we used spline function to address time-dependency with province of residence.
We fitted our flexible parametric models using freely available stpm2 software developed by Lambert and Royston for the Stata package. We estimated descriptive summary measures using Stata/SE 12.1 (Stata Corporation, College Station, TX).
We employed a random split technique with a ratio of 70:30 for creating derivation and validation datasets. We used the baseline hazard function and ß-coefficients from the best fitted model to calculate the probability for claim closure in the validation set. Calibration indicates the amount of agreement between the predicted and observed risks. Calibration curves indicate expected versus predicted probability of LTD claim closure based on deciles of risk. LTD claimants are placed in probability quantiles ranked on the horizontal axis and vertical axis from lowest to highest. Predicted probability quantiles were plotted on the horizontal axis while observed probability quantiles are plotted on the vertical axis. We visually examined the calibration plots and employed the Hosmer-Lemeshow X2 test to examine the agreement between the observed and predicted probabilities. P-values above 0.05 indicated adequate calibration for the models.
Discrimination implies the ability to predict those with and without the outcome of interest. We assessed discrimination using a receiver operating curve (ROC) which plots sensitivity against 1 minus specificity on different cut-off values [15]. Our reference standard was observed disability claim closure from the SSQ database. Area under the curve (AUC) between 0.90 to 1.0 is considered excellent, whereas values between 0.80 to 0.90 are good, 0.70 to 0.80 are fair, 0.60 to 0.70 are poor, and less than <0.60 are fail [24].
We employed two-sided tests with a significance level of ≤0.05. All data analyses were performed using Stata (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP).
Results
The descriptions of variables and anticipated direction of the effects for long-term disability closure are provided in Table 1. Our study cohort was comprised of 3,113 LTD claimants with depression out of which 2,679 cases (87%) were closed when our data was captured, and 434 (13%) were censored. A total of 198 claimants resided in Ontario (mean age 45 years) while 2,915 LTD claimants resided in Quebec (mean age 44 years) (Table 2). Duration of claim approval was longer and salary was higher in Ontario (Table 2). Flow diagram of long-term disability claimants is depicted in Figure 1.
Figure 2 presents Kaplan-Meier Curves for the proportion of LTD claimants remaining on benefits, stratified by province of residence. The median time on LTD was approximately 200 days in Quebec and 550 days in Ontario. The log-rank test for equality of survivor functions between provinces indicated that the curves were significantly different (p=0.0001). A graphical display shows hazard rates of long-term disability claimants remaining on benefit in Ontario and Quebec (Figure 3). The difference in hazard rate is higher at 500 days (Figure 3).
The final fitted model included age, gender, salary, physical workplace demands, time to claim approval and province of residence. Based on AIC and BIC criteria, the optimal number of knots was five under the hazard scale in our models (Table 3). Our Royston-Parmar models for LTD claimants indicated older age (HR=0.81, 99% CI= 0.77-0.85, per decade), female sex (HR=0.82, 99% CI= 0.74-0.91) and shorter time to claim approval (HR=0.98, 99% CI= 0.97-0.99, per day) were associated with slower LTD claim closure. Furthermore, LTD claimants with depression who resided in Quebec were more likely to have shorter disability closure as compared to Ontarians (HR=2.3, 99% CI= 1.73-3.2) (Table 4, Figure 3).
We assessed prognostic model performance. The 3,113 claimants with a major depression diagnosis were randomly classified as a derivation dataset (2/3, 70/100, n=2179) and a validation dataset (1/3, 30/100, n=934). There was not a significant difference between the derivation and validation datasets; therefore, we did not employ stratification in order to balance prognostic markers between the groups (Table 5). Figure 4 presents the observed versus predicted 1-year rate of LTD claim closure in the derivation dataset, demonstrating relatively small differences between predicted and observed values and no systematic over or underestimate. We produced an ROC curve for 1-year follow-up period in the training and validation datasets. The area under the curve estimates were 0.70 for the derivation dataset and 0.69, for the validation dataset (Figures 5 and 7). Figure 6 depicts the calibration plot of the validation dataset which did not show a significant difference between expected versus predicted probability of long-term disability claim closure based on deciles of risk (P > 0.05).
Discussion
Major Findings
We confirmed that LTD claimants with depression who resided in Ontario remained on LTD disability benefits longer than those who resided in Quebec. Our study also confirmed that older age, female gender, and longer time to claim approval were associated with longer LTD claim duration. Application of the model showed modest but acceptable calibration at 1 year (0.70) and a close relation between predicted and observed probability of claim closure (Figure 3).
Strengths and limitations
This is a large prognostic model study of disability claimants with depression with an adequate data source and analysis plan [16]. We defined objectives, prognostic markers and outcomes as ad-hoc. Explicit eligibility criteria limited the risk for misclassification. The prognostic markers incorporated in our model have objective and reproducible measurement properties [16]. Other strengths of our study included a priori creation of regression models and the anticipated direction of effects of included prognostic markers (Table 3).
We were limited by the existing database for quality and quantity of the variables. Another limitation is the absence of information about health risk behaviours, such as alcohol intake and drug dependence. As a result, we may have missed potentially important prognostic markers. Since we used an administrative database, diagnostic validity is expected to be low [17].
Findings in the context with previous evidence
Depression is a common cause of disability. Improved understanding of the factors associated with claim resolution may lead to strategies to improve prognosis among patients at high risk of prolonged claim duration [18-20]. A prior Canadian study indicated, unexpectedly, that residents of Quebec were more likely to resolve their LTD claim faster when compared to the Ontario residents (HR=1.93, 99% CI=1.78-2.09) [7]. This is congruent with our findings.
The association of province of residence with delayed recovery may be explained by differences in the quality and quantity of mental health services provided in Ontario and Quebec, or differences in claim management policies and processes in Quebec. These findings require further exploration.
Our findings are consistent with previous evidence that suggests a link between older age, female gender, and higher pre-disability salary with prolonged duration of disability claims [21, 22]. In contrast, our results do not support previous findings that have reported an association between low physical job demands and longer disability benefit duration [23].
Implications of our findings
Further research is needed to explore the determinants of the differences in LTD claim closures between provinces. A prospective well-designed and well-conducted cohort study would provide more reliable evidence than our results. Exploration of the factors associated with disability claim duration will require linking records from several databases to retrieve complete information about health care utilization. In the presence of important prognostic markers, model performance will be likely to improve. Our model’s discrimination and calibration was sufficient to warrant application in another database for validation.
To conclude, we developed a prognostic model for predicting the possibility of LTD claim closure in Canadian workers disabled by depression. Our study found that disabled workers who reside in Ontario receive LTD benefits longer than similar patients who reside in Quebec. Further study is needed to determine if this association is affected by differences in patients, clinical management, or case management between provinces. Our prognostic model has modest but acceptable discrimination and good calibration and warrants validation in an independent data set.
Competing interests
The authors declare that they have no competing interests.
Acknowledgements:
We thank the SSQ Life Insurance Company Inc. for making their database available for our study. Nigar Sekercioglu received Ontario Graduate Fellowship.
References
1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. Washington, DC: Psychiatric Press Inc; 2000.
2. Snow V, Lascher S, Mottur-Pilson C. Pharmacologic Treatment of Acute Major Depression and Dysthymia: Clinical Guideline, Part 1. Annals of internal medicine 2000;132 (9):738-42.
3. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry 2005; 62(6):593-602.
4. Windle M, Windle RC. Recurrent depression, cardiovascular disease, and diabetes among middle-aged and older adult women. Journal of Affective Disorders 2013;150 (3):895-902.
5. Kouzis AC, Eaton WW. Psychopathology and the initiation of disability payments. Psychiatric Services 2000; 51(7):908-13.
6. Kouzis AC, Eaton WW. Psychopathology and the development of disability. Social Psychiatry and Psychiatric Epidemiology 1997;32(7):379-86.
7. Ebrahim S, Guyatt GH, Walter SD, Heels-Ansdell D, Bellman M, Hanna SE, et al. Association of Psychotherapy with Disability Benefit Claim Closure among Patients Disabled Due to Depression. PLoS ONE 2013;8(6):67162.
8. Hayden JA, van der Windt DA, Cartwright JL, Cote P, Bombardier C. Assessing bias in studies of prognostic factors. Annals of internal medicine 2013;158 (4):280-6.
9. Lambert PC, P R. Further development of flexible parametric models for survival analysis. The Stata Journal 2009; 9:265-90.
10. Royston P, MK P. Flexible parametric proportional-hazards and proportional-odds models for censored survival data, with application to prognostic modelling and estimation of treatment effects. Statistics in Medicine 2002; 21(15):2175-97.
11. Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: Developing a prognostic model. British Medical Journal 2009;338:604.
12. Altman DG, Lyman GH. Methodological challenges in the evaluation of prognostic factors in breast cancer. Breast Cancer Research and Treatment 1998;52(1-3):289-303.
13. Miladinovic B, Kumar A, Mhaskar R, Kim S, Schonwetter R, Djulbegovic B. A flexible alternative to the Cox proportional hazards model for assessing the prognostic accuracy of hospice patient survival. PLoS One 2012;7(10):47804.
14. Dekker FW, de Mutsert R, van Dijk PC, Zoccali C, Jager KJ. Survival analysis: time-dependent effects and time-varying risk factors. Kidney International 2008;74(8):994-7.
15. Fletcher RH, Fletcher SW, EH W. Clinical Epidemiology: The Essentials. Baltimore: Lippincott Williams & Wilkins; 1996.
16. Moons KGM, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how? British Medical Journal 2009;338:375.
17. Dorresteijn JA, Visseren FL, Ridker PM, Wassink AM, Paynter NP, Steyerberg EW, van der Graaf Y, Cook NR. Estimating treatment effects for individual patients based on the results of randomised clinical trials. British Medical Journal 2011;343:5888. DOI:10.1136/bmj.d5888.
18. Kronström K, Karlsson H, Nabi H, Oksanen T, Salo P, Sjösten N, et al. Optimism and pessimism as predictors of work disability with a diagnosis of depression: A prospective cohort study of onset and recovery. Journal of Affective Disorders 2011;130(1–2):294-9.
19. Nielsen MBD, Madsen IEH, Bültmann U, Christensen U, Diderichsen F, Rugulies R. Predictors of return to work in employees sick-listed with mental health problems: findings from a longitudinal study. European Journal of Public Health 2011;21(6):806-11.
20. Ebrahim S, Montoya L, Truong W, Hsu S, Kamal el Din M, Carrasco-Labra A, et al. Effectiveness of Cognitive Behavioral Therapy for Depression in Patients Receiving Disability Benefits: A Systematic Review and Individual Patient Data Meta-Analysis. PLoS ONE 2012;7(11):50202.
21. Cornelius LR, van der Klink JJ, Groothoff JW, Brouwer S. Prognostic factors of long term disability due to mental disorders: a systematic review. Journal of Occupational Rehabilitation 2011;21(2):259-74.
22. Blank L, Peters J, Pickvance S, Wilford J, Macdonald E. A systematic review of the factors which predict return to work for people suffering episodes of poor mental health. Journal of Occupational Rehabilitation 2008;18(1):27-34.
23. Detaille SI, Heerkens YF, Engels JA, van der Gulden JW, van Dijk FJ. Common prognostic factors of work disability among employees with a chronic somatic disease: a systematic review of cohort studies. Scandinavian journal of work, environment & health: 2009; 35(4):261-81.
24. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143 (1):29-36.
This work is licensed under a Creative Commons Attribution 4.0 International License. To view a copy of the license, visit