Health reporting from different data sources: Does it matter for mental health?

Background — Mental disorders are typically stigmatized conditions associated with negative stereotypes, which may lead individuals to underreport them. Thus, survey data may be subject to biases. Although administrative data has some limitations, it is an alternative data source that may be considered more objective.

Aims of the Study — This study aimed to identify the degree of agreement between survey and administrative healthcare data for mental health conditions, factors affecting underreporting, and whether underreporting also occurs for physical health conditions.

Methods — We used Ontario data from the Canadian Community Health Survey linked to health records to examine the presence of mental health conditions (i.e., schizophrenia and mood disorders) and select physical health conditions (i.e., diabetes and cancer). Using administrative data as the reference standard, we created four categories for each health condition based on the level of agreement between the two data sources: consistent cases and non-cases (i.e. individuals with concordant data based on their reported health condition), and people who were found to underreport and overreport a condition (i.e. where the condition was present in the administrative data, but not in the survey data and vice-versa, respectively). The overall level of agreement was assessed using Cohen’s kappa statistic. Probit regressions were estimated to determine the factors affecting underreporting.

Results — The Kappa statistics for mood disorder was fair (k= 0.26) and moderate for schizophrenia (k = 0.49). Physical health conditions had higher kappa values (diabetes, k = 0.81; ever having cancer, k = 0.68), with the exception of currently having cancer (k = 0.24). Underreporting was highest for the most stigmatizing condition, schizophrenia (63%), followed by mood disorders (39%) and cancer (39%), and lowest for diabetes (25%). Older age, being born in Africa and Asia, and being employed all increased the probability of underreporting among individuals identified in the administrative data; the opposite held for social assistance.

Discussion — We extended previous work on mental health reporting by combining survey data with administrative data to examine the level of agreement between respondents’ self-reported mental health and administrative records. The data include some mental disorders not studied previously. We examined the entire adult population; this is important because prevalence of schizophrenia may be less common among older population groups due to higher mortality among this patient population. Additionally, there may be potential age-related differences in stigma and mental health conditions. The administrative health data captured only health services covered by the public provincial health insurance plan and thus did not capture medical care provided by psychologists, social workers, and nurses. While this would affect Kappa statistic values, it does not directly affect the underreporting analyses.

Implications for Healthcare Provision and Use — Our results suggest that disclosure of mental health conditions may differ by the level of stigma, which has implications for obtaining accurate estimates of mental health prevalence from self-reported data sources.

Implications for Health Policies and Implications for Further Research — It may be useful to use a combination of both survey and administrative data when estimating the prevalence of mental disorders. Future research should seek to examine overreporting and its determinants.