WI21-01: Using Online SSDI Conversations to Improve Communication and Outreach
Center for Financial Security
MetadataShow full item record
Text analysis of data collected from online forum conversations, a form of user-generated content (UGC), reveals that Social Security Disability Insurance (SSDI) applicants and recipients, the “customers,” share concerns and confusion about the application and appeal process rules and policies. Extant research suggests that confusions about how SSDI rules are interpreted and applied significantly contribute to high SSDI rejection and appeal rates. This study attempts to provide insights into designing effective communication strategies to reduce confusion and improve customer service experiences and welfare. Given the size of the data, we first use unsupervised machine learning algorithms to derive topics and model them using epistemic network analysis (ENA) via conversational connections. The resulting ENA provides insights on the structural relationships between different issues surrounding SSDI (e.g., struggles the applicants faced in communicating with and obtaining information from SSA). Taking advantage of the longitudinal nature of the data, we also model trajectory ENAs to investigate how these issues evolve against the backdrop of environmental and policy changes. To provide deeper contextual value through human judgment, we use the derived topics as seed words in nCoder (an automated classifier). The resulting codes can be used in different applications, from analyzing the efficacy of existing policy to providing practical policy recommendations.
H530 National Government Expenditures and Welfare Programs - Disability Insurance
H550 Social Security and Public Pensions
J29 Other - Time Allocation, Work Behavior, and Employment Determination
Text analysis of online discussions can identify points of confusion to overcome information asymmetry of the 60 percent rejected applications to reduce the administrative burden of SSA. Existing practice in policy and program evaluations are primarily based on research surveys/interviews using self-reported data and administrative data that may not reveal individual user experience or cover financially vulnerable populations. Collecting and analyzing user-generated content (UGC) from online forums provides insights from the individuals' perspectives regarding user experience and knowledge sharing. The study’s two primary objectives are to provide insights on effective communication strategies to reduce confusion and improve customer service experiences and welfare. First, the analysis identifies the major areas of confusion about SSA rules and decision criteria using a machine-learning hybrid approach to natural language processing (NLP) and text analytics, and second, to evaluate the impact of how and when SSA customers obtain such information on their interpretation of this information.
Wong, Nancy, Lydia Ashton, Jaeyoon Choi, and Brett Puetz. 2021. "Using Online SSDI Conversations to Improve Communication and Outreach." FY2021 Research Projects. Retirement & Disability Research Center. https://cfsrdrc.wisc.edu/project/wi21-01.