While Casebook helps child welfare caseworkers and supervisors do their jobs more effectively, agency managers also needed tools to support their efforts to improve both personnel performance and family outcomes.

Not working directly in the application or with front-line workers everyday, it was challenging for these most senior organizational leaders to measure how their employees were doing, to decide what to change when things weren't going well, or to measure the impacts of policy changes once implemented.  Furthermore, for both agency managers and our internal team, it was also important that we find a way to measure the influence of the application itself — or changes to it — on policy and practice outcomes.

We designed the Casebook Analytics site and Casebook Reporting Studio to provide both clients and internal users regularly updated views of the “First 30” metrics: measures the Case Commons Data Science & Policy Team has determined to be most important in representing both child welfare agency performance and the safety and long-term well-being  of the children in state care.

Having access to both these Casebook companion tools measurably helps agency managers shape policy and practice and manage their worker and supervisory staff. It also allows them to see the immediate impacts of changes we are making to the application in real time, inspiring confidence in the work we are doing and allowing them to inflect and iterate where desired.

Gallery View

This is the landing page for the Casebook Analytics site. Most of the reports here are purely employee performance-oriented, since these are the measures that agency managers told us they review every day and wanted to be able to see "at a glance." Prior to the release of this site, managers tracked these metrics on manually compiled Excel spreadsheets.

Timeliness of Caseworker Contacts With Children

Beyond the gallery view, we aimed to marry surface-level process outcomes with individual child experience, such that the intention behind state policies could be exposed through the reports. In this example, graph lines indicate the share of children in different types of care who were visited within the previous 30 days by their Family Case Managers (FCM’s).  

Federal mandates recommend FCM/Child contacts at least once per calendar month, which for the most part, workers were successfully achieving. However, this report and others showed us that many children, especially those living at home, actually saw an FCM closer to every 60 days. While not technically a failure to meet requirements, visiting once per calendar month often failed to fulfill on the intention of  the guidelines for visit frequency, which were based on evidence that more closely spaced FCM/Child contacts have a dramatically positive impact on both child well being and case outcomes.

In response to this information, my team implemented dashboard metrics within the Casebook application emphasizing an actual count of days between visits, rather than the calendar date of the previous visit, as is the practice in many state systems. The dotted vertical line above represents the day this feature went live — resulting in both measurably dramatic decreases in the time between subsequent FCM/Child contacts and very happy clients.

Involvements Overwitten

In addition to targeting practice improvements, we also keep a close eye on data quality across the Casebook application, again focusing UX interventions in response to our findings. 

After this graph confirmed our fears that users were accidentally overwriting, rather than adding to, longitudinal data related to child service levels (Involvements), we shifted the location and UI-mechanism for updating this information. Again, the dotted vertical line represents the date these changes went live, as well as the subsequent drop in data overwrites. The result has been a more complete historic data set, more accurately representing the true experience of the children in state care. 

Casebook Reporting Studio

In contrast to the curated views of the Casebook Analytics site, we built the Casebook Reporting Studio to allow clients with some data expertise to manipulate the views and create their own custom reports. Highly flexible, this tool allows users to drill down into specific subsets of data, even sending custom reports to workers and supervisors about groups of children needing attention.

< Previous  |  Next >