The Problem
After initial release, all submitted claims were rejected
The claims form had been released online to a randomized selection of website users, but the initial claims submitted were all rejected. The documentation users provided did not match the requirements, despite those requirements being listed on the page.
Initial Research
Studied rejected claims to see what was incorrect
We went through the rejected claims to determine what specific information was missing. In some cases, the wrong document was being submitted, and in others, the documentation did not contain all the items listed as required.
Initial changes and results
We needed a quick fix to get the claims form back online
We had taken down the claims form down to determine needed changes, and needed a quick fix to get the claims form back online. Our initial changes would be to content only while we tried to find a better solution for the long term.
I went through all the content on the upload pages and revised it so that there was a bullet for each required piece of information, as well as bolded text for easier scanning. I also moved the note about asking their provider for a detailed statement to the top of the page so it wouldn't get lost in the lengthy text that followed.
Initial Results
With the revised content, the rejected claims rate went down to 50%.
This was still not ideal, but this rejection rate matched the send-back rate of the paper form, so we left this version of the online form up while we worked on further solutions.
Experimenting with AI
Designing longer-term solutions
One idea I was interested in trying was scanning the uploaded documents to ensure that the required information was present. Because the documentation wasn't standardized, however, using an LLM to scan was preferable to using OCR. I worked with engineering to make this a viable solution.
User Testing
We tested the full claims flow with file verification
The claims form had last been tested a year prior, and we had revised the flow since then to include resubmissions and various fixes, so re-testing the form was important. We decided to add the LLM file verification to our test plan for our claims form research.
The engineering team did a lot of work to try to get the LLM file verification working at an acceptable level for testing, but due to the tight timeline, we were only able to show some generic messaging during our user testing rather than the complete verification. We did gather some valuable information on how users felt about file verification, which was mostly positive.
-
User research goals
-
Key findings
-
Users
Results
User testing validated the flow and file verification
Users wanted a way to know that their documentation was correct prior to submitting and definitely prefered that to finding out it was incorrect after processing. We felt we had a mandate to continue on the file verification path.