Did the new version break any answers?
Give EvalTwig your rules and the saved answers from two versions of your assistant. It checks each answer for required and forbidden text, length, valid JSON and your JSON Schema, then lists what regressed. It doesn't judge whether an answer is factually right.
Your rules and answers are checked in memory on the server and are not stored or logged. Running live model calls isn't part of this tool.