The AI Specialist Onboarding Tool collapses AI Specialist readiness from weeks of manual scoping into days of guided, validated setup. Point it at your closed-incident history and the tool identifies the specific issue types your L1 AI Specialist can resolve, validates each one against your own KB and catalog content, runs simulated resolutions scored by an LLM judge, and generates the routing criteria that send the right incidents — and only the right incidents — to the AI Specialist in production.
As a by-product, it surfaces where content gaps exist, pointing to the KBs and catalog items that would unlock the next wave of automation.
Deployment guidance: Run this tool on a sub-production instance (a production clone with recent incident, KB, and catalog data). The simulation stage generates evaluation incidents against the AI Specialist, which is not appropriate for production environments. Generated routing criteria can then be promoted to production through your standard update set process.
Here is the documentation on how to use this tool - 2026-Jul-2-Onboarding_tool.docx
1. Ability to identify content coverage and gaps
2. Run the ZTSD evals and followed by LLM judge evaluation of ZTSD response
3. Recommended routing criteria on what types of incidents can be routed to L1 AIS
1. This release focuses on migrating the data to the new L1 AIS onboarding tables
2. Once upgraded, a migration script will trigger and move all the data in the current tables to the new tables. This migration will trigger automatically
3. You should only upgrade once all your in-progress evals are completed or cancelled. Any in-progress evals will be automatically errored out once the migration happens.
NA