Skip to content
ARCHYTASAI

ENV-OPS-105 · IT Operations & Service Management

ITServiceOps. Resolve employees' IT problems with the least disruption, following change control and least privilege, and keep tickets accurate.

Internal IT for a mid-market company: service desk tickets, sign-in and SaaS incidents, device-management changes, access requests and change control, with tickets, identity, device and asset records and the policies that govern them.

Custom buildIT Service Management · Identity & Access · Change Control

Overview. What the environment trains, and the work it's built around.

Core competency

Resolve employees' IT problems with the least disruption, following change control and least privilege, and keep tickets accurate.

  • Triaging tickets and finding the shared cause behind many reports
  • Setting priority from impact and urgency
  • Recovering device and SaaS incidents through authorized, staged changes
  • Granting access by least privilege across joiner, mover and leaver events
  • Classifying changes and obtaining the right authorization

Built around the work of

IT Service Desk Analyst · Systems Administrator · Identity & Access Administrator · Change Manager

Simulated systems

  • ITSM
  • Identity provider and SSO
  • Device management
  • SaaS admin consoles
  • Asset registry
  • Change management

Sample task

Since 07:30, macOS users can't connect to the VPN, while Windows users are fine. A device-management profile was pushed to all laptops at 07:20. Reverting it is a standard change, provided it goes to a pilot group first. Confirm the cause, restore VPN access through the authorized procedure, and close the incident with evidence.

Specification. The environment, field by field.

spec · env/itserviceops.yamlENV-OPS-105
product_type:
Executable RL training and evaluation environment
category:
IT Service Management · Identity & Access · Change Control
commercial_model:
Environment development and validated task packages
environment_id:
ENV-OPS-105
target_roles:
IT Service Desk Analyst · Systems Administrator · Identity & Access Administrator · Change Manager
core_competencies:
  • Triaging tickets and finding the shared cause behind many reports
  • Setting priority from impact and urgency
  • Recovering device and SaaS incidents through authorized, staged changes
  • Granting access by least privilege across joiner, mover and leaver events
  • Classifying changes and obtaining the right authorization
simulated_systems:
ITSM · Identity provider and SSO · Device management · SaaS admin consoles · Asset registry · Change management
interfaces:
Structured ITSM, identity, device and change tool API · Knowledge base search · Simulated admin actions
interaction:
Stateful, multistep tool use with simulated employees, approvers and change authorities. Device and sign-in health respond to the agent's actions.
state:
Tickets, identities and entitlements, device policies, SaaS configurations, changes and assets. Each episode starts from an isolated, seeded snapshot.
disturbances:
Floods of duplicate tickets, misleading user reports, change freezes, over-broad access requests and pressure to act without approval.
task_families:
  • TASK-OPS-105-01 · Service desk triage & major incident detection
  • TASK-OPS-105-02 · Device & SaaS incident recovery
  • TASK-OPS-105-03 · Access request handling
  • TASK-OPS-105-04 · Change & escalation management
evaluation:
Ticket and incident records, sign-in and device health after the episode, authorization of every change, entitlements against role profiles, and side effects the agent introduced.
reward:
State assertions on tickets, health, changes and entitlements, plus action constraints on change authorization and access.
training_use:
RL training, held-out evaluation and SFT on reference trajectories.
deployment:
Containerized, isolated execution, with a private deployment option.
licensing:
Commercial use rights. Exclusivity and ownership are agreed for each engagement.

Workflow coverage. What agents do, and what the verifier checks.

  • Workflow

    Service desk triage

    Example tasks

    Link a burst of sign-in tickets to one cause and set priority.

    What is verified

    The shared cause, linked tickets and the priority from the impact-urgency matrix.

  • Workflow

    Device and SaaS recovery

    Example tasks

    Revert a device-management profile that broke VPN, pilot group first.

    What is verified

    The change followed its catalog procedure and devices recovered.

  • Workflow

    Access management

    Example tasks

    Process a mover's access request.

    What is verified

    Entitlements match the new role's profile, and old access is removed.

  • Workflow

    Change management

    Example tasks

    Classify and schedule a batch of changes around a freeze.

    What is verified

    Each change has the right type, authority and window.

Task families. Each family has its own datasheet: prompt, verification criteria and failure modes.

Agent tools. What the agent can call.

The agent works the ticket queue, reads identity, device and change records, and acts through access updates, device-policy reverts and change requests. Device and sign-in health respond to what it does.

tools · env/itserviceopsJSON-schema tools
  • get_ticket
  • search_tickets
  • check_identity_provider
  • get_device_policy_history
  • query_device_health
  • revert_device_policy
  • get_entitlements
  • update_access
  • check_change_policy
  • request_change_approval
  • update_ticket

Verification methodology. Checked against state the verifier computes itself.

Ticket and incident records, sign-in and device health after the episode, change authorization, entitlements and any side effects the agent introduced.

The verifier checks

  1. 01Related tickets are linked to one incident with the correct cause and priority.
  2. 02Sign-in and device health after the episode meet the scenario's recovery objective.
  3. 03Every change has the right type, followed its catalog procedure or was authorized by the right change authority before execution.
  4. 04Entitlements match role profiles plus approved exceptions.
  5. 05No side effects outside the scope of the change, such as reverting unrelated profiles.
  6. 06Reference solutions and adversarial cases run against the verifier before training use.

Reward

Escalation or abstention can be the correct answer. The reward never favors faster, broader changes just to clear the queue.

Scores zero on

  • Executing a change without the required approval or catalog procedure
  • Closing an incident while users are still affected
  • Granting access beyond least privilege

Included deliverables. What you receive, and what we agree before we build.

Delivered with the environment

  • Containerized environment with simulated identity, device and SaaS systems
  • Validated task instances with reference solutions
  • Verifier and grading scripts
  • Agent traces and failure-mode analysis across repeated runs
  • Technical datasheet and integration documentation

Agreed before development

  • Change policy, standard change catalog, priority matrix and access profiles to model
  • Identity, device and SaaS systems in scope
  • Recovery objectives used by the verifier
  • Execution limits per episode
  • Acceptance criteria

Every environment is built to order. Tell us which workflow you need to train or evaluate, and where you are in the process.

Request sample tasksDiscuss custom developmentRequest technical integration detailsSee ways to buy