Skip to content
← Back to blog
multi-agentaimicrosoftengineering

Building TSGBuilder: A Multi-Agent System That Actually Ships

How I built a multi-agent pipeline at Microsoft that researches, writes, and reviews troubleshooting guides — and what I learned about making agentic systems reliable.

There's a gap between AI demos and AI that ships. I've spent the last year closing that gap with TSGBuilder — a multi-agent system that auto-generates troubleshooting guides for Azure AI support engineers.

The Problem

Support engineers write troubleshooting guides (TSGs) to help diagnose and resolve customer issues. Good TSGs require:

  • Deep research across documentation, GitHub issues, community forums
  • Structured writing that follows established templates
  • Technical review for accuracy and completeness
  • PII scrubbing to ensure nothing sensitive leaks through

The manual process takes hours per guide. With hundreds of topics to cover and products evolving constantly, the backlog never shrinks.

The Architecture

TSGBuilder uses a three-stage pipeline:

  1. Research Agent — Crawls official docs, GitHub repos, community sources, and internal knowledge bases. Produces a structured research brief.
  2. Writer Agent — Takes the research brief and generates a TSG following the team's template and style conventions.
  3. Reviewer Agent — Validates the output against a quality checklist, flags gaps, and runs automated PII detection.

Each agent is a focused unit with clear inputs and outputs. No god-agent that tries to do everything.

What I Learned

Agents are only as good as their constraints. Giving an agent free rein to "write a troubleshooting guide" produces slop. Giving it a specific template, research context, and quality criteria produces something useful.

PII detection can't be an afterthought. We block sensitive data at the pipeline level — before any human sees the output. This isn't a nice-to-have when you're working with enterprise customer data.

Adoption beats perfection. The first version was rough. But it saved enough time that engineers started using it immediately, which gave us feedback to make it better. Ship early, iterate with real users.

Results

  • 70–80% reduction in guide authoring time
  • Rolling out across Microsoft support engineering teams
  • Feedback from the field: "great tool man, amazing work" and identified as a critical need in Government sector support

The best compliment: engineers stopped asking how it works and just started using it every day. That's when you know a tool has crossed from demo to product.