Almost every Azure DevOps engagement starts with the same question: what is actually in here? Before a migration, a process redesign, a licensing review, or a coaching engagement, somebody has to find out how many projects there are, which process each one uses, who has access to what, how much work is really being tracked, and which of all those repositories anyone still commits to. The answer the customer gives you on day one is usually a guess. The answer the organization gives you is the truth, but it's spread across dozens of settings pages.
We used to collect it by hand, clicking through Project Settings and filling in a spreadsheet. That took days, and was out of date by the time we finished. So we wrote a script to do it, used it on real engagements, and have now put it on GitHub.
github.com/accentient/ado-assessment · MIT licensed · PowerShell 7 or Windows PowerShell 5.1
What it reports
Point it at an organization and it assesses every project the PAT can see, a list of named projects, or the projects in a CSV file. It only ever reports what the token has access to: projects the token's owner can't see are left out, and one you name explicitly is reported as "not found or not visible" rather than assessed. For each one you get the process (including the parent of an inherited process), teams and their admins, security groups, people, work item counts per type, area and iteration paths, Git repositories, build and release pipelines, artifact feeds, wikis, test plans, service connections, and available agent pools. The report opens with an organization-wide summary table and closes with a work item type matrix across all projects, which is often the first place process drift becomes obvious.
A few details make the numbers trustworthy rather than just plentiful:
- Nested groups are expanded. "114 people" means 114 distinct identities, not 114 group memberships. Each person gets their roles (Project Admin, Contributor, Reader), teams, other groups, access level, and last access date.
- Work item types come from the process, not from assumptions, so custom types from inherited processes show up with real counts from Analytics.
- Repos and pipelines are profiled. Repository size, branch count, last commit and author, disabled and empty repos. Pipelines are split into YAML and classic, with paused and disabled ones flagged, plus the last run and its result.
- It degrades gracefully. If the token can't read an area, that area shows
n/a, the Warnings section explains why (including the server's own message, so you can tell a missing scope from a missing license), and the rest of the assessment carries on.
The output is Markdown for people and JSON for tools. The repo includes an example report from a real four-project organization, anonymized: every name is fictional, and the counts are real.
Read-only, and provably so
If you are going to run a script against a customer's production organization, the first thing they will ask is whether it can change anything. It can't. Every REST call goes through a single function, Invoke-AdoGet, that hard-codes -Method Get. That meant making a couple of deliberate choices: work item counts come from the Analytics OData service instead of WIQL, and identities are resolved with Graph GET endpoints instead of the subjectlookup batch endpoint, because both of the alternatives require a POST.
The token itself should put a security team at ease, too. It's a custom-defined PAT with Read checked on a short list of scopes (Project and Team, Work Items, Code, Build, Graph, and a few others) and nothing else: no Write, no Manage, no Full access. The README lists each scope, what it's used for, and what you lose without it. It's scoped to the one organization being assessed, and you can give it a short expiration and revoke it the moment the assessment is done. Even the token's storage is conservative. It lives in Windows Credential Manager, never in the script, the repo, or the report, and it's never echoed to the console. The only things the tool writes are the report files on your own disk.
You don't have to take our word for it. The Pester test suite parses the script's syntax tree to confirm that only Invoke-AdoGet makes web requests and that every one is a body-less GET, then checks every request at run time as well. The tests are hermetic and never touch a real organization. The web and credential cmdlets are replaced with guards that throw, so any code path that tries to reach the network or Credential Manager fails the test. The end-to-end tests then mock the single web cmdlet Invoke-AdoGet calls, so every request is answered by a small fake organization instead of Azure DevOps. That's enough to run a full end-to-end assessment, including a project the token can only partly read, and check the report and JSON that come out. GitHub Actions runs the suite on both PowerShell 7 and Windows PowerShell 5.1 on every push.
Getting started
There are no modules to install. It's one script and three switches. Store a read-only PAT in Windows Credential Manager (the script can prompt for it and save it for you), set the organization URL at the top of AssessProjects.ps1, choose -All, -Projects, or -CsvPath, and run it. For a complete picture, the token should belong to a Project Collection Administrator with a Basic (or higher) access level. The report and JSON land in a gitignored reports/ folder, and the results stay in $Assessment afterwards so you can explore them in the terminal.
Two things to know. This is for Azure DevOps Services; Azure DevOps Server isn't yet supported. Contact us if you have that need. And the report contains names, email addresses, group memberships, and last access dates for everyone in the organization, so treat it like the personnel data it is.
Inventory, not judgment
The tool shows what's there. It doesn't score the organization or tell you what to change, and that's on purpose. Knowing that a project has 109 Contributors and 4 Readers, or that 9 of its 39 repositories are disabled, as in the example report, is the start of a conversation, not the end of one. Deciding what those numbers mean for your teams, your process, and your plans is the hard part. If you'd like help with that, whether it's an assessment, a migration, or improving your teams' agility, get in touch.