By General Applications · October 25, 2026 · 6 min read

Owning the Black Box: How to Audit Systems When the Builders Are Gone

Software ArchitectureOwners AdvisoryTechnical Governance

Most businesses have one. A system that runs payroll, ships orders, or keeps the line moving, built by someone who left years ago. Nobody wrote it down. It works, so nobody touches it, and every year it gets harder to find out how.

Sooner or later you have to decide what to do with it: keep supporting it, replace it, move it to a hosted service, or leave it alone. A good decision needs facts, and the people who had them are gone.

The good news is that a system can't hide what it does. A technical audit collects that evidence, writes it down in one place, and turns it into findings you can act on. Here's how to audit an undocumented system without relying on anyone's memory.

Gather the Evidence the System Leaves Behind

Five sources cover most of what you need.

Network traffic: A few hours of capture shows which systems talk to which, and how often. It surfaces the connection nobody listed, and the server that looks idle but may run a job once a month.

Configuration files: Firewall rules, server settings, and DNS records show what someone once meant the system to do. Compare them with the traffic and you find where intent and reality have drifted apart.

Logs: Application and system logs show which failures are routine. A restart that happens every week is either a scheduled job or a scheduled failure. You need to know which.

A physical walk: Walk the server room and the wiring closets and read the labels. This is how you find the unmanaged switch under a desk that half the office depends on. No scan reports it. For cloud systems, the account's resource list and billing statement do the same job.

The people who get called: Ask who gets paged when it breaks and what they do first. Their answer often reveals a dependency no diagram shows.

No single source is complete, so cross-check. A system confirmed by traffic, a config, and a label is real. A system seen in only one place is an open question.

Build an Asset Inventory and Dependency Map

Write everything into one record as you go. Notes you plan to write up later rarely get written up. A spreadsheet is enough to start, with four parts:

Keep the record somewhere the owner controls. Documentation that lives on the auditor's laptop recreates the problem the audit was meant to solve. If the audit points toward replacement, the same record carries straight into legacy system modernization.

Use Tools for Discovery and People for Confirmation

Scanners and scripts do the tedious part well. They inventory hosts, flag outdated software, and compare configurations faster than any person can.

What they can't tell you is what happens when a box goes down, who could bring it back, or whether its support contract lapsed. Treat tool output as a draft. An entry is confirmed when a person has checked it against a label, a config, or the people who use it.

Find the Risks That Matter

An inventory lists everything. A useful audit tells you what deserves attention. Ask three things of every entry.

What stops: Trace each system to the work it supports. A reporting server nobody reads can fail quietly. A switch that carries every order-entry terminal can't fail for a minute. Write the impact in business terms: "order entry stops" means more to an owner than a network segment number.

Who can fix it: If only one person can restore a system, that's a single point of failure, however healthy the hardware is. The same goes for a vendor whose support contract has run out.

What is failing quietly: Expiring certificates, failed backup tests, nearly full disks, hardware past end of support. Nothing is broken yet, so these are easy to miss. They're also the cheapest findings to fix.

Write Audit Findings an Owner Can Fund

A finding is only useful if the owner can make a decision with it. That takes four things: what you saw, what it puts at risk, what it would cost to fix, and where it ranks against everything else.

The weak column is what a scanner produces. The usable column comes from someone who traced the switch to the work it carries.

Put every finding in one ranked list, so the owner knows what to fund first. Include the cost of doing nothing, too. Some systems are right to leave alone, and saying so is part of the job.

Check What Your Vendors Tell You

Each vendor audits its own piece and stops at the edge of its contract. The problems in an undocumented system usually live between those edges: the shared file server, the integration nobody owns, the remote-access account two vendors use.

Someone on the owner's side has to hold the whole picture. That can be an employee or an outside owner's engineer. Their job is to keep the record, audit the gaps between vendors, and check what vendors claim. When a vendor says the backups work, ask when they last restored one, and whether the application actually came back.

Keep the Documentation Alive

An audit is wasted if the record goes stale. Keep it where the on-call person uses it during an incident, so it gets corrected every time it's wrong. Make every change that adds, moves, or retires a system update the record as well.

Write down why things exist, next to what they are. A firewall rule with no reason attached gets deleted in the next cleanup and restored in the next outage.

Where to Start

Pick the system whose outage would hurt most and audit backward from it: what it runs on, who can fix each part, and what evidence backs each entry. A focused first pass produces real findings quickly and shows what a full audit would deliver. Then move to the next system, building on what the first pass already confirmed.