How Engineering Managers Quantify Application Security Risk

1"How Many Open Findings Do We Have" Is Not a Metric

Ask an engineering manager how secure their organization is, and the honest answer is usually a shrug followed by a number pulled from whatever scanner ran most recently. That number is almost useless on its own. Twelve open findings could mean a healthy, actively-triaged backlog, or it could mean twelve critical issues that have sat untouched for months. The finding count tells you nothing about direction, and direction is what a manager actually needs to answer for.

Quantifying risk isn't about producing a single scary or reassuring number. It's about building a small set of signals that change meaningfully over time, so a manager can tell the difference between "we're behind and it's getting worse" and "we're behind and catching up."

2Engineering KPIs That Actually Translate to Risk

Most security tooling reports findings. Few report the KPIs that make findings meaningful to someone managing a team rather than fixing the code directly. Four are worth tracking deliberately:

• Time-to-remediate by severity. Not an average across all findings — averaging a five-minute config fix with a two-week architecture change produces a number nobody can act on. Track median time-to-fix separately for Critical, High, Medium, and Low, and watch each trend independently.
• SLA compliance rate. If your team has committed to fixing Critical findings within, say, five business days, what fraction actually met that window this month? This is the single number that most directly answers "are we keeping our commitments," which is usually the question leadership is actually asking.
• Backlog age distribution. Not "how many open findings," but how old is the oldest one in each severity band. A Critical finding open for six months is a materially different story than the same finding open for six days, even though both show up identically in a raw count.
• Accepted-risk count and age. Findings marked "won't fix" or "accepted risk" are a real, tracked category of organizational risk — not resolved findings. They deserve their own metric, including how long ago each was accepted and whether it's been reviewed since.

Every one of those four KPIs is a query over the same underlying record, not four separate tracking systems. The fields that make the queries possible are the ones that matter — a finding record needs to carry enough state to answer "how long has this been open" and "did we meet our commitment" without a human reconstructing the timeline by hand:

{
  "id": "f_8a21c4",
  "severity": "critical",       // critical | high | medium | low
  "owasp_category": "M9",       // where applicable
  "cwe": "CWE-798",
  "project_id": "proj_mobile_ios",
  "discovered_at": "2026-05-02T10:14:00Z",
  "sla_deadline": "2026-05-09T10:14:00Z",   // per-severity policy, see below
  "status": "in_progress",      // open | in_progress | fixed | accepted_risk
  "resolved_at": null,
  "accepted_at": null,
  "accepted_by": null,
  "accepted_reason": null,
  "review_due": null
}

Median time-to-fix, SLA compliance rate, and backlog age are all aggregate queries overdiscovered_at, sla_deadline, resolved_at, andseverity. Accepted-risk age is the same query, filtered tostatus = accepted_risk and measured against accepted_atinstead. If your tracking system can't answer those queries directly, the dashboard sitting on top of it is doing guesswork.

3From Individual Scans to Portfolio Visibility

A single scan report describes one application at one point in time. Organizational risk visibility requires rolling that up across every project without losing the severity and age detail that makes the rollup meaningful. The shape of that pipeline looks roughly like this:

The failure mode most teams hit is stopping at the first stage — each project has its own dashboard, and "portfolio visibility" means opening five tabs and doing the aggregation by hand. The second stage is the one that actually turns individual scan output into something a manager can use without becoming a part-time data analyst.

4Technical Debt Is a Backlog Category, Not a Feeling

"We have security technical debt" is a true statement for almost every organization and an actionable one for almost none, because it isn't tracked as a discrete, countable thing. Treat it as one: every finding marked as an accepted risk is technical debt by definition, and it should carry the same fields a financial ledger would — who accepted it, when, why, and when it's scheduled for re-review.

Without that structure, accepted risk quietly becomes permanent risk. Nobody decided that on purpose; it just never came back up because nothing forced it to. A backlog age metric applied specifically to the accepted-risk category — not just open findings — is what keeps that from happening silently. In practice that means an acceptance record looks less like a checkbox and more like a decision log entry:

{
  "finding_id": "f_2c918e",
  "status": "accepted_risk",
  "accepted_at": "2026-03-11T00:00:00Z",
  "accepted_by": "j.rivera@example.com",
  "accepted_reason": "Endpoint requires internal VPN; compensating "
                    + "control tracked in INFRA-4471. Revisit when "
                    + "VPN requirement is removed.",
  "review_due": "2026-06-11T00:00:00Z"
}

5Prioritizing the Backlog

Severity alone is a weak prioritization signal, because it describes the finding in isolation and ignores whether it's actually reachable, how much engineering effort a fix requires, and what it would cost the business if exploited. A deeper look at combining severity with exploitability and business context lives in Vulnerability Prioritization Beyond CVSS — the short version for a manager is: use severity to sort within a priority tier, not to set the tier itself.

A simple triage table turns that judgment into something consistent enough to apply without re-litigating every finding individually:

6What an Executive Summary Should Actually Contain

A report that leadership will actually read is short and answers a fixed set of questions every time, rather than presenting a variable-length list of findings. A useful structure:

Notice what's absent: a list of every individual finding. That belongs in the tool your engineers work from day to day, not in the document leadership reads once a month. The two audiences need different altitudes on the same underlying data.

7Common Mistakes

• Averaging time-to-fix across all severities. A single blended average hides whether Critical findings are actually being fixed fast — a handful of quick Low-severity fixes can make the overall number look healthy while every Critical sits untouched.
• Treating "zero open Criticals" as "secure." It often just means the Highs are aging into Criticals next quarter, or that Critical-severity findings are being reclassified as High to keep the headline number clean.
• No review cadence on accepted risk. Without a review_dueforcing function, "accepted for now" silently becomes "accepted forever" — nobody revisits it because nothing requires them to.
• Confusing scan cadence with data freshness. A weekly scan produces a risk trend with weekly resolution. Presenting it as if it reflects daily reality overstates how current the picture actually is.
• No audit trail on status changes. If any engineer can mark a finding "fixed" or "accepted risk" with no record of who and why, the metrics built on top of that status are only as trustworthy as everyone's memory.

8Limitations of a Metrics-Only View

A dashboard can be precise and still wrong. If the underlying scanner has a systematic false-negative gap — a finding category it simply doesn't check for — every metric built on top of its output will look clean while missing that entire category of risk. Metrics are also gameable: a team under deadline pressure can mark findings "accepted risk" faster than it fixes them, and a trend line alone won't distinguish genuine improvement from a change in how liberally that status gets applied. Periodic manual review of a sample of findings — not just trust in the dashboard's own numbers — is what keeps the measurement honest.

9How This Maps to Strata

Strata's risk dashboards track the signals above — severity trend, SLA compliance by policy, backlog age, and accepted-risk history — across every mobile app and repository in a portfolio. See AppSec Reporting for Managers for how the platform applies this directly, or AppSec Teams for the practitioner-facing workflow underneath it.