One of our beta teams stopped opening Qualyn for three weeks. Nothing dramatic behind it: a launch crunch, then two of their four engineers on holiday, and the tab quietly fell out of the morning routine.
When they came back, the release score on their main repo read 49. It had been 63 when they left. The first message we got asked, politely, whether we had changed the scoring.
We hadn't. What changed was three weeks of evidence that nobody was reading.
The 63 they left behind
At 63, the repo sat in amber. Tests were mostly green, changed-code coverage was acceptable, and there was a backlog of critical and high findings the team knew about. It was the kind of list that gets a Slack message saying "we should book a day for these" and never gets the day.
That state is common, and it is livable. Plenty of teams ship from amber for months. But unresolved severe findings age, and in Qualyn's scoring, age makes them more expensive over time.
Three quiet weeks
Nobody stopped shipping. Releases went out on green CI, small ones, and Qualyn kept scanning each candidate in the background. Nobody read the calls.
Meanwhile the risk moved on its own. Two high-severity security advisories arrived through dependency review, one of them in a library that sits close to the payment flow. A cluster of open findings crossed Qualyn's 30-day staleness threshold, where the age multiplier steps up from 1.0 to 1.25. And a few new critical and high findings from those in-between releases joined a backlog nobody was triaging.
None of this shows up in a diff. A repo can get riskier in three weeks in which almost nothing about the code itself got worse.
- 2 high-severity security advisories arrive via dependency review
- Open findings cross the 30-day staleness threshold (age ×1.25)
- New critical/high findings join an untriaged backlog
Reading the 49
The drop had two drivers, and the score's breakdown named both. First: two high security findings that needed either mitigation or explicit risk acceptance. High security findings act as a hard gate in Qualyn's scoring, so until someone fixes them or formally accepts them, they cap the score no matter how healthy the tests look.
Second: 25 unresolved critical and high backlog findings, carried as residual risk. Qualyn does not pretend it re-observed all 25 that morning. Each one is weighted by verification confidence and age, so an old unverified finding counts for less than one confirmed in the current scan, but it still counts. Twenty-five of them, aging together, add up.
That breakdown mattered more than the number. A score that drops 14 points and can't say why gets argued with. This one could be walked through line by line, which turned the conversation from "is the tool being dramatic?" into "which of these do we fix first?"
The surprise is the product working
Here's the honest part: Qualyn discovered almost nothing new that morning. Most of the 25 findings were already on the list three weeks earlier, when the score was 63 and the mood was fine. The two advisories would have arrived whether or not anyone was watching.
Nothing broke in production during those weeks either. Shipping on green CI through the gap worked out, and pretending otherwise would be revisionism. The 49 was the same risk the team had been carrying, three weeks older, with two new security findings priced in.
Risk doesn't pause because attention does.
What it took to climb back
The way back was mechanical. One of the two security findings was a dependency upgrade, tested and shipped the same afternoon. The other needed a small code change, so the team accepted the risk in the meantime, with an owner, a reason, and a two-week expiry. Expired acceptances return to open, so this is a bridge, not a hiding place.
Then a single triage hour on the backlog. Nine findings were healed outright, several because the code that produced them had since been rewritten and revalidation confirmed the risk source was gone. Six were accepted with owners and expiries. The rest got scheduled. A week later the score sat at 68, and the team could say exactly what would change the answer on their next release.
Key takeaways
- Release risk drifts on its own: advisories land, findings age past thresholds, and an untriaged backlog compounds even when the diff barely moves.
- High security findings need mitigation or explicit risk acceptance. Until one of those happens, they cap the score however green the pipeline is.
- Recovery is mechanical, not heroic: fix what is fixable, accept what you are genuinely accepting (owner, reason, expiry), and let revalidation clear what is already gone.
Frequently asked questions
Can a release score drop without code changes?
Yes. The score reads evidence, and evidence moves on its own: new security advisories arrive through dependency review, open findings cross age thresholds and weigh more, and risk acceptances expire back to open. A repository left alone tends to drift down, not hold steady.
Why did this release score fall from 63 to 49?
Two reasons. Two high security findings needed mitigation or explicit risk acceptance, which acts as a hard gate on the score, and 25 unresolved critical and high backlog findings were carried as residual risk, weighted by age and verification confidence.
What is residual risk in Qualyn's release score?
Unresolved critical and high findings from earlier scans that remain open. They are carried into the score weighted by verification confidence and age, so an old unverified finding counts for less than one re-observed in the current scan, but it does not silently disappear.
What does mitigation or risk acceptance mean for a security finding?
Mitigation removes the risk: a fix, an upgrade, or a configuration change. Risk acceptance means a named owner records why the team is shipping with the finding open, with a reason and an expiry date. When the acceptance expires, the finding returns to open and affects the score again.
Does a score of 49 mean the release cannot ship?
No. It means the release is red and the recommendation is to hold, with the reasons listed. The reasoning is inspectable, so a team can challenge it, accept specific risks explicitly, or fix the gating findings and re-run. The score is a recommendation with evidence, not a lock.
How do teams recover a dropped release score?
Address the hard gates first, since they cap the score: mitigate or formally accept the severe security findings. Then triage the backlog: heal what is fixable, close what revalidation shows is already gone, accept what you are knowingly carrying, and schedule the rest.