Dry Run Testing

Preview and calibrate custom Drixy rules against historical pull requests before enforcing them across your engineering team.

When introducing a new security rule or style guideline, the biggest risk is false positive noise that frustrates developers and slows down release velocity.

ScanDrix includes a native Dry Run simulation engine that allows architects and security leads to test new rules against past pull requests before enabling them in production.

Why use Dry Run?

  • Zero Developer Friction: Test rules completely in the background without posting comments to live pull requests.
  • Accurate Hit Rates: Measure exactly how many historical PRs would have triggered the rule over the last 30, 60, or 90 days.
  • Noise Calibration: Inspect AST matches and adjust query selectors or threshold limits until false positives reach 0%.

Running a Dry Run from the Dashboard

1

Navigate to Rules Editor

In the ScanDrix dashboard, go to Rules → Create Rule or edit an existing custom rule.

2

Enter your AST Pattern or Policy

Define your rule's YAML definition and AST pattern matcher:

yaml
match:
  ast_call:
    - "jwt.Parse($TOKEN)"
  where:
    $TOKEN: "!calls(verifyKey)"
3

Select Simulation Scope

Click Run Simulation (Dry Run) and select the evaluation scope:

  • Select target repositories or choose all organization repos.
  • Set the lookback window (e.g. Last 50 Pull Requests).
4

Review the Impact Report

ScanDrix parses historical diffs against your AST matcher and produces an interactive impact matrix showing:

  • Total triggers across sample PRs.
  • Diff line snippets showing where the comment would have landed.
  • Estimated review delay and developer interaction metrics.
5

Promote to Production

Once satisfied that the rule only catches genuine violations, toggle the rule status from Draft / Dry Run to Active.

Running Dry Runs via the CLI

You can also run dry-run simulations locally in your terminal or inside CI pre-flight checks:

bash
# Test a new local rule file against the current branch diff
scandrix dryrun --rule ./rules/sec-auth.yaml --branch main

# Simulate rule against a specific remote pull request
scandrix dryrun --rule ./rules/sec-auth.yaml --pr 142