Prompt Security Leaderboard

Overview

Model Robustness

Sorted by no-defense ASR, lower is better. Clean ASR is the no-attack harmful-query baseline; induced harmfulness is computed on matched sample identifiers.

Primary judger: -

Black-Box Models

Hosted API endpoints under black-box attack access.

White-Box Models

Locally served open-weight models; white-box-only attacks are access-compatible only here.

Defense Effectiveness

ASR Gain uses matched no-defense counterfactuals. Utility delta compares each defended endpoint with its own no-defense benign-question baseline.

Black-Box-Compatible Defenses

White-Box-Only Defenses

Attack Strength

Sorted by defended residual ASR. No-defense ASR, induced harmfulness, suppression, cost, and cross-model spread are reported separately.

Black-Box Attacks

White-Box-Only Attacks

Model x Attack: No-Defense ASR

Only no-defense runs are shown. Rows split black-box and white-box models; columns split black-box and white-box-only attacks. Missing cells are not filled from defended runs.

Defense x Attack: Defense Gain

Entries are matched ASR reduction in percentage points. Positive values reduce harmful outputs; negative values indicate defense backfire.

Comparison under Evaluation Protocol

Compare paper-compatible runs by model, attack, defense, sample, and independent judger. Click a row to inspect its local placeholder samples.