Monitors are Fragile under
Information Asymmetry

Partial knowledge does not improve monitor effectiveness. Figure 1a exhibits consistent patterns across vulnerabilities. First, including all information substantially reduces attack success, as the drop from partially to the fully informed scenario is large for most of the vulnerabilities. Second, and more strikingly, knowing about partial information, not including the target vulnerability, provides almost no benefit (and is, in some cases, actually harmful) when compared to the case where no information is provided. This suggests the monitor's detection capability is highly specific, and knowing about other vulnerabilities does not transfer to catching a different one it was never informed of.

Structural similarity between vulnerabilities does not generalize. Figure 1b isolates the symlink vulnerability family, testing four closely related variants (symlink_cache, symlink_log, symlink_run, symlink_spool). Even among variants that share the same underlying exploit mechanism, knowing about similar vulnerabilities in the family does not help the monitor detect the target one, and the all_but_target column remains elevated across all four. This rules out the possibility that the transfer failure observed in Figure 1 is simply due to vulnerabilities being too dissimilar, as even structurally similar attacks do not benefit from partial knowledge.

Figure 1: Attack success rate measured under three monitor information conditions at threshold 0.75 for both vulnerability sets described in Table 1 and Table 2 under three monitor information scenarios. We observe that in most cases the attacker success rate mostly increases when the monitor is partially informed.

Figure 2: Distribution of scores across three monitor information scenarios for the vulnerability set described in Table 1. As we could infer from Figure 1, the monitor starts giving more high scores to untrusted model attacks only when fully informed.

Our results support the central claim of this work that monitors are fragile under information asymmetry. Across our experiments, detection capability does not transfer across vulnerability types, does not improve with partial knowledge of the attack surface and does not generalize even within structurally homogeneous vulnerabilities. We interpret this as evidence that the gap between the fully informed and uninformed conditions reflects a dependence on explicit prior knowledge that has direct implications for how much safety current control evaluations calibrated against known attacks actually provide.

In real-world deployments, this highlights the importance of carefully defining the monitor’s threat model. A monitor trained or prompted on a fixed set of known vulnerabilities may overfit to those patterns, and fail to detect attacks that fall outside this set. As a result, if an attacker exploits an unknown vulnerability, the monitor’s detection capability can degrade significantly, despite having prior knowledge of other, related threats.

Limitations. The present work also has limitations worth noting. Due to time constraints, results are based on a single model pair, so it remains an open question whether the observed results generalizes across different model capabilities. The attack policy also operates in a single-step setting, which is a simplification of real-world deployments where attacks are more likely to unfold across multiple actions.

Our experimental setup also assumes that the attacker has perfect knowledge of the vulnerability it aims to exploit, while the monitor does not share this knowledge and instead relies only on its own prior information. This creates a clear asymmetry for evaluation, but may overstate the attacker’s advantage relative to real-world scenarios, where both parties operate under uncertainty.

Future Work. First, our setup evaluates monitor performance at three discrete knowledge levels, but a more complete picture could be obtained by adding vulnerabilities to the monitor's informed set one by one, exploring how detection capability evolves as monitor prior knowledge grows. Another natural extension of our work is running our experiments in a multi-step agentic setting, which changes both attack success rates and monitor behavior in non-trivial ways, reflecting a more realistic threat model.