蛋白质语言模型审计:置信度门控预测与重复引导的边界
The Confessions of a Protein Language Model: Steering, Activations, and What Predicted Confidence Cannot Certify

预印本审计已发表的重复引导、ESMFold置信度与激活读出,发现引导在操作点保持置信度,但拟合悬崖出现在 alpha_50 = 1.240x(扫描自身对照 64.0%)。无引导生成器激活可预测置信度门控(ROC-AUC 0.786 ± 0.004),但在组成与长度匹配的设计-打乱配对上,实验稳定性的平衡准确率区间均未排除随机。预测门控并不等于可折叠性。
原文摘要
出版方登记 · CrossrefProtein generate-and-screen pipelines use interventions and readouts to select candidates, but rarely test whether these track their intended targets. We audit published repetition steering, ESMFold confidence, and an activation readout. Steering preserves confidence at its operating point, but a fitted cliff follows at alpha_50 = 1.240x (sweep's own control: 64.0%). Direction matters at moderate matched-norm push, but not at saturation. Unsteered generator activations predict the confidence gate (ROC-AUC 0.786 +/- 0.004), supporting cost-aware, model-specific triage. We also replicate established length sensitivity, through which shortening can manufacture false nulls. Yet on composition- and length-matched design--scramble pairs, no balanced-accuracy interval excludes chance for experimental stability. Predicting the gate does not establish foldability. We should test what a readout measures before using it to select. Code and data: https://github.com/paninishah/plm-steering-audit
页面未能读取全文,以上为出版方登记的摘要;方法与图表见原文
来源:bioRxiv 生物信息学 · biorxiv.org