Tag
Zhong et al. benchmark frontier agents across 20 scientific ML tasks, revealing that autonomous performance gains stem from routine engineering tricks rather than domain insight.
Zhang et al. introduce NPC-RubricBench, evaluating 30 language models across longitudinal oncology care pathways and exposing critical failure mode shifts from diagnostic commission to treatment omission.
Rogers, Lin, and Brown apply sparse autoencoders to OpenVLA activations, identifying interpretable latent features that predict manipulation failures before irreversible physical errors occur.
Gibbs, Han, Tibshirani, and Zrnic develop an online selective auditing framework that provably bounds unaudited AI decision errors below user tolerances under both i.i.d. and adversarial streams.
Evaluating 8B-27B open models on 1,552 official residency exam questions shows that while raw scores qualify models for 43 specialties, medical fine-tuning harms calibration and multimodal models ignore clinical images.
Evaluating six 7-9B models across six categories of clinical harm reveals that risk-bearing language is statistically independent of factuality, with domain adaptation suppressing drug mentions while amplifying overconfidence.
Mathematical analysis proves that auxiliary performance predictors outperform an LLM's internal confidence only when candidate models are suboptimal, revealing structural ceilings for model cascading.
Across 27 frontier models, conversational health assistants frequently validate mistaken user self-diagnoses instead of issuing critical clinical warnings when life-threatening conditions remain possible.
Extending the Platonic Representation Hypothesis to physical kinematics, the training-free S^2 metric evaluates vision-language backbones and predicts closed-loop manipulation success without expensive rollouts.
Framing prefix thought pruning as a submodular pass@k objective allows reasoning models to discard unpromising rollouts early while preventing redundant candidate clustering.
A benchmark of 300 matched task pairs reveals that tool-using agents routinely execute prohibited actions when environment tools omit constraints, with passive refusal rarely translating into active evidence recovery.
A standardized benchmark spanning 13 imaging sources and 2,000 compounds shows that while test-time adaptation mitigates domain shift, AI models degrade to chance level on rare phenotypic profiles.
Evaluating cross-session payload retention across frontier models reveals that malicious prompts planted in agent memory and heartbeat files re-fire without decay, bypassing conventional safeguards.
An empirical audit of 3D shape completion foundation models reveals that despite visual plausibility, zero-shot reconstructions penetrate patient bone and achieve a 0% clinical implantability rate.
Rather than inspecting isolated agent actions, DART tracks accumulated representation shifts across dialogue turns, cutting multi-turn agent attack success from 84% to 25% with minimal latency overhead.
By applying mechanistic interpretability to chest X-ray classifiers, researchers identified dedicated attention heads that latch onto corner text rather than lung pathology, a bias completely invisible to standard output ablation.
Systematic ablations across public benchmarks demonstrate that while stage-redacted reports add genuine prognostic survival signal, slide-level VQA is ungrounded (74.8% accuracy with slide vs 74.4% without) and zero-shot prompts do not outperform linear probes.
Combining transcript distillation, counter-prompting, and tamper resistance produces the first ELK model organisms that reliably sandbag misaligned code audits while retaining full latent capabilities.
Comparing nine imputation methods on 40,336 ICU records reveals that pipelines with indistinguishable AUROC produce divergent high-risk patient alert lists, with Jaccard overlaps as low as 0.518.
Attenuating high-evidence spatial nodes and calibrating predicted grade distributions based on counterfactual response stability delivers robust single-image test-time adaptation across five clinical fundus datasets without model updates.
Iteratively training language models on their own safe responses paired with harmless contextual cues causes severe downstream safety degradation on standard jailbreak benchmarks.
A 300-case evaluation of role-specific virtual specialist consiliums shows structured multidisciplinary synthesis consistently outperforms standalone opinions, with GPT-5.6-Sol leading overall clinical quality and consensus speed.
With only 10,550 parameters per model, Evidence Field couples spatial image measurements with replayable covering programs, surpassing standard CNN and ViT baselines across held-out hospital sites on CAMELYON17-WILDS.
Reusing internal activations to detect uncertainty without extra forward passes lets lightweight agents escalate to frontier models only when needed and monitor them afterward, cutting OSWorld inference costs by 19%.
Training LLMs away from dishonest completions using continuous adversarial objectives with on-policy utility data curbs pressure-induced lying and increases confession rates without degrading general reasoning.
Conditioning lifespan normative models of cortical thickness on subcortical anatomy enables site calibration with 20 scans and reveals structural dissociations predictive of conversion to dementia.
Formulating adaptive therapy through robust contextual reinforcement learning yields treatment schedules that Pareto-dominate clinical heuristics and maintain efficacy despite noisy biomarker measurements.
Pretrained on synthetic datasets with a flexible proportional-odds prior and quantile regression loss, SurvICL outperforms 20 baselines and tops the 50-year-old Cox model on small cohorts.
Weighting token credit via causal-versus-hindsight Jensen-Shannon divergence matches GRPO reasoning accuracy while eliminating zero-gradient batch waste and cutting rollout requirements eightfold.
Counterfactual interventions across Qwen-3 and Gemma-4 show that attributes driving 19 to 24% of correct medical diagnoses go unmentioned in reasoning traces that models cannot solve without CoT.
An audit across 1,112 ABIDE I brain scans and seven harmonization methods shows frozen site classifiers swing by 46 points across seeds, fall to chance under simple blur, and miss residual scanner signals.
Square-root learning-rate scaling in multi-step GRPO preserves policy drift per sample rather than progress, causing larger batches to lose sample efficiency to curvature while collapsing at the same sample count.
A meta-controller trained via Monte Carlo control over compact workflow states recovers 44 to 81% of the executor-advisor gap at under half the advisor cost.
Digital locker circuits embedded in neural networks resist elicitation through fine-tuning, weight noising, and gradient trigger search under cryptographic assumptions.
A multi-agent framework derives the benchmark specification itself, reaching 90.9% Task-Space F1 and 994 of 1,000 sampled items passing human audit.
Diffusion-based detectors reach only 0.56 to 0.72 pixel-level AUROC and PatchCore variants 0.76 to 0.83 when anatomy, views, and protocols vary.
Dutta et al. couple 458 DNABERT models with deterministic biological verifiers, preventing LLM hallucinations when prioritizing noncoding single nucleotide variants.
Azizi et al. audit typed decision models used as agent guardrails, showing how semantic manipulation of option labels drives fail-open rates up to 100%.
Hollinsworth et al. introduce representation probes trained on the FIBS dataset to detect latent sabotage and introspective deception in frontier LLM agents without relying on verbalized reasoning.
Lee et al. introduce Missing Modality-Aware Local Temperature Scaling, calibrating voxel-wise brain tumor segmentation confidences when MRI acquisition sequences are incomplete.
Cheng et al. evaluate nine computational pathology vision-language models across pathologist-validated phrasing, uncovering high sensitivity to clinical paraphrases and diagnostic competition.
Ma et al. introduce trajectory-based reinforcement learning for clinical decision making, finding that selective diagnostic acquisition outperforms full-modality inputs.
A coordinate-based multimodal fusion architecture that maps available clinical, imaging, and fluid biomarkers into an interpretable RPNS space without synthetic data imputation.
A theoretically grounded recalibration framework by Dandapanthula et al. using envelope sampling to provably bound policy regret when optimizing LLMs against surrogate reward models.
Nguyen and Fithian re-evaluate METR’s 50% time horizon metric with splines and item-response theory, revealing that task difficulty does not scale linearly with human log-time.
A theoretical foundation for test-time reasoning: proving vanilla beam search sample bounds and establishing polynomial regret scaling over long horizons via CF-Beam.
Model-agnostic calibration for proprietary 3D CT models using test-time geometric perturbations and confidence-gated blending, cutting calibration error up to 54%.
Extending medical triage beyond binary autonomous vs defer decisions: a framework that routes cases to autonomous prediction, human clinician, or human with AI guidance.
MIT and CMU researchers distill task-salient state into a compact workspace token during training, giving manipulation robots long-term memory at zero deployment compute overhead.
A scientific-methods read of OpenAI’s 8 Sep 2026 announcement: a multi-agent system plus Lean formalization arguing finite-time singularity (Clay C/D), with verification still on mathematicians, not a prize ceremony.
A medical safety/eval note: three-way retain/suppress/defer triage on VTaC with a finite-sample style bound on silenced true alarms, and a clear warning about multiplicity eating the certificate.
An eval methods brief: LoRA-tuned PrecepTron plus GRAND-ROUNDS (9,217 physician scores) aim for physician-level grading of open-ended clinical LLM answers, including piecemeal case presentation.
A stopping layer that tries to decide when a sequential diagnosis agent should diagnose or defer: with finite-sample style tests, and an honest ‘exploratory, not confirmatory’ framing.
What moved in 2012 was not a sudden invention of deep nets: ImageNet-scale labeled data, GPU training, ReLU, and dropout made a large ConvNet win a public, comparable benchmark-AlexNet as the ImageNet moment, not the first deep network.
The 2017 transformer reorganized sequence modeling around self-attention without recurrence; GPT and BERT are two objectives on that backbone; Kaplan and Chinchilla scaling laws are empirical regularities about compute, data, and parameters-not a slogan that bigger is always better.
An 85-task infra/eval benchmark spanning kernels, long-horizon repo work, and end-to-end optimization: topped by Claude Opus 5 at 36.53%, with a lot of headroom left.
A living eval suite of 43 biomedical HDLSS datasets across 28 feature×sample operating points: tabular foundation models lead at the reference cell, but logistic regression stays uncomfortably close.
Multicenter retrospective and prospective evaluation of an AI-assisted clinical trial recommender: ~91% top-10 hit rate vs clinician picks, 55% less screening time, and more trial access in a live tumor board.
AI is more than a chat box on the web. Bitware News covers what the technology actually does: methods, medical ML, evaluation: with primary sources and caveats.