Feyospace ships 86% misuse, Occamy splits benchmarks, Gemma tops Claude

TL;DR

• Feyospace-v1 hits 63.24% on CyberGym at its parameter scale with 86%+ harmful compliance.
• Occamy-1.0 edges GPT-5.6 Sol by 0.4 points on team-co-designed Claw-Eval.
• Same Occamy model loses by 18 points on WildClawBench and 30 on Terminal-Bench 2.1.
• Gemma-3-27B scores 88.2% on ReactHuman safety versus Claude Opus 4.8’s 64.7%.
• All 7 frontier MLLMs miss 1 in 3 physical hazards on ReactHuman decisions.

Today’s research beat runs on three unrelated release stories. Vera Praxis’s Feyospace-v1 cracks CyberGym’s top ten at its parameter scale but ships without an alignment stage, clearing harmful-request compliance above 86%. Occamy-1.0, a 35B MoE with 3B active, edges GPT-5.6 Sol by 0.4 points on Claw-Eval — a benchmark its team co-designed — then loses by 18 to 30 points on the adjacent WildClawBench and Terminal-Bench 2.1 evals. And Gemma-3-27B outscores Claude Opus 4.8 by more than 23 points on the new ReactHuman physical-safety benchmark, in a field where every frontier MLLM tested still mishandles roughly one hazard in three.

The briefs tilt toward simpler-beats-complex results. DataFlex-RL argues uniform sampling matches adaptive RLVR curricula on Qwen2.5-7B and Llama-3.1-8B, and Ambient’s ECCV 2026 Wearable-AI sweep came from a 2B vision-language model distilled from a tool-using agent.

Feyospace-v1 open-sources cyber agent with 86% misuse rate

Source: hf-daily-papers · published 2026-09-07

TL;DR

• Vera Praxis post-trained Qwen checkpoints to 63.24% on CyberGym — 10th overall, 1st at its parameter scale.
• Choulea harvests hidden reasoning traces from GPT/Claude/Gemini via gray-market API accounts at ~$0.27 each for 273× cost leverage.
• With no alignment stage, harmful-request compliance climbed above 86% — independent write-ups call it a “hard regression.”
• Dream’s Hero hits 96.6% on the same benchmark via multi-agent orchestration on GLM-5.2.

The pitch: a seven-person team at the frontier

Feyospace-v1 is a data-centric post-training recipe from Vera Praxis Lab that turns open-weight Qwen checkpoints (35B-A3B, 27B, 122B-A10B) into agentic cyber operators. The headline number is 63.24% verified success on CyberGym for Feyospace-s1 — 10th place overall and first among comparable-parameter models — with a 23.76% average lift over base checkpoints across the suite. On NYU CTF Bench the s0 checkpoint jumps from 21.28% to 28.19%, and reasoning-token usage drops 13–18% while accuracy holds. The “Mercury Seven” framing — a small team going toe-to-toe with frontier labs on 164,269 audited trajectories — is real, and the artifact is scheduled for release.

The method frontier labs are now calling an attack

The five-component data engine is where the story turns. Choulea recovers proprietary reasoning traces by fingerprinting model-specific “cognitive dialects” in GPT/Claude/Gemini outputs. Hongzwang bypasses teacher-model refusals via task decomposition, wording mutation, and role framing to elicit CVE analysis and exploit-development trajectories. SkyReal claims 273× cost leverage by programmatically rotating “low-cost account channels from global gray markets.”

Independent security researchers have a less clinical name for that infrastructure: Hydra Clusters — sprawling networks of thousands of fraudulent accounts built on stolen credentials and fake payment methods to evade rate limits and geo-blocks ¹. Two days after the paper dropped, Anthropic published a 154-page report identifying and blocking large-scale distillation by seven Chinese labs, coining the term “illicit distillation attacks” and moving advanced cyber capabilities behind a gated Trusted Access program ². Vera Praxis is widely presumed to be one of the seven.

The dissent is worth surfacing. Lawfare’s policy read notes that if model outputs aren’t copyrightable, distillation looks more like functional imitation than theft, and cites industry figures calling cross-model validation “standard practice” despite ToS language ³. This is a live legal question, not a settled one.

Benchmark context deflates the headline

63.24% sounds like a frontier result until you read the neighbouring leaderboards. Dream’s “Hero,” built on GLM-5.2 open weights via multi-agent orchestration, reportedly hits 96.6% on the same CyberGym benchmark ⁴. Feyospace is neither the frontier nor the only open-weight team clearing it. Berkeley’s CyberGym maintainers add a further caveat: agent runs are stochastic enough that any gap under 15 points is “indistinguishable from measurement error” ⁵. The 23.76% lift over base checkpoints survives that filter; the leaderboard rank does not.

The safety number the paper mentions in passing

The paper acknowledges — in one paragraph — that skipping an alignment stage raised attack success rates on red-teaming benchmarks. Independent write-ups pin the actual figure: successful execution of harmful cyber requests climbed above 86%, which AI Weekly’s editors label a “hard regression” against the base models ⁶. Combined with Hongzwang being explicitly engineered to defeat teacher-model refusals, the artifact scheduled for public release is a competent, un-aligned offensive cyber agent trained on trajectories a frontier lab would refuse to produce.

That is the story. Not “plucky small team matches frontier,” but a working test case for whether gray-market distillation is a tolerated path to open-weight offensive capability — with contested legality, contested benchmark significance, and uncontested safety degradation.

---

Occamy-1.0 beats GPT-5.6 on Claw-Eval, trails on Terminal-Bench

Source: hf-daily-papers · published 2026-09-03

TL;DR

• Occamy-1.0 (35B MoE, 3B active) edges GPT-5.6 Sol on Claw-Eval, 82.20 vs 81.80.
• Same model collapses to 49.16 vs 67.20 on WildClawBench and 59.0 vs 88.80 on Terminal-Bench 2.1.
• Pareto-frontier plot uses Accio’s own frozen pricing protocol across four benchmarks the team co-designed.
• The HDPO training recipe has independent validation — it cut redundant tool calls from 98% to 2% on Alibaba’s Metis agent.

A win on home turf, losses next door

Accio Team’s Occamy-1.0 is pitched as a Pareto-frontier “co-work” agent: a 35B-parameter MoE with roughly 3B active parameters, post-trained from Qwen3.6-35B-A3B to run long-horizon digital workflows cheaply. The headline number is real — 82.20 on Claw-Eval edges GPT-5.6 Sol’s 81.80 ⁷. The problem is what happens one benchmark over. On WildClawBench, Occamy scores 49.16 against GPT-5.6 Sol’s 67.20. On Terminal-Bench 2.1, it manages 59.0 against 88.80 ⁷. That is not a small gap; it is a shell-competence chasm.

The pattern is consistent with a model tuned to a specific “execution replay” distribution its authors designed, rather than a genuinely general agent. Independent commentary frames Occamy as “specialized for execution-oriented replays rather than broad reasoning” ⁸, which is a polite way of saying the Pareto claim generalizes about as far as the training grader does.

The Pareto plot grades itself

The cost-performance frontier in the paper is macro-averaged across Claw-Eval, WildClawBench, AutomationBench and GDPval, priced under Accio’s own “frozen pricing protocol” ⁸. No third-party cost audit exists. Worse, the Claw-Eval maintainers themselves note that swapping the agent harness alone can shift a model’s score by up to 18 points ⁹ — meaning “cost per completed task” is entangled with scaffolding choices the vendor controls end-to-end.

Alibaba’s parallel press claim that Accio ran a 107-task e-commerce benchmark at 50% lower cost than Codex and Claude Code ¹⁰ is directionally consistent but carries the same methodological fingerprint: same lab, same graders, same harness.

The efficiency claims rely on the authors’ own frozen pricing protocol and self-selected benchmark mix… the Pareto-frontier result may not generalize to different API structures or hardware. ⁸

HDPO is the part that travels

The strongest external evidence for Occamy is not about Occamy at all. Hierarchical Decoupled Policy Optimization — the RL stage that trains the “Marathon expert” for long-horizon execution — was previously deployed on Alibaba’s Metis agent, where it cut redundant tool invocations from 98% to 2% while improving reasoning accuracy ¹¹. Decoupling the accuracy and efficiency reward channels appears to be a real advance in agentic RL, and it is separable from whatever is going on with the Occamy benchmark suite.

The specialization-then-merge recipe (Marathon + Sprint experts, weight-averaged, then consolidated with Single-Rollout Asynchronous Optimization) is the more interesting artifact here than the leaderboard row.

Runs on a 24GB card

Community quantizations landed within days: GGUF, FP8, NVFP4 and MLX builds hit 80–90 tokens/sec on prosumer dual-GPU rigs and run on Apple Silicon ¹². The 3B-active footprint is the actual hook — a frontier-adjacent agent that fits in 24GB VRAM is a genuine deployment story, regardless of how the vendor’s own Pareto plot shakes out. LocalLLaMA users report swapping it into automation pipelines in place of Qwen 3.8, while flagging that multi-file debugging still wants a larger dense model.

The takeaway: trust the recipe and the quants. Re-run the benchmarks yourself before trusting the frontier plot.

---

Gemma-3-27B beats Claude Opus on ReactHuman safety

Source: hf-daily-papers · published 2026-09-08

TL;DR

• Frontier MLLMs mishandle ~1 in 3 physical hazards on ReactHuman, averaging just 54.0% Semantic Action Accuracy.
• Gemma-3-27B (88.2%) outscores Claude Opus 4.8 (64.7%) on safety — reactive competence does not track model scale.
• Zero of 7 models flagged adversarial objects — foam anvils, steel apples — across 280 decisions.
• Freezing in place drives 60%+ of safety failures, with slow-moving hazards triggering inaction twice as often as fast ones.

The scaling paradox is the headline

ReactHuman puts seven MLLMs behind a simulated Unitree G1 humanoid and asks a narrower question than most embodied benchmarks: given 0.6 seconds of video of a knife falling or a ladder slipping, what do you do in the next moment? The answers are bad, and they’re bad in a specific way. Average Semantic Action Accuracy across the seven models is 54.0%. GPT-5.5 tops the leaderboard at 63.7% — trailed by Gemma-3-27B at 62.4% — and on the safety metric the open-weights model wins outright: 88.2% versus Claude Opus 4.8’s 64.7%.

Independent readers picked up on this immediately. cctest.ai’s writeup notes that larger, more sophisticated models often score worse on safety than smaller ones, and that “frozen” is the default failure mode — inaction accounts for over 60% of safety violations in some slices ¹³. That framing matters: it reframes the result from “MLLMs need more scale” to “text-safety RLHF does not transfer to physical priors.”

Adversarial objects break every model

The sharpest single finding is on adversarial physics. The benchmark plants objects that look one way and behave another — a foam anvil, a steel apple, a lightweight-looking pan that lands like lead. Across 280 decisions involving these objects, not one model expressed doubt about material properties. Everyone reacted from visual category. Hacker News and alphaXiv discussions have latched onto this as a “vibe check” that current MLLMs fail categorically ¹⁴, and it dovetails with the kinematic result: when models do choose “Catch,” their hands land a median 0.48m from the impact point, with 89% of misses short — arms extended, feet planted.

The speed-perception failure is similar in shape. Models treat velocity as binary. On slow-moving hazards they freeze 42% of the time versus 21% on fast ones — essentially failing to register that a slow slide is still a threat.

The physics grounding has live critiques

Two methodological caveats deserve prominent placement. First, the Genesis physics engine underneath ReactHuman is itself contested: Stone Tao’s re-benchmarks argue Genesis’s headline throughput came from substeps=1 configurations that trade accuracy for speed, and with standard settings Genesis can run 3–10× slower than ManiSkill ¹⁵. ReactHuman’s “exact labels” claim rests on 240 Hz simulator ground truth, so solver fidelity is load-bearing.

Second, the “freeze-and-predict” protocol is open-loop by construction. TheMoonlight.io’s review argues biological reactive systems don’t work this way — they rely on continuous predictive processing, and open-loop scoring cannot capture how a model’s own policy would induce novel states downstream ¹⁶. Meter-scale interception errors may partly reflect the protocol, not the model.

There’s also a shared-confound issue worth naming. Every MLLM in the comparison is bottlenecked by the same pre-trained RL walking controller for the Unitree G1 — the “spine” in the now-standard brain-spine-body decoupling ¹⁷. Differences between models are filtered through one controller’s competence, which could compress or exaggerate the ranking.

flowchart LR
    A[0.6s video<br/>3 viewpoints] --> B[MLLM 'Brain'<br/>JSON reaction plan]
    B --> C[RL 'Spine'<br/>Unitree G1 walker]
    C --> D[Simulated 'Body'<br/>Genesis @ 240Hz]
    D --> E{Outcome:<br/>catch / dodge / hit}
    style B fill:#fce4a6
    style C fill:#d4e8d4

What’s actually at stake

ReactHuman’s negative-capability findings — the scaling paradox, the adversarial-object blindness, the freezing-under-uncertainty pattern — are the kind of result that reshapes how embodied evals get designed. The dataset ships on Hugging Face with a videos variant, though early community notes flag setup friction for the high-fidelity physics stack ¹⁸. If the closed-loop version of this benchmark shows the same rank order, “scale plus text-RLHF gets you a household robot brain” is dead as a hypothesis.

Round-ups

DataFlex-RL finds uniform sampling beats adaptive RLVR data policies

Source: hf-daily-papers

Uniform sampling matches or beats adaptive rollout selection, reweighting, and domain mixing on RLVR training, according to DataFlex-RL benchmarks across math, logic, and science. Tests on Qwen2.5-7B-Base and Llama-3.1-8B-Base include GPQA-Diamond, questioning the payoff of complex data curation for GRPO-style training.

Ambient sweeps ECCV 2026 Wearable-AI Challenge on egocentric video

Source: hf-daily-papers, hf-daily-papers

First-place finishes in both EgoLongQA and EgoProactive tracks came from a 2B vision-language model distilled from a tool-using agent, with its multilingual embedding table pruned to hit parameter limits. The proactive-assistance entry recasts intervention timing as single-token classification supervised by a video agent.

SAS trains sparse-attention selectors end-to-end via softmax gating

Source: hf-daily-papers

Sparse attention gains a learned context ranker by inserting continuous softmax gates during training and swapping in Top-K selection at inference. Tencent Hunyuan’s SAS ships a Triton kernel compatible with FlashAttention, improving long-context accuracy under tight token budgets versus dense-attention distillation baselines.

StepAudio 3 Gen unifies TTS, sound effects, and music in one model

Source: hf-daily-papers

A single discrete autoregressive model handles text-to-speech, voice design, sound effects, and music by sharing a residual vector quantization code space across modalities. StepAudio 3 Gen pairs a causal Transformer with an RVQ Adaptor and interference-aware progressive pretraining to avoid task collisions.

Pelican-Sim 1.0 offers a general world-model simulator for robots

Source: hf-daily-papers

Predicting future frames from visual context and robot actions, Pelican-Sim 1.0 uses unified URDF-based action representations, action-visual injection, and a sparse mixture-of-experts backbone. Few-step distillation and causal adaptation speed rollouts, targeting controllable video generation across varied embodiments and manipulation tasks.

PLC-DPO cleans noisy preference labels using policy-reference margins

Source: hf-daily-papers

Noisy or ambiguous pairwise labels get routed into clean, flipped, or tied buckets by PLC-DPO using calibrated policy-reference margins during preference optimization. The posterior correction step targets a common failure mode of vanilla DPO, where mislabeled pairs push the policy in the wrong direction.

Python executor lifts clinical-calculator accuracy only for larger LLMs

Source: hf-daily-papers

Delegating arithmetic to a restricted Python executor improves clinical calculator scores on MedCalc-Bench Verified for larger open-weight models but not reliably for smaller ones. The program-solve interface does not substitute for verified formulas or accurate variable extraction, limiting deterministic gains in clinical LLM pipelines.

Footnotes

1. ogwilliam.com — ‘Hydra Clusters’ analysis — https://blog.ogwilliam.com/post/ai-distillation-attacks-hydra-clusters

   These services often rely on ‘Hydra Clusters’—sprawling networks of thousands of fraudulent accounts created using stolen credentials or fake credit card numbers to circumvent rate limits and geographic blocks.

   ↩

2. Dark Reading — US government on Chinese AI distillation — https://www.darkreading.com/application-security/us-government-chinese-ai-firms-distilling-frontier-models

   Anthropic released a 154-page report identifying and blocking large-scale distillation by seven Chinese labs… redefined these activities as ‘illicit distillation attacks,’ distinguishing them from legitimate model compression.

   ↩

3. Lawfare — Responding to AI distillation without panic — https://www.lawfaremedia.org/article/responding-to-ai-distillation-without-panic

   Dissenters note that if AI outputs are not copyrightable, distillation may simply be a form of functional imitation rather than theft… prominent figures have argued that using other AIs to validate one’s own is ‘standard practice.’

   ↩

4. Industrial Cyber — Dream’s Hero on CyberGym — https://industrialcyber.co/news/dreams-hero-scores-96-6-on-cybergym-to-rank-first-globally-in-autonomous-cybersecurity-research-benchmark/

   Dream’s Hero—built on GLM-5.2 open weights—has achieved scores as high as 96.6% by orchestrating multiple specialized agents, versus Feyospace-s1’s 63.24%.

   ↩

5. UC Berkeley RDI blog — CyberGym methodology — https://rdi.berkeley.edu/blog/cybergym/

   Experts note that while top-tier agents now consistently achieve high scores, modest point differences may not represent meaningful capability gaps due to the stochastic nature of agent runs; any performance gap smaller than 15 points is ‘indistinguishable from measurement error.’

   ↩

6. AI Weekly editors’ blog — Feyospace safety regression — https://aiweekly.co/editors-blog/found-first-seven-person-team-s-cyber-agent-hits-63-24-on-cybergym-ranks-1st-at

   Since they prioritized capability and helpfulness without a separate alignment stage, successful execution of harmful cyber requests climbed above 86%, a ‘hard regression’ in safety alignment.

   ↩

7. AI Weekly benchmark alert — https://aiweekly.co/alerts/occamy-10-open-35b-edges-gpt-56-sol-on-claw-eval-average

   Occamy scored 82.20 on Claw-Eval, edging GPT-5.6 Sol’s 81.80… but only 49.16 on WildClawBench versus GPT-5.6 Sol’s 67.20, and trailed significantly on Terminal-Bench 2.1 (59.0 vs 88.80).

   ↩ ↩²

8. cctest.ai analysis — https://cctest.ai/en/articles/occamy-1-0-targets-the-cost-performance-sweet-spot-for-35b-co-work-agents

   Because the model’s efficiency claims rely on the authors’ own frozen pricing protocol and self-selected benchmark mix (Claw-Eval, WildClawBench, AutomationBench, GDPval), the Pareto-frontier result may not generalize to different API structures or hardware.

   ↩ ↩² ↩³

9. claw-eval GitHub repo — https://github.com/claw-eval/claw-eval

   Switching the agent harness alone could shift a model’s score by up to 18 points, making it difficult to attribute performance gains solely to the underlying LLM.

   ↩

10. Unite.AI on Accio — https://www.unite.ai/alibaba-com-says-accio-ran-e-commerce-tasks-at-over-50-lower-cost/

    In a 107-task e-commerce benchmark, Accio completed tasks at an estimated 50% lower cost than OpenAI’s Codex and Anthropic’s Claude Code.

    ↩

11. VentureBeat (Alibaba Metis coverage) — https://venturebeat.com/orchestration/alibabas-metis-agent-cuts-redundant-ai-tool-calls-from-98-to-2-and-gets-more-accurate-doing-it

    HDPO reduced redundant tool invocations from 98% to just 2% while simultaneously improving overall reasoning accuracy.

    ↩

12. Hugging Face Accio-Lab/occamy-1.0-GGUF (community quants) — https://huggingface.co/Accio-Lab/occamy-1.0-GGUF

    Community-provided GGUF, FP8, NVFP4 and MLX builds enable execution on Apple Silicon and 24GB VRAM GPUs at 80–90 tokens/sec on consumer dual-GPU setups.

    ↩

13. cctest.ai review of ReactHuman — https://cctest.ai/en/articles/reacthuman-tests-whether-multimodal-models-can-react-safely

    Larger, more sophisticated models often perform worse on safety metrics than their smaller counterparts… ‘frozen’ default; when faced with complex physical events, they frequently fail to move at all, accounting for over 60% of safety failures in some tests.

    ↩

14. alphaXiv discussion of ReactHuman — https://www.alphaxiv.org/abs/2609.10895

    Discussions… emphasize that while models can articulate physics principles in text, they struggle to apply them in real-time ‘closed-loop’ environments… developers on Hacker News highlighted the ‘adversarial objects’ as a particularly effective ‘vibe check’ for embodied AI.

    ↩

15. Medium (Sai Manideep) — Genesis engine review citing Stone Tao critique — https://medium.com/@saimanideep.ch12345/genesis-training-robots-430-000x-faster-than-real-time-c6b6775eee63

    Corrected benchmarks using standard accuracy settings suggest that Genesis may actually be 3–10x slower than alternative GPU simulators like ManiSkill… headline-grabbing benchmark numbers were achieved by using ‘the fastest physics solver configurations,’ such as setting physics substeps to 1, which significantly degrades simulation accuracy.

    ↩

16. TheMoonlight.io review — methodological critique — https://www.themoonlight.io/fr/review/reacthuman-a-physics-grounded-benchmark-for-human-like-reactive-decision-making-in-embodied-multimodal-llms

    Biological brains do not ‘freeze’ their synapses or wait for a discrete t0; instead, they rely on continuous adaptation and predictive processing… open-loop predictions often diverge from real-world performance because they do not account for how a model’s own policy might induce novel, unobserved states.

    ↩

17. Embodied-AI-Daily GitHub (Whole-Body-Control survey) — https://github.com/luohongk/Embodied-AI-Daily/blob/main/papers/Whole-Body-Control.md

    A leading architectural trend is the decoupling of the ‘brain’ (MLLM planner) from the ‘spine’ (RL-based motor controller) and the ‘body’… a pre-trained RL controller handles the high-dimensional contact dynamics and balance required for the humanoid torso and limbs.

    ↩

18. Hugging Face dataset (Alan123/reacthuman-benchmark-scaled) — https://huggingface.co/papers/2609.10895

    Repositories such as Alan123/reacthuman-benchmark-scaled and Alan123/reacthuman-benchmark-scaled-with-videos contain the structured dataset and high-resolution execution videos… initial community discussions on Hugging Face highlight the technical complexity of setting up local environments for such high-fidelity physics simulations.

    ↩