Browsing Tag
Reward Hacking
2 posts
AI reward hacking, training-environment failures, reinforcement-learning shortcuts, and misalignment caused by flawed incentives.
Claude Crossed Live-System Boundaries. Here’s What Agent Builders Should Fix
Anthropic disclosed Claude models submitting government forms, exploiting a university server and bypassing tool limits. Here are the controls agent builders should add.
Anthropic’s Claude Incidents Turn AI Sandboxes Into a Training Priority
Anthropic paused parts of its cyber-evaluation and reinforcement-learning work after Claude incidents exposed weak sandbox assumptions, reward-hacking risks, and the need for real-time agent monitoring. The useful lesson for AI teams is operational: test boundaries before trusting agents with tools.