Senior Software Engineer – Design Long-Horizon Coding Challenges for AI Agents

About the role

We're looking for seasoned software engineers to help shape the next generation of AI coding agents by designing realistic, long-horizon engineering tasks. This isn't about writing toy problems — it's about crafting investigation-heavy challenges pulled from the kind of complexity found in real, production-grade codebases. Your work will directly inform how advanced coding agents are evaluated and trained, so the quality, realism, and rigor of your task design matters enormously.

You'll bring your own experience navigating large, multi-file systems — understanding how a single change can ripple across modules, services, and tests — and translate that understanding into problems that a strong human engineer would recognize as genuine work.

What you'll do

• Author complex, multi-step coding tasks rooted in realistic engineering scenarios (bug investigations, feature additions, refactors, performance fixes, etc.)
• Build reproducible, containerized environments (Docker or similar) that reliably reproduce the task context
• Design rigorous verifiers/test suites that grade solutions the way a careful senior reviewer would
• Draw on your own experience with large, complex, private codebases to create authentic, non-trivial problem sets
• Document tasks clearly enough that another engineer (or an AI agent) can understand scope, constraints, and success criteria without ambiguity
• Iterate on task difficulty and clarity based on feedback from evaluation runs
• Optionally, use coding agents yourself to validate that tasks are solvable and appropriately challenging

Requirements

• Proficiency in at least one modern programming language (JavaScript/TypeScript, Java, Go, Rust, C/C++, Python, etc.)
• Demonstrated experience architecting or maintaining substantial, complex, large-scale software projects
• Hands-on experience with Docker or similar containerization tools
• Strong technical writing, documentation, and testing skills
• Access to a large, complex, private codebase you own or have deep authority over, which you can author tasks against
• Prior experience working with or evaluating coding agents on long-horizon tasks is a strong plus
• Comfortable working independently and asynchronously in a remote, global team

Compensation

$70–$100 per hour, depending on experience and task complexity. Remote, flexible engagement — global applicants welcome.