The FrontierOR Challenge represents a fundamental shift in how we evaluate the efficacy of agentic workflows. Rather than assessing a model’s ability to provide a static, correct answer, the competition requires teams to design, test, and iteratively improve algorithms capable of solving complex optimization problems. For builders, the challenge is to engineer a system that can navigate unseen instances under predefined operating rules.
The competition structure, detailed in the challenge white paper, is designed to test scalability and generalization. Stage 1 consists of 60 problems drawn from a 180-problem audited benchmark, covering six application categories and five formulation types, including combinatorial, mixed-integer, quadratic, stochastic, and robust or bilevel models. The stage unfolds over four weekly rounds of 15 problems, commencing Jan. 4, 2027. Prospective participants must finalize registration and team formation via the Getting Started page by Dec. 27, 2026, at 23:59 EST. Public instances are provided for development and local testing, but official scoring relies exclusively on private, hidden instances. Stage 2, scheduled for Feb. 8–14, 2027, introduces problems outside the published benchmark, testing systems on entirely unseen instances.
Builders must navigate two distinct tracks, each imposing different engineering constraints. The Main track requires algorithm code for a network-disabled standardized sandbox, limited to 2 vCPU, 4 GB of memory, and a 4 MB archive; only open-source dependencies are allowed, and each problem has a strict time limit. The Self-hosted track permits any software, including commercial solvers, and accepts solutions computed on participants’ own infrastructure within a timed window. Because the tracks utilize separate private instance sets and operate under different resource environments, they are ranked separately. Consequently, scores are not direct comparisons across the two environments.
The challenge’s scoring makes the distinction between a runnable submission and a successful algorithm concrete. Every returned solution is first checked for feasibility. If the solution is feasible, its objective value is compared against a frozen reference on hidden instances. Teams are ranked by average problem scores, with normalized solving time used as a tie-breaker; infeasible or missing instances receive a score of 0. Each problem carries equal weight in the stage average, so raw instance totals alone do not define the ranking. Main entrants submit code for the organizers’ sandbox, while Self-hosted entrants upload solutions produced on their own machines. A team choosing the Main track must ensure their code functions within the published resource and dependency limits. Conversely, Self-hosted teams can utilize broader software but must solve and upload each instance within its specific window. Both tracks evaluate performance on unseen instances, but the artifacts differ: algorithm code in the Main track and computed solutions in the Self-hosted track.
The Challenge white paper cites a 39% result, noting that the strongest model met the stated condition on 39% of “large instances”: returning a feasible solution with an objective gap no greater than 1% against Gurobi and running no slower than the reference. The original benchmark paper describes the scope as “Full-set task instances.” These labels differ and should not be treated as identical. This statistic belongs to the prior one-shot benchmark, not the current Challenge. The earlier benchmark also allowed generated algorithms to call Gurobi, unlike the Challenge Main track’s open-source-only rule.
This 39% figure is benchmark-specific and should not be treated as a general agent success rate. It serves as historical context rather than a forecast for Main-track participants. It cannot predict how new workflows will perform across the Challenge’s rounds or on the new Stage 2 problems. No Challenge outcomes are available yet, and the schedule and rulebook alone cannot show whether teams will meet its feasibility and quality requirements.
For builders, the immediate decision is which track matches the system they can actually submit. Preparing against public problems can exercise a system, but only private instances determine scored performance. The challenge’s eventual results will provide evidence about these particular workflows and rules; until then, past benchmark performance cannot substitute for competition results.