● Benchmarks
Measured against OR-Tools.
Same instances, same time budget, every solution re-scored by the problem’s own rules. Where another solver wins or ties, the table says so.
27 / 32
best or tied
of the head-to-head instances
2
solvers
optimaru, Google OR-Tools
30 s
per run
wall-clock budget for every solver
- optimaru
- OR-Tools
Vehicle routing — CVRPLIB Set X
Gap to the best-known solution, one median instance per size bucket (100–1000 customers). OR-Tools runs its Routing solver with guided local search. · lower is better
Planning problems — against OR-Tools CP-SAT
Twelve scheduling and assignment problems, 24 instances. Both solvers get the same model, and every solution is re-scored by the problem’s own rules. Values are the penalty on the score level that decides the row. · lower is better
| Problem | Instance | optimaru | OR-Tools CP-SAT |
|---|---|---|---|
| Bed allocation soft | demo 77 stays / 13 beds / 10 rooms | 6,954 | 6,640 best |
| Bed allocation soft | medium 183 stays / 30 beds / 24 rooms | 13,686 best | 21,696 |
| Bed allocation soft | large 551 stays / 61 beds / 48 rooms | 34,556 best | no solution |
| Conference scheduling soft | demo demo conference | 54,420 tie | 54,420 tie |
| Facility location soft | demo 60 cons / 30 fac | 1,343,284 tie | 1,343,284 tie |
| Facility location soft | medium 200 cons / 50 fac | 3,144,802 | 3,099,950 best |
| Facility location soft | large 600 cons / 80 fac | 6,498,876 | 6,381,914 best |
| Flight crew scheduling soft | demo 62 seats / 50 crew / 14 flights | 23,000 tie | 23,000 tie |
| Flight crew scheduling hard | medium 120 seats / 50 crew / 28 flights | 2 best | 11 |
| Flight crew scheduling hard | large 236 seats / 50 crew / 56 flights | 21 best | 41,471 |
| Food packaging soft | demo demo plant | 95,221,284 best | no solution |
| Maintenance scheduling soft | small 14 jobs / 3 crews / 40 dates | 71 | 61 best |
| Maintenance scheduling soft | large 48 jobs / 5 crews / 80 dates | 3,472 best | no solution |
| Meeting scheduling soft | demo demo (24 meetings) | 3,040 best | no solution |
| Order picking soft | small 22 picks / 2 trolleys | 8,244 tie | 8,244 tie |
| Order picking soft | medium 56 picks / 4 trolleys | 20,478 best | 20,488 |
| Project job scheduling soft | demo 24 jobs / 2 projects / 7 res | 7 tie | 7 tie |
| Project job scheduling soft | medium 72 jobs / 4 projects / 13 res | 20 tie | 20 tie |
| Project job scheduling soft | large 156 jobs / 6 projects / 19 res | 35 | 18 best |
| Sports league scheduling hard | demo demo league | 0 best | 13 |
| Task assigning soft | demo demo | 10,179,610,011,485 tie | 10,179,610,011,485 tie |
| Tournament scheduling soft | demo 72 slots / 7 teams / 18 days | 1.55 tie | 1.55 tie |
| Tournament scheduling soft | large 180 slots / 10 teams / 36 days | 0 best | 2 |
| Tournament scheduling hard | xlarge 800 slots / 20 teams / 100 days | 0 best | 956 |
School timetabling
Full school timetabling model, total weighted penalty with every hard rule met. · lower is better
Public instances
Classic sets from the literature, against OR-Tools CP-SAT.
| Problem | Measure | optimaru | OR-Tools CP-SAT |
|---|---|---|---|
| Pickup & delivery (synthetic Li & Lim, 200 requests) | vehicles · distance | 43 · 11,300 | no solution |
| Pickup & delivery (Li & Lim lc101, 10 requests) | distance | 156.0 tie | 156.0 |
| Project scheduling (Patterson Pat1) | makespan | 65 | 62 |
| Bin packing (t60_00, u120_04) | bins | 21 · 50 tie | 21 · 50 |
| Assembly line balancing (3 × 100 tasks) | stations | 24 · 22 · 26 | 23 · 22 · 25 |
How we measure
- Every solver gets the same instance and the same wall-clock budget. optimaru runs on one thread; OR-Tools CP-SAT gets 8 workers and its time includes building the model.
- Solutions are never taken at their word: each one is re-scored by an independent evaluator, so every solver is judged on the same objective.
- OR-Tools 9.15.
- A single run per instance and seed on one machine — a snapshot, not a statistical study. The harnesses are open source; re-run them on your own hardware.
Measured 2026-11-04 on one Apple M4 Max Mac (16 cores), 30 s per solver per instance. Benchmark code on GitHub →