Date | Solvers | Models | Comment |
---|---|---|---|

18.05.2012 | CBC 2.7, CPLEX 12.4.0.0, GUROBI 4.6.1, SCIP 2.1.1 | MIPLIB 2010 | |

28.02.2011 | CBC 2.4, CPLEX 12.2.0.2, GUROBI 4.0.1, SCIP 2.0.1 | LINLib | |

09.02.2010 | CBC 2.4, CPLEX 12.1, GUROBI 2.0, SCIP 1.2 | LINLib | |

18.01.2009 | CPLEX 11.2, CBC 2.2, SCIP 1.1 | LINLib | |

04.10.2008 | CPLEX 11.1, CBC 2.2, SCIP 1.1, SCIP 1.0, GLPK 4.30 | LINLib | |

14.09.2008 | CPLEX 11.1, CBC 2.2, SCIP 1.0, GLPK 4.30 | LINLib | |

25.08.2008 | CBC 1.1, 1.2, 2.0, 2.1, 2.2 | LINLib | |

19.07.2008 | CBC 2.1 | LINLib | one vs. two threads |

Date | Solvers | Models | Comment |
---|---|---|---|

04.03.2011 | CONOPT 3.14V, IPOPT 3.8, KNITRO 7.0.0, MINOS 5.51, PATH-NLP 4.7.02, SNOPT 7.2-4 | GlobalLib | |

20.11.2009 | IPOPT 3.7, CONOPT 3.14T, MINOS 5.51, SNOPT 7.2-4 | GlobalLib | |

20.11.2009 | IPOPT 3.7 | GlobalLib | linear solver MA27 vs. MUMPS 4.8.3; GAMS version vs. locally compiled version |

19.10.2008 | IPOPT 3.5, KNITRO 5.1.2, CONOPT 3.14S, MINOS 5.51, SNOPT 7.2-4 | GlobalLib | |

21.09.2008 | IPOPT 3.5 | GlobalLib | linear solver MA27 vs. MUMPS 4.8.3 |

Date | Solvers | Models | Comment |
---|---|---|---|

21.01.2009 | CPLEX 11.2, CLP 1.8, GLPK 4.35 | LINLib | |

07.03.2011 | CPLEX 12.2.0.2, CLP 1.11, GLPK 4.43, GUROBI 4.0.1, MOSEK 6.0.0.96 | LINLib |

Date | Solvers | Models | Comment |
---|---|---|---|

12.11.2012 | Baron 11.5.2, Couenne 0.4, Lindo API 7.0.1.497, SCIP 3.0.0 | MINLPLib | |

13.09.2012 | Baron 11.3.0, Couenne 0.4, Lindo API 7.0.1.497, SCIP 3.0.0 | MINLPLib | |

14.09.2012 | AlphaECP 2.09.02, Bonmin 1.6, DICOPT, Knitro 8.0.0, SBB, SCIP 3.0.0 | convex MINLPs from IBM-CMU MINLP, MINLPLib, Günlück and Linderoth | |

25.02.2011 | AlphaECP 1.75.04, Bonmin 1.4, DICOPT 2x-C, Knitro 7.0.0, SBB | convex MINLPs from Bonami, Kilinç, Linderoth | |

25.02.2011 | Bonmin 1.4 | convex MINLPs from Bonami, Kilinç, Linderoth | B-BB vs. B-ECP vs. B-Hyb vs. B-OA vs. B-QG |

29.09.2008 | Baron 8.1, LindoGlobal 5.0 | MINLPLib | |

29.09.2008 | Bonmin 0.99 | MINLPLib | B-BB vs. B-Hyb vs. B-OA, only convex models |

29.09.2008 | Baron 8.1, LindoGlobal 5.0, AlphaECP 1.63, Bonmin 0.99, DICOPT 2x, SBB, OQNLP | MINLPLib | only convex models |

29.09.2008 | Baron 8.1, LindoGlobal 5.0, AlphaECP 1.63, Bonmin 0.99, DICOPT 2x, SBB, OQNLP | MINLPLib |

This page contains details about benchmarks of COIN-OR solvers with GAMS interface vs. other GAMS solvers.

Benchmarks were made by creating trace-files using the GAMS trace option and processing them with the PAVER server.

**NEW:**

- 25.08.2008: benchmarking CBC from various GAMS versions and current 2.2
- 19.07.2008: benchmarking CBC single threaded vs. multithreaded version (running 2 threads).
- 06.07.2008: updated benchmarks of MIP solvers and of GAMS BCH using a newer CBC version.
- 01.06.2008: benchmarking MIP solvers CPLEX, CBC, GLPK, SCIP on LINLib.
- 19.05.2008: benchmarking GAMS BCH-facility on the FCNetLib (CPLEX, SCIP, CBC).
- 28.01.2008: LP solvers 4 years ago and today (CLP, GLPK, IPOPT(today)).
- 07.01.2008: Comparision with updated MIP solvers (CPLEX 11, GLPK 4.25).
- 07.01.2008: Comparision with updated LP solvers (CPLEX 11, GLPK 4.24).
- 27.09.2007: Comparision of DICOPT stop rules 0 to 3
- 28.09.2007: Comparision of SBB with CONOPT and default and high memnodes
- 28.09.2007: Updated comparision of SBB, AlphaECP, DICOPT, BONMIN.
- 01.10.2007: Updated comparision of CPLEX, CLP, GLPK, IPOPT on LPs.

Content:

- Solvers are always run on each model with a timelimit of 1 hour and without iteration or domain violation limit.

- The PerformanceTools scripts behind the PAVER server currently contain a bug that make PAVER omit the "locally optimal" row/column in the solver square summary for discrete models (MIP, MINLP).
This also leads to wrong sum-ups in the "total"-row/column.

Resource time summary and performance profiles should be correct.

- LINLib results were produced on a machine with Intel Core2 2.40 GHz, 2 GB RAM, and running under Linux 2.6.18.

- MINLPLib results were produced on a machine with Intel Pentium IV 3 GHz, 1 GB RAM, and running under Linux 2.6.11.

The following trunk revisions were used: CBC rev 770, CLP rev 1092, CGL rev 520, CoinUtils rev 837, OSI rev 1073

GAMS/CBC uses the OSI/CLP interface and CbcSolver routines which are also used by the CBC-standalone version. Additionally the following options are changed:

- Dual and primal optimality tolerances are set to 1e-7.

Additionally the following options are set or changed:

- Dual and primal optimality tolerances are set to 1e-7.
- Scaling is set to "geometric mean scaling, then equilibrium scaling".
- The primal simplex algorithm is used.
- LP presolve is turned on.
- Textbook pricing is used.
- Backtracking by best projection heuristic is used.
- The integer tolerance is 1e-5.
- All cutgenerators are turned on.

The GAMS/IPOPT interface changes the defaults of the following IPOPT options:

- mu_strategy is set to adaptive
- bound_relax_factor is set to 0.0

02.09.2007:

- results of the following comparision:
- CPLEX-0: CPLEX with default parameters (simplex algorithm).
- CPLEX-1: CPLEX with "lpmethod 4" (barrier algorithm).
- CBC-0: CLP with default parameters.
- GLPK-0: GLPK with default parameters (see above).
- IPOPT-0: IPOPT with default parameters (see above).
- IPOPT-1: IPOPT with mehrotra_algorithm, jac_c_constant, jac_d_constant, hessian_constant turned on.

- results of the following comparision (as above, but less solvers):
- CPLEX-0: CPLEX with default parameters (simplex algorithm).
- CBC-0: CLP with default parameters.
- GLPK-0: GLPK with default parameters (see above).
- IPOPT-1: IPOPT with mehrotra_algorithm, jac_c_constant, jac_d_constant, hessian_constant turned on.

- results of the following comparision:
- IPOPT-0: IPOPT with default parameters (see above).
- IPOPT-1: IPOPT with mehrotra_algorithm, jac_c_constant, jac_d_constant, hessian_constant turned on.

- results of the following comparision:
- CPLEX-0: CPLEX with default parameters (simplex algorithm).
- CBC-0: CLP with default parameters.
- GLPK-0: GLPK 4.22 with default parameters (see above), but steepest depth pivot choice algorithm.
- GLPK-4.20: GLPK 4.20 with default parameters (see above).
- IPOPT-1: IPOPT with mehrotra_algorithm, jac_c_constant, jac_d_constant, hessian_constant turned on.

- results of the following comparision:
- 1-CPLEX-0: CPLEX 11 with default parameters (simplex algorithm).
- 2-CPLEX-1: CPLEX 11 with "lpmethod 4" (barrier algorithm).
- 3-CPLEX-0: CPLEX 10.2 with default parameters (simplex algorithm).
- CBC-0: CLP with default parameters.
- GLPK-0: GLPK 4.22 with default parameters (see above), but steepest depth pivot choice algorithm.
- GLPK-4.20: GLPK 4.20 with default parameters (see above).
- IPOPT-0: IPOPT with default parameters.
- IPOPT-1: IPOPT with mehrotra_algorithm, jac_c_constant, jac_d_constant, hessian_constant turned on.

- results of the following comparision:
- CBC2004: CLP from GAMS 21.4 (2004) with default parameters
- CBC2008: CLP 2.0 (2008) with default parameters
- GLPK2004: GLPK 4.5 from GAMS 21.4 (2004) with default parameters
- GLPK2008: GLPK 4.25 (2008) with default parameters
- IPOPT2008: IPOPT (2008) with mehrotra_algorithm, jac_c_constant, jac_d_constant, hessian_constant turned on.

- results of the following comparision on
**110**mip models from the LINLib running- 22.4.trc: CBC 1.1 from GAMS 22.4
- 22.5.trc: CBC 1.2pre from GAMS 22.5
- 22.7.trc: CBC 2.0 from GAMS 22.7
- 22.8.trc: CBC 2.1 from GAMS 22.8
- 22.9.trc: CBC 2.2 from 21.08.2008 (CBC stable/2.2 rev. 1050; CLP stable/1.8 rev. 1262; Cgl stable/0.53 rev. 676; Osi stable/0.99 rev. 1288; CoinUtils stable/2.3 rev. 1070; compiled with flags "-DDISALLOW_PRINTING -DCLP_FAST_CODE -DCOIN_FAST_CODE -DCOIN_USE_RESTRICT -DNEW_STYLE_SOLVER=0")

**1800s**

gap tolerance = 0%

default solver parameters

machine: Core2 Duo T7500, 2 GB RAM, Linux 2.6.22, GCC 4.2.1

- results of the following comparision on the 125 mip models from the LINLib running
- CBC-1: Cbc 2.1 (stable version from approx. 24.06.08) compiled in singlethreading mode
- CBC-2: Cbc 2.1 (stable version rev. 1004 from approx. 15.07.08) compiled in multithreading mode and run with 2 threads

default solver parameters

machine: Core2 Duo T7500, 2 GB RAM, Linux 2.6.22, GCC 4.2.1

- results of the following comparision on the 125 mip models from the LINLib:
- CPLEX 11.0 (as in GAMS 22.6)
- SCIP 1.0
- CBC 2.1 (stable version from approx. 24.06.08)
- GLPK 4.26

default solver parameters

machine: Core2 Duo T7500, 2 GB RAM, Linux 2.4.16, GCC 4.2.1

- results of benchmarking MIP solvers on the GAMS FCNetLib (Fixed Cost Network Flow Library)
analysing the effect of dicuts computed via the GAMS BCH facility.
The following solvers were used:
- CPLEX 11.0 (as in GAMS 22.6)
- SCIP 1.0
- CBC 2.1 (stable/2.1 from approx. 24.06.08)

- results of the following comparision:
- CPLEX 11.0 (as in GAMS 22.6)
- SCIP 1.0
- CBC 2.1 (stable version from approx. 25.05.08)
- GLPK 4.26

default solver parameters

machine: Core2 Duo T7500, 2 GB RAM, Linux 2.4.16, GCC 4.2.1

- results of benchmarking MIP solvers on the GAMS FCNetLib (Fixed Cost Network Flow Library)
analysing the effect of dicuts computed via the GAMS BCH facility.
The following solvers were used:
- CPLEX 11.0 (as in GAMS 22.6)
- SCIP 1.0
- CBC 2.1 (stable/2.1 revision 946)

- results of the following comparision
(we run with a gap tolerance of 0.01%):
- 1-CPLEX-1: CPLEX 11 with default parameters.
- 2-CPLEX-1: CPLEX 10.2 with default parameters.
- CBC-1: CBC 2.0 with default parameters.
- GLPK-1: GLPK 4.25 with default parameters (see above).

02.09.2007:

- results of the following comparision
(we run with a gap tolerance of 0.01%, except for GLPK which does not have a MIP gap tolerance yet):
- CPLEX-1: CPLEX with default parameters.
- CBC-1: CBC with default parameters.
- GLPK-1: GLPK with default parameters (see above).

02.09.2007:

- results of the following comparision:
- CONOPT-1: CONOPT with parameters rtredg=1e-6, rtnwmi=1e-6, and rtnwma=1.01e-6.
- IPOPT-1: IPOPT with parameters tol=1e-6 and acceptable_tol=1e-6.
- KNITRO-1: KNITRO with parameters feastol=1e-6 and opttol=1e-6.

The gap tolerance is set to 1%, the feasibility tolerance to 1e-4.

02.09.2007:

- results of the following comparision:
- BARON-1: BARON.
- LaGO-1: LaGO with violation subdivision.

We run with a gap tolerance of 1%.

02.09.2007:

Solver A vs. Solver B:

- results of the following comparision on all 161 models without semicontinuous variables and SOS constraints (since these are not supported by GAMS/Bonmin yet):
- SBB-1: SBB with CONOPT as subsolver.
- AlphaECP-1: AlphaECP with CPLEX as subsolver.
- DICOPT-1: DICOPT with CONOPT and CPLEX as subsolver.
- BONMIN-1: BONMIN.

- results of the following comparision on all 40
**convex**models:- SBB-1: SBB with CONOPT as subsolver.
- AlphaECP-1: AlphaECP with CPLEX as subsolver.
- DICOPT-1: DICOPT with CONOPT and CPLEX as subsolver.
- BONMIN-1: BONMIN.

- results of the following comparision on all 149 models which BARON can handle (not SOS, no semicontinuous variables, no functions like sin, cos, ...):
- BARON
- LaGO-1: LaGO with violation subdivision and "unfixed discrete" as node selection type.
- BONMIN-1: BONMIN.

- results of the following comparision on all 40
**convex**models (no SOS, semicontinuous variables, or functions like sin, cos, ...):- BARON
- LaGO-1: LaGO with violation subdivision and "unfixed discrete" as node selection type.
- BONMIN-1: BONMIN.

- results of the following comparision on all 149 models which BARON can handle (not SOS, no semicontinuous variables, no functions like sin, cos, ...):
- BARON
- LaGO-1: LaGO with violation subdivision and "unfixed discrete" as node selection type.

- results of the following comparision on all 40
**convex**models (no SOS, semicontinuous variables, or functions like sin, cos, ...):- BARON
- AlphaECP-1: AlphaECP with CPLEX as subsolver.
- BONMIN-1: BONMIN.
- DICOPT-1: DICOPT with CONOPT and CPLEX as subsolver.
- LaGO-1: LaGO with violation subdivision and "unfixed discrete" as node selection type.
- SBB-1: SBB with CONOPT as subsolver.

- results of the following comparision on all 167 models:
- DICOPT-4: DICOPT with CONOPT and CPLEX as subsolvers and options maxcycles 10000 and stop 0
- DICOPT-5: DICOPT with CONOPT and CPLEX as subsolvers and options maxcycles 10000 and stop 1
- DICOPT-6: DICOPT with CONOPT and CPLEX as subsolvers and options maxcycles 10000 and stop 2
- DICOPT-7: DICOPT with CONOPT and CPLEX as subsolvers and options maxcycles 10000 and stop 3

- results of the following comparision on all 167 models:
- SBB-1: SBB with CONOPT
- SBB-7: SBB with CONOPT and memnodes 100000

- results of the following comparision on all 161 models without semicontinuous variables and SOS constraints (since these are not supported by GAMS/Bonmin yet):
- SBB-7: SBB with CONOPT as subsolver and memnodes 10000.
- AlphaECP-1: AlphaECP with CPLEX as subsolver.
- DICOPT-5: DICOPT with CONOPT and CPLEX as subsolver and options maxcycles 10000 and stop 1 (stop on crossover)
- BONMIN-1: BONMIN.

- results of the following comparision on all 40
**convex**models:- SBB-7: SBB with CONOPT as subsolver and memnodes 10000.
- AlphaECP-1: AlphaECP with CPLEX as subsolver.
- DICOPT-5: DICOPT with CONOPT and CPLEX as subsolver and options maxcycles 10000 and stop 1 (stop on crossover)
- BONMIN-1: BONMIN.

02.09.2007:

Influence of NLP subsolver on SBB performance:

- results of the following comparision:
- SBB-1: SBB with CONOPT as subsolver. CONOPT option Rtmaxv is set to 1e+20.
- SBB-2: SBB with IPOPT as subsolver and IPOPT option expect_infeasible_problem set to yes.
- SBB-3: SBB with IPOPT as subsolver and IPOPT options expect_infeasible_problem and warm_start_init_point set to yes.
- SBB-4: SBB with CONOPT and IPOPT in a infeasseq as subsolver: If CONOPT reports a subproblem as infeasible and we are at atmost depth 50 of the Branch-and-Bound tree, SBB calls IPOPT. IPOPT option expect_infeasible_problem is set to yes.
- SBB-5: SBB with IPOPT as subsolver. IPOPT option expect_infeasible_problem set to yes and print_level set to 0.

- results of the following comparision on all 40
**convex**models:- SBB-1: SBB with CONOPT as subsolver. CONOPT option Rtmaxv is set to 1e+20.
- SBB-2: SBB with IPOPT as subsolver and IPOPT option expect_infeasible_problem set to yes.
- SBB-3: SBB with IPOPT as subsolver and IPOPT options expect_infeasible_problem and warm_start_init_point set to yes.
- SBB-4: SBB with CONOPT and IPOPT in a infeasseq as subsolver: If CONOPT reports a subproblem as infeasible and we are at atmost depth 50 of the Branch-and-Bound tree, SBB calls IPOPT. IPOPT option expect_infeasible_problem is set to yes.

- results of the following comparision:
- SBB-1: SBB with CONOPT as subsolver. CONOPT option Rtmaxv is set to 1e+20.
- SBB-5: SBB with IPOPT as subsolver. IPOPT option expect_infeasible_problem set to yes and print_level set to 0.
- SBB-4: SBB with CONOPT and IPOPT in a infeasseq as subsolver: If CONOPT reports a subproblem as infeasible and we are at atmost depth 50 of the Branch-and-Bound tree, SBB calls IPOPT. IPOPT option expect_infeasible_problem is set to yes.

- results of the following comparision on all 40
**convex**models::- SBB-1: SBB with CONOPT as subsolver. CONOPT option Rtmaxv is set to 1e+20.
- SBB-5: SBB with IPOPT as subsolver. IPOPT option expect_infeasible_problem set to yes and print_level set to 0.

Influence of NLP and MIP subsolver on DICOPT performance:

- results of the following comparision:
- DICOPT-1: DICOPT with CPLEX and CONOPT as subsolver.
- DICOPT-2: DICOPT with CPLEX and IPOPT as subsolver.

- results of the following comparision:
- DICOPT-1: DICOPT with CPLEX and CONOPT as subsolver.
- DICOPT-3: DICOPT with CBC and CONOPT as subsolver.

Influence of MIP subsolver on AlphaECP performance:

- results of the following comparision:
- AlphaECP-1: AlphaECP with CPLEX as subsolver.
- AlphaECP-2: AlphaECP with CBC as subsolver.