I’m pretty new to optimization field, so forgive me if my question is too simple. I ran an optimization using Scipy (method SLSQP) and another one using Pyomo (IPOPT solver).
Pyomo runs in less than one minute and Scipy takes 4 hours. Both have the same inputs, constraints and objective function. However, I got different results and my final results are 3% lower in Pyomo.
There is no constraint violation, so I wonder if there is anything that happens under the hood to justify this difference?
I put the log of solvers below.
Many thanks!
Pyomo (IPOPT):
Number of nonzeros in equality constraint Jacobian...: 0
Number of nonzeros in inequality constraint Jacobian.: 2328
Number of nonzeros in Lagrangian Hessian.............: 5796
Total number of variables............................: 1656
variables with only lower bounds: 0
variables with lower and upper bounds: 1656
variables with only upper bounds: 0
Total number of equality constraints.................: 0
Total number of inequality constraints...............: 1164
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 0
inequality constraints with only upper bounds: 1164
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
0 1.2175516e-001 2.00e-003 1.10e+000 -1.0 0.00e+000 - 0.00e+000 0.00e+000 0
1 1.7159127e-001 4.64e-004 2.68e+001 -1.0 9.89e-002 - 1.00e+000 2.99e-002f 1
2 1.4757673e-001 1.02e-006 4.63e-001 -1.0 1.18e-002 - 1.00e+000 9.90e-001h 1
3 1.4578777e-001 0.00e+000 1.91e+001 -1.7 1.13e-003 - 7.16e-001 1.00e+000f 1
4 1.4522458e-001 0.00e+000 3.57e-004 -1.7 4.35e-004 - 1.00e+000 1.00e+000f 1
5 1.4516754e-001 0.00e+000 5.01e-008 -3.8 5.97e-006 - 1.00e+000 1.00e+000f 1
6 1.3779755e-001 0.00e+000 2.13e-001 -5.7 1.10e-003 - 5.33e-001 1.00e+000f 1
7 1.2833665e-001 0.00e+000 8.20e-002 -5.7 1.50e-003 - 6.14e-001 1.00e+000f 1
8 1.2186589e-001 0.00e+000 3.96e-002 -5.7 1.58e-003 - 5.14e-001 8.75e-001f 1
9 1.1779793e-001 0.00e+000 1.98e-002 -5.7 1.26e-003 - 5.80e-001 9.20e-001f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
10 1.1546554e-001 0.00e+000 8.06e-003 -5.7 1.29e-003 - 7.18e-001 1.00e+000f 1
11 1.1457517e-001 0.00e+000 6.17e-005 -5.7 7.62e-004 - 1.00e+000 1.00e+000f 1
12 1.1440512e-001 0.00e+000 5.72e-006 -5.7 2.98e-004 - 1.00e+000 1.00e+000f 1
13 1.1285677e-001 0.00e+000 3.15e-003 -8.6 1.37e-003 - 6.05e-001 7.84e-001f 1
14 1.1237444e-001 0.00e+000 1.24e-003 -8.6 1.35e-003 - 6.37e-001 6.43e-001f 1
15 1.1214949e-001 0.00e+000 1.70e-003 -8.6 1.54e-003 - 4.40e-001 6.26e-001f 1
16 1.1203489e-001 0.00e+000 1.40e-003 -8.6 2.82e-003 - 4.49e-001 7.18e-001f 1
17 1.1200180e-001 0.00e+000 4.97e-004 -8.6 1.21e-003 - 6.60e-001 5.89e-001f 1
18 1.1197130e-001 0.00e+000 2.65e-004 -8.6 1.62e-003 - 6.69e-001 9.46e-001f 1
19 1.1196671e-001 0.00e+000 1.70e-004 -8.6 8.83e-004 - 1.00e+000 8.58e-001f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
20 1.1196558e-001 0.00e+000 8.21e-007 -8.6 7.88e-004 - 1.00e+000 1.00e+000f 1
21 1.1196558e-001 0.00e+000 4.69e-009 -8.6 3.18e-005 - 1.00e+000 1.00e+000h 1
Number of Iterations....: 21
(scaled) (unscaled)
Objective...............: 1.1196557946295761e-001 1.1196557946295761e-001
Dual infeasibility......: 4.6902322746109060e-009 4.6902322746109060e-009
Constraint violation....: 0.0000000000000000e+000 0.0000000000000000e+000
Complementarity.........: 2.5450573136704817e-009 2.5450573136704817e-009
Overall NLP error.......: 4.6902322746109060e-009 4.6902322746109060e-009
Number of objective function evaluations = 22
Number of objective gradient evaluations = 22
Number of equality constraint evaluations = 0
Number of inequality constraint evaluations = 22
Number of equality constraint Jacobian evaluations = 0
Number of inequality constraint Jacobian evaluations = 22
Number of Lagrangian Hessian evaluations = 21
Total CPU secs in IPOPT (w/o function evaluations) = 3.105
Total CPU secs in NLP function evaluations = 64.320
EXIT: Optimal Solution Found.
Scipy (SLSQP):
Optimization terminated successfully. (Exit mode 0)
Current function value: -0.218957755761
Iterations: 16
Function evaluations: 8875
Gradient evaluations: 16
fun: -0.21895775576074572
jac: array([ -2.86530703e-05, 2.38219555e-02, -1.52000505e-02,
-8.21491890e-03, 3.59528698e-03, -7.48533383e-03,
-1.92692690e-03, -3.41123156e-03, -3.21419816e-02,
-9.97265056e-03, -6.48400187e-03, -5.71907870e-02,
-4.65486385e-03, 1.15180114e-01, -1.14405552e-01,
-9.20804739e-02, -5.98104000e-02, -1.90570783e-01,
-2.40653940e-03, 8.92927870e-03, -3.03730890e-02,
-3.75409909e-02, -1.84157491e-02, -6.97886087e-02,
-7.83495419e-03, 7.51326103e-02, -1.36542384e-01,
-1.64468037e-01, -5.80270607e-02, -2.43128262e-01,
9.03517008e-04, 1.44044254e-01, -4.12496198e-02,
1.62238544e-01, 5.17717227e-02, 6.62118383e-03,
-6.94024377e-03, -1.62795186e-06, -7.60566294e-02,
2.84731667e-02, 9.48129594e-03, 1.77269895e-02,
-1.79448333e-02, 4.12751781e-01, -5.09224996e-01,
-4.75407017e-01, -1.37610277e-01, -3.66811626e-01,
-1.09504517e-02, 8.36522859e-02, -8.82836487e-02,
-1.12652794e-01, -9.48307663e-03, -1.70594741e-01,
-4.89408337e-03, 1.44064132e-01, -2.83695426e-01,
-6.33431289e-01, -1.10606764e-01, -1.23438478e+00,
6.29534945e-04, 2.11109556e-02, -8.12242366e-03,
-5.52550517e-03, -1.05711445e-03, -4.10291739e-03,
-1.12794340e-04, 8.43631104e-04, -4.40889411e-03,
-3.51147540e-03, -3.96092795e-03, -1.60357766e-02,
-2.87283026e-03, 9.95702017e-02, -8.42474047e-02,
-5.85573092e-02, -1.97137874e-02, -1.17533877e-01,
-4.55826521e-04, 8.99187662e-03, -9.73374955e-03,
-1.53909251e-02, -1.01474449e-02, -3.47449742e-02,
-7.40325637e-03, 5.10204509e-02, -1.03980640e-01,
-1.28275935e-01, -3.68812345e-02, -1.87080827e-01,
3.97755578e-03, 1.27710635e-01, 1.24892965e-03,
6.84398040e-03, -4.52733599e-03, 3.05649638e-03,
-1.26620755e-03, 4.24076803e-03, -1.80831421e-02,
2.43064966e-02, 9.49857384e-03, -1.87398158e-02,
-1.77961476e-02, 2.05051536e-01, -3.54529817e-01,
-1.13319729e-01, -8.52574222e-02, -2.59923333e-01,
-4.84287739e-08, 5.70387375e-02, -1.34056807e-02,
-2.43987478e-02, -9.49930027e-03, -4.61928491e-02,
-6.80417009e-03, 1.12068275e-01, -1.86054656e-01,
-3.74100447e-01, -4.32863012e-02, -6.24756081e-01,
4.32096422e-05, 5.92724048e-03, -8.94419849e-04,
-1.60194561e-03, -1.33785978e-03, -7.82429613e-03,
1.15226954e-04, 4.10270691e-03, 9.56920907e-04,
-9.39304009e-04, -1.52550451e-03, -1.23954304e-02,
-2.23761424e-04, 4.16153856e-03, -5.92420809e-03,
-4.37961146e-03, -8.66637938e-03, -9.28674955e-02,
-1.03672035e-03, 8.45305808e-03, -9.75430571e-03,
-1.07939821e-02, -1.42787714e-02, -7.95205198e-02,
-5.15344553e-03, -1.00889970e-02, -1.49491001e-02,
-1.03760045e-02, -1.52482167e-02, -4.55282964e-02,
-3.72127816e-03, -6.14217855e-03, -1.30692031e-02,
-2.25217454e-03, -5.74426726e-03, -2.93964595e-02,
6.40904531e-04, 3.90792079e-02, 1.43495500e-02,
-2.58227624e-03, -3.28428857e-03, -2.17399020e-02,
7.04728067e-04, 1.24288686e-02, 4.11624648e-03,
-1.81579404e-03, -4.29862924e-03, -2.11058315e-02,
-7.32630491e-04, 1.12954117e-02, -1.88982859e-02,
-2.35779863e-02, -3.08214929e-02, -2.37093600e-01,
-4.50097211e-03, 1.66281778e-02, -6.18589669e-03,
-2.63096057e-02, -5.14719374e-02, -1.85552407e-01,
-1.11916158e-02, -2.29235720e-02, -5.41237053e-02,
-1.93158500e-02, -1.13643501e-02, -3.10595483e-02,
-1.07513871e-02, -1.90889183e-02, -3.54674589e-02,
-1.76651645e-02, -2.94905212e-02, -7.99291506e-02,
0.00000000e+00, 9.88531858e-03, 1.05758756e-03,
-2.41546892e-03, -1.73184648e-03, -8.43126141e-03,
2.30846927e-04, 5.94364479e-03, 1.13965571e-03,
-8.49841163e-04, -1.96019933e-03, -6.37469999e-03,
-9.47732478e-05, 4.79303859e-03, -2.08349712e-03,
-3.83082405e-03, -3.19225155e-03, -3.60421091e-02,
-5.54338098e-04, 5.27323410e-03, -4.00500000e-03,
-4.98357229e-03, -9.01233219e-03, -4.78239655e-02,
-2.61466764e-03, -1.01818740e-02, -9.53836553e-03,
3.85768712e-03, -4.17769700e-03, -8.56735930e-03,
-3.28000821e-03, -4.06493433e-03, -4.02838178e-03,
9.29726288e-04, -5.92118129e-03, -7.46217184e-03,
7.45382160e-04, 1.99196283e-02, 7.32803158e-03,
-2.56313197e-03, -1.77604519e-03, -8.92506912e-03,
2.32316554e-04, 1.44336931e-02, 5.32799214e-03,
-1.52365863e-03, -3.25956382e-03, -1.21336430e-02,
-2.78177857e-03, 8.04873183e-03, -3.14733274e-02,
-3.45978905e-02, -4.22597863e-02, -1.58204751e-01,
-3.20594013e-03, 1.20428633e-02, -1.32644773e-02,
-1.90007072e-02, -3.05818226e-02, -1.06163416e-01,
-4.95312177e-03, -2.27362346e-02, -2.74502411e-02,
-1.91133153e-02, -1.42665114e-02, -5.53950872e-02,
-1.28527358e-02, -1.83296073e-02, -1.71343014e-02,
-8.90292972e-03, -2.31084358e-02, -6.46356232e-02,
3.73758376e-05, 9.93125141e-04, 4.11760062e-04,
-2.15793028e-04, -2.44235620e-04, -1.41125172e-03,
9.87295061e-05, 1.04581565e-03, 2.64123082e-04,
-1.77111477e-04, -3.58037651e-04, -1.76383182e-03,
-2.64507905e-03, 1.42250024e-03, -4.89842519e-03,
-9.01906751e-03, -9.05502029e-03, -3.83878704e-02,
-1.05027296e-03, 5.29479980e-03, -4.14970145e-03,
-4.26785462e-03, -6.99245743e-03, -2.56510004e-02,
-2.48845667e-03, -2.41506658e-03, -1.20230783e-02,
-4.56554629e-03, -1.13946758e-02, -5.04478421e-02,
-4.73168865e-03, -6.09773025e-03, -1.12827662e-02,
1.85097754e-03, -5.41935302e-03, -1.82703547e-02,
6.51884824e-04, 1.60064809e-02, 8.85203481e-04,
-1.98119693e-03, -1.10076927e-03, -4.82245721e-03,
3.59894708e-04, 6.94737397e-03, 1.89301558e-03,
-6.03890046e-04, -1.72022544e-03, -5.99285960e-03,
-1.18161738e-03, 7.13099912e-03, -8.30562785e-03,
-1.86010320e-02, -2.09499765e-02, -1.07730741e-01,
-1.35865994e-03, 1.07049122e-02, -5.89556433e-03,
-6.04888797e-03, -1.54975597e-02, -4.50542532e-02,
-6.56010769e-03, -1.71564929e-02, -3.16504389e-02,
7.77484290e-03, -2.54669767e-02, -4.29303497e-02,
-1.27771329e-02, -1.22673512e-02, -1.34785436e-02,
-3.55788693e-03, -1.59052592e-02, -4.60611172e-02,
2.73883343e-05, 1.28661655e-03, 1.32496655e-03,
-7.52860680e-04, -3.18214297e-04, -2.64157727e-03,
1.04481354e-04, 1.06248818e-03, -6.68087974e-04,
-4.96750697e-04, -7.30423257e-04, -7.89061375e-03,
-6.99050725e-05, 2.25268491e-03, -2.59658322e-04,
-2.08515488e-03, -3.44311260e-03, -4.08612806e-02,
-3.41659412e-04, 8.25025514e-03, -4.19360027e-03,
-4.02395613e-03, -8.04402493e-03, -3.61145660e-02,
-4.10106033e-03, -3.54836322e-03, -1.21456627e-02,
-8.09197687e-03, -9.88831185e-03, -4.19944301e-02,
-3.35960463e-03, -5.55329956e-03, 3.87430191e-07,
-3.00006196e-03, -1.08503066e-02, -2.48223785e-02,
4.98574227e-04, 7.95373507e-03, 3.16526555e-03,
-1.25443563e-03, -1.09320320e-03, -6.55071996e-03,
4.44874167e-04, 7.76365399e-03, 9.42828134e-04,
-7.99467787e-04, -2.13225558e-03, -1.05341990e-02,
-6.21005893e-05, 6.95007853e-03, -3.90985049e-03,
-5.49996458e-03, -7.63768889e-03, -5.58281839e-02,
-2.26614997e-04, 1.76616255e-02, -4.19690460e-03,
-5.20926714e-03, -1.44895259e-02, -4.61813156e-02,
-4.19271179e-03, -9.09698568e-03, -1.06259286e-02,
-3.54743563e-03, -5.10646775e-03, -1.10679623e-02,
-5.74621186e-03, -7.78321736e-03, -1.95963234e-02,
2.40541995e-03, -1.28107201e-02, -3.22154127e-02,
0.00000000e+00, 1.15281790e-02, -5.89642674e-04,
-2.56875530e-03, -1.60184316e-03, -9.01488960e-03,
1.49490312e-04, 3.96637246e-03, 7.23605976e-04,
-3.39373946e-04, -7.29555264e-04, -3.41222622e-03,
4.91933897e-04, 4.79298644e-03, -1.36148743e-03,
-3.09252739e-03, -1.30598992e-03, -3.25014275e-02,
-1.48264691e-04, 5.81834279e-03, 4.93273139e-04,
-3.36393714e-03, -6.53593056e-03, -3.08387689e-02,
-2.83966586e-03, 1.41968429e-02, -1.80919841e-03,
-1.42674148e-03, -5.79566322e-03, -2.15087645e-02,
-1.31441467e-03, 6.16572797e-05, -8.16728547e-03,
6.94575347e-03, -6.67758286e-06, 6.49690628e-06,
6.40964136e-04, 1.71578471e-02, -2.34624371e-04,
-8.65986571e-04, -9.42235813e-04, -6.54706173e-03,
5.97715378e-04, 8.92345421e-03, 3.90317291e-05,
4.61790711e-04, -1.38256326e-03, -5.06369583e-03,
-1.08058378e-03, 1.90478899e-02, -1.26504526e-02,
-1.80059075e-02, -2.02428084e-02, -8.32739137e-02,
-4.57635149e-04, 1.48563143e-02, -4.61235270e-03,
-4.35184315e-03, -1.43206567e-02, -5.11662122e-02,
-2.01708637e-03, -8.17853212e-03, -1.13892797e-02,
-7.81113654e-03, -5.06492332e-03, -1.62509475e-02,
-4.45904024e-03, -3.49108502e-03, 1.26063824e-05,
-4.43559140e-04, -6.13087229e-03, -9.33923945e-03,
9.77199525e-05, -1.20302662e-04, 6.49858266e-05,
-6.16315752e-04, -1.09991059e-04, -8.05266201e-04,
1.33268535e-04, 1.58888288e-03, 4.01115045e-04,
-4.16098163e-04, -6.09697774e-04, -2.22290866e-03,
-6.64424151e-05, 2.38372199e-03, -3.04354168e-03,
-5.99475019e-03, -5.06760180e-03, -2.39575263e-02,
-3.29341739e-04, 1.10920742e-02, -3.33048403e-03,
-4.38063592e-03, -1.45150907e-03, -2.05246266e-02,
-3.02891247e-03, -2.38477811e-03, -1.07535366e-02,
-3.84285301e-03, -1.89998019e-02, -5.75217623e-02,
-6.42297603e-03, -7.19357841e-03, 1.13621354e-06,
1.26846135e-06, -4.82495315e-03, -2.00816058e-02,
5.89428470e-04, 1.43375155e-02, -6.91805035e-05,
-7.68952072e-04, -7.63844699e-04, -4.05739807e-03,
3.40092927e-04, 6.51947781e-03, 1.81198679e-03,
-7.32991844e-04, -1.34203769e-03, -3.88008356e-03,
-4.05080616e-04, 9.03119706e-03, -1.21198408e-03,
-9.45428014e-03, -1.04830898e-02, -6.03188500e-02,
1.47185102e-03, 2.29265150e-02, -6.64852560e-05,
2.63871066e-03, -2.76003219e-03, -1.08072124e-02,
-7.17901625e-03, -1.83483101e-02, -3.11868638e-03,
-1.54440161e-02, -2.84395684e-02, -4.30134293e-02,
-1.42674167e-02, -9.29352455e-03, -1.13600492e-02,
-5.69218211e-03, -1.53469685e-02, -2.21603531e-02,
0.00000000e+00])
message: 'Optimization terminated successfully.'
nfev: 8875
nit: 16
njev: 16
status: 0
success: True
