Problem with callback functions in fmincon (sometimes not called)

Tools Atoms packages
1

I am using fmincon 1.1 in Scilab 2026.1. I need a callback function in fmincon and I had some problems. I have finally discovered that the callback function was sometimes not called.

I have just put a print into the callback function:

function stop = callback(x, optimValues, state)
	mprintf("Callback %d \n", optimValues.iteration) ;
	stop = %f ; 
endfunction

My optimisation parameters are defined by:

opt = optimoptions("fmincon") ;
opt.SpecifyObjectiveGradient = %t ;
opt.SpecifyConstraintGradient = %t ;
opt.MaxFunctionEvaluations = 1000000 ;  
opt.MaxIterations = 1000000 ;  
opt.Display = "iter" ;
opt.OutputFcn = callback ;
problem =  struct() ;
problem.options = opt ;
problem.objective = objfungrad ;
problem.nonlcon = confungrad ;

This is an extract of an output:

Callback 0 
 Iter  Func-count  Function value   Feasibility   Step Length     Step Norm   First-order
																			   optimality
	0           4    5.858596e-03    1.9413e+01    0.0000e+00    0.0000e+00    6.4954e+01
Callback 0 
	1           6    5.778105e-03    1.9398e+01    7.5406e-04    4.6870e+01    6.4872e+01
Callback 1 
	2           8    5.632390e-03    1.9382e+01    8.5691e-04    6.7331e+01    6.4830e+01
Callback 2 
	3          10    3.389552e-03    1.9180e+01    1.0351e-02    8.9222e+01    6.4792e+01
Callback 3 
...
...
   32          70   -2.605954e-02    1.4594e+01    3.3048e-04    7.1779e+01    4.6712e+03
Callback 32 
   33          72   -2.605749e-02    1.4593e+01    7.0292e-05    2.6607e+02    4.9388e+03
Callback 33 
   34          78   -2.605749e-02    1.4593e+01    3.2579e-07    0.0000e+00    2.8633e+03
   35          79   -5.460658e-02    1.0682e+01    7.8844e-01    3.3620e+01    1.1262e+03
   35          80   -5.460658e-02    1.0682e+01    3.2579e-07    4.8637e+02    6.9815e+01
Callback 35 
   36          82   -5.513975e-02    1.0503e+01    1.6577e-02    1.7998e+01    6.1430e+01
Callback 36 
...
...
   46         102   -5.631263e-02    1.0302e+01    2.3449e-04    4.2686e+02    4.6923e+03
Callback 46 
   47         104   -5.631391e-02    1.0301e+01    6.9122e-05    1.3182e+02    4.2164e+03
Callback 47 
   48         111   -5.631391e-02    1.0301e+01    4.8086e-07    0.0000e+00    1.8238e+03
   49         112   -5.771186e-02    1.0221e+01    6.0825e-02    2.4022e+01    1.8516e+03
   50         113   -6.122140e-02    9.7431e+00    2.4964e-01    2.0228e+01    1.4180e+03
   51         114   -6.218662e-02    9.3517e+00    1.8286e-01    1.2756e+01    1.2415e+03
   52         115   -6.283865e-02    9.3188e+00    2.5355e-01    9.3855e+00    8.9534e+02
   53         116   -6.217617e-02    9.9860e+00    7.2290e-01    7.1422e+00    3.1128e+02
...

I have identified that pattern: each time the “Step Norm” column is zero (except for the iteration 0), the callback function is not called anymore (see iterations 34 and 48). Sometimes during a few iterations, sometimes tens or hundreds iterations (no pattern detected).

1 Like
2

Do you have access to - and are you able to - place a bug report on GitLab ?

With kind regards,

Claus

1 Like
3

Yes. I think I should first create a minimalist optimization problem starting from a difficult example, to get something easily reproducible.

I will look into fmincont/demos at main · jlmoya/fmincont · GitHub to see if I can reproduce the same behavior with one of these demos.

Vincent

4

Hello,

I think it is probably a side effect of the fix I made after the problem reported in https://scilab.discourse.group/t/problem-with-the-maxfunctionevaluations-limit-in-fmincon-options. I will have a look at it. However, just to be sure, can you test your code with the previous version of fmincon ?

S.

1 Like
5

I can try tomorrow.

Is this the way to go back to 1.0.9? :

atomsRemove(“fmincon”)
atomsInstall([“fmincon” “1.0.9”])

1 Like
6

I tried to install fmincon 1.0.9, but it is not available:

--> atomsSystemUpdate()
Examen du dépôt https://atoms.scilab.org/2026.1/TOOLBOXES/64 ... Terminé


--> atomsInstall(["fmincon" "1.0.9"])
atomsInstallList: The package "fmincon - 1.0.9" is not registered.
Please check on the ATOMS repository that it is available for Scilab 2026.1 on Linux.
If it is, run atomsSystemUpdate() before trying atomsInstall(..) again.
...
7

Hello.

I am afraid that atoms cannot handle this. To test you should first remove fmincon 1.1, then reinstall sci_ipopt and apifun, then download the source of fmincon 1.0.9 and build/load the toolbox from there:

atomsRemove fmincon
// maybe restart Scilab, then
atomsInstall sci_ipopt
atomsInstall apifun
http_get("https://atoms.scilab.org/toolboxes/fmincon/1.0.9/files/fmincont-1.0.9.tar.gz","fmincont109.tgz")
decompress fmincont109.tgz
cd fmincont-109
exec("builder.sce",-1)
exec("loader.sce",-1)
9

I have executed those instructions successfully, but can not access to optimoptions:

--> exec("loader.sce",-1)
Start Fmincon 1.0.9: loading macros, help, demos.
--> cd ..
--> exec('/run/media/mylogin/USB/generateurs_de_distributions_avec_symetrie/distribution_multihyperuniforme.sce', -1)
à la ligne   131 du fichier exécuté /run/media/mylogin/USB/generateurs_de_distributions_avec_symetrie/distribution_multihyperuniforme.sce

Variable non définie : optimoptions
-->

Line 131 is:

opt = optimoptions("fmincon") ;
10

Hello,

It seems to work on my machine :

Startup execution:
  loading initial environment

Start Apifun
	Load macros
	Load help
	Type "help apifun_overview" for quick start.

Start SciIPOpt 1.3.5: load gateways, help, demos.

--> cd fmincont-1.0.9/

 ans = 

  "/Users/mottelet/fmincont-1.0.9"

--> exec loader.sce

--> // This file is released under the 3-clause BSD license. See COPYING-BSD.

--> // Generated by builder.sce: Please, do not edit this file


--> oldmode = mode();      mode(-1);
Start Fmincon 1.0.9: loading macros, help, demos.

--> opt=optimoptions("fmincon")

 opt = (optim_options_fmincon tlist) with fields:

  fmincon default options:
  [struct] with fields:
  Algorithm = "ipopt"
  CheckGradients = %f
  ConstraintTolerance = 0.000001
  Display = "final"
  FiniteDifferenceStepSize = 0.0000061
  FiniteDifferenceType = "central"
  FunctionTolerance = 0.000001
  HessianApproximation = "lbfgs"
  HessianFcn = []
  HonorBounds = %t
  MaxFunctionEvaluations = 3000
  MaxIterations = 1000
  OptimalityTolerance = 0.000001
  OutputFcn = []
  PlotFcn = []
  SpecifyConstraintGradient = %f
  SpecifyObjectiveGradient = %f
  StepTolerance = 1.000D-10
  TypicalX = "ones(numberOfVariables,1)"
  InitBarrierParam = 0.1
  BarrierParamUpdate = "monotone"
  HessPattern = sparse([],[],[0,0])
  ScaleProblem = %f
11

If you have a general clear somewhere it may be the cause.

1 Like
12

Indeed! Thanks for guiding me.

fmicon 1.0.9 is now loaded. But my optimization has exactly the same problem, at the same iterations, than with 1.1 concerning the callback function.

13

I have tested all the fmincon demos and examples and didn’t see missed OutputFcn calls. Can you share your code or a small example code having the same problem ? I guess this is due to ipopt not calling the callback during a restoration phase or something like this (see my question on the IPopt github: About intermediate callback during all phases · Issue #864 · coin-or/Ipopt · GitHub).

S.

14

Hello,

I have fixed this problem in the new version of sci_ipopt (1.3.6). It will be used by fmincon if you remove/reinstall the package:

atomsRemove fmincon
atomsSystemUpdate
atomsInstall fmincon
atomsLoad fmincon
1 Like
15

Thanks a lot @mottelet ! I can confirm that the problem is now fixed in my computation.

I have such instructions in my callback function:

 if modulo(optimValues.iteration, 250) == 0 then

Until now I was only using it to make backups with save("variables.dat", "x") ;, so I did not notice the problem. But I am now using the callback function for making things really necessary in my model, and the problem became visible.

Thanks again for you reactivity.
Vincent

16

Thanks for reporting the problem. There are not that many users of fmincon, additionally solving hard problems making IPopt pass through the restoration phase, so the potential problem was completely hidden…

S.

1 Like