This is a compilation of answers to questions frequently asked by

`levmar`users.

*Q1 -- What is*`levmar`?*Q2 -- What types of minimization problems can*`levmar`solve?*Q3 -- Where can I find more information on nonlinear regression/data fitting?**Q4 -- Who is using*`levmar`?*Q5 -- What do I need to use*`levmar`?*Q6 -- How can I compile*`levmar`?*Q7 -- Where are the arguments of the various routines explained in detail?**Q8 -- Are matrices stored in row or column major order?**Q9 -- How can I pass my own data for use by the objective function and its Jacobian?**Q10 -- Can I use*`levmar`for problems whose Jacobian is unavailable or difficult/expensive to compute?*Q11 -- How can I obtain an analytic Jacobian for my minimization problem??**Q12 -- How can I verify the correctness of a Jacobian?**Q13 -- How should I choose the starting point for a minimization?**Q14 -- How can I force an early termination of the minimization?**Q15 -- How can I deal with badly scaled problems?**Q16 -- How can I compile just the single or double variants of*`levmar`'s functions?*Q17 -- How can I force the linear solvers free their allocated memory before returning?**Q18 -- What is the*`covar`matrix computed by the various routines?*Q19 -- How can I solve a weighted non-linear least squares problem using*`levmar`?*Q20 -- How can I solve a non-linear least squares problem with some fixed parameters?**Q21 -- How can I solve a problem involving box constraints with infinite endpoints?**Q22 -- What if my constrained optimization problem has an infeasible starting point?**Q23 -- Where can I find an example of using*`levmar`for curve fitting?*Q24 -- Where can I find an example of a real-world problem using*`levmar`?*Q25 -- Why am I getting syntax errors in*`lmaccess.h`with Visual Studio?*Q26 -- Why am I getting a link error for*`MAIN__`?*Q27 -- Why am I getting a link error for*`sgemm`/`dgemm`?*Q28 -- How can I determine what kind of BLAS library I have installed?**Q29 -- Can I use*`levmar`from matlab?*Q30 -- How can I compile*`levmar`'s MEX-file?*Q31 -- Can*`levmar`benefit from a multi-core architecture?*Q32 -- Can I use*`levmar`from perl?*Q33 -- Can I use*`levmar`from python?*Q34 -- Can I use*`levmar`from haskell?*Q35 -- Can I use*`levmar`from tcl?*Q36 -- Can I include*`levmar`in a commercial product?

-- [top]**Q1 -- What is**`levmar`?`levmar`is a C/C++ implementation of the Levenberg-Marquardt nonlinear least squares minimization algorithm that is distributed under the GNU General Public License.-- [top]**Q2 -- What types of minimization problems can**`levmar`solve?`levmar`supports unconstrained nonlinear least squares minimization with analytic or approximate Jacobians. Such problems typically arise from the solution of systems of nonlinear equations or in data fitting applications.`levmar`has also some support for constrained nonlinear least squares, allowing linear equation, linear inequality and box inequality constraints to be imposed.-- [top]**Q3 -- Where can I find more information on nonlinear regression/data fitting?**Nonlinear regression, i.e. the problem of fitting a nonlinear model to your data, is far too broad to be covered here. A useful introduction can be found in these notes from TU Denmark. Another free source is http://www.curvefit.com/; yet another option is to consult sections 15.4 and 15.5 of the Numerical Recipes book.

-- [top]**Q4 -- Who is using**`levmar`?`levmar`is being used by thousands of individuals to solve nonlinear regression problems in various disciplines. It has also been integrated into open source and commercial products, the list of which currently includes:- Hugin, a panorama photo stitcher
- SciGraphica, a package for scientific data analysis and technical graphics
- Tarquin, an analysis tool for NMR spectroscopic data
- Teem, a group of libraries for representing, processing, and visualizing scientific raster data
- CellML, a language for describing mathematical models
- Astrometry.net, an astrometric calibraton service
- Stimfit, a software for the analysis of electrophysiological data
- StochFit, a software for modeling specular x-ray reflectivity or neutron reflectivity data.
- Autopano, a program for creating image panoramas

Additionally, several commercial license agreements have been signed with private companies, allowing them to use

`levmar`in their products.

`levmar`has also been employed in a number of research papers, check Google Scholar for an incomplete list.-- [top]**Q5 -- What do I need to use**`levmar`?To be able to use all

`levmar`functions, you should have installed LAPACK or an equivalent vendor library. On Intel processors, MKL is highly recommended. Netlib has an f2c'ed free version at http://www.netlib.org/clapack.`levmar`users have also reported that the precompiled MSWin libraries at Netlib are broken and suggest to build them yourself. Alternatively, precompiled MSWin LAPACK/BLAS libraries are offered here and here. It is also noted that a subset of`levmar`'s functionality is available even without LAPACK. The use (or not) of LAPACK is controlled by the`HAVE_LAPACK`macro definition in`levmar.h`.-- [top]**Q6 -- How can I compile**`levmar`?First, you have to decide if you want LAPACK or not. In case that you plan to use it, you should make sure that it is installed at your site. The second step is to compile

`levmar`itself. The tarfile contains makefiles for Unix/Linux using gcc (Makefile) or Intel's icc (Makefile.icc) and MSWin using Visual Studio/Visual Studio Express (Makefile.vc). Please read the comments within those files for more information.

There is also the option of not using any of the supplied Makefiles but rather generate platform-specific compilation commands using the supplied`CMakeLists.txt`file with the CMake build system; please refer to CMake's online documentation for more information on how to use it.-- [top]**Q7 -- Where are the arguments of the various routines explained in detail?**The source code of each function is preceeded by detailed comments explaining the role of each argument. See the *_core.c files.

-- [top]**Q8 -- Are matrices stored in row or column major order?**Matrices are stored using the normal C convention, i.e. in row major order.

-- [top]**Q9 -- How can I pass my own data for use by the objective function and its Jacobian?**Assume that you want to pass two arrays, one of doubles and one of integers. A quick but not very elegant solution is to use global variables. The simplest way to avoid using globals is to declare a structure as

struct mydata{ double dar[XXX]; int iar[YYY]; };

where XXX and YYY denote the appropriate array sizes. Then, define a structure variable withstruct mydata data;

and fill it:data.dar[0]=7.0; data.iar[0]=-17; // etc

Following this, call the appropriate LM routine passing it the address of data as the adata argument, e.g.ret=dlevmar_der(func, fjac, ..., (void *)&data);

Your func and fjac routines should interpret the supplied data using type casting:struct mydata *dptr; dptr=(struct mydata *)adata; // adata is passed as void * // supplied data can now be accessed as dptr->dar[0], etc

-- [top]**Q10 -- Can I use**`levmar`for problems whose Jacobian is unavailable or difficult/expensive to compute?Yes.

`levmar`includes routines that approximate the Jacobian using a combination of finite differences and secant updates. These routines have the suffix`_dif`, as opposed to those using analytic Jacobians that end in`_der`. Typically, minimizing with approximate Jacobians takes longer to converge compared to when the Jacobian is supplied analytically (see also Q11). However, cases where the Jacobian is very expensive to compute are exceptions to this rule. In such situations, better performance can be achieved by employing the`_dif`routines, even if the analytic expression for the Jacobian is known.-- [top]**Q11 -- How can I obtain an analytic Jacobian for my minimization problem?**Employing analytic Jacobians is generally recommended due to faster convergence. When the objective function is too complicated for hand-coding its Jacobian, the latter can be generated through the use of automatic differentiation tools such as those listed here, that work by systematically applying the chain rule to a given code segment.

-- [top]**Q12 -- How can I verify the correctness of a Jacobian?**`levmar`includes the functions`dlevmar_chkjac()`and`slevmar_chkjac()`which allow you to check the Jacobian of a function for consistency with the function itself. More details can be found in the comments preceeding the source code.-- [top]**Q13 -- How should I choose the starting point for a minimization?**Selecting the starting point for a particular minimization problem is largely problem-specific. In some cases, your background knowledge of the problem may guide you to select reasonable initial values for the problem's parameters. In other cases, you might be able to define a simplified linear version of the non-linear problem under consideration, whose solution will serve as the starting point for the non-linear problem. You might even use random values for your starting point. Beware, however, that the success of this approach is highly dependent on the nature of your problem: If it is highly non-linear or it has several local minima, it is very likely that you will end up with unexpected surprises.

-- [top]**Q14 -- How can I force an early termination of the minimization?**In certain applications, a minimization problem can be considered to be satisfactory solved as soon as the least squares error becomes smaller than some value. Despite that more iterations can reduce this error further, the reduction is not so critical for the problem at hand and might take more computation time to be achieved. In such cases, by setting

`opts[3]`to a suitable value, the user can instruct`levmar`to return (with code`6`) as soon as the least squares error becomes sufficiently small.-- [top]**Q15 -- How can I deal with badly scaled problems?**Practical applications of minimization often involve variables that vary greatly in magnitude. Such minimization problems are characterized as being badly scaled and, unless adjusted, may lead to poor convergence.

`levmar`does not include an explicit mechanism for dealing with badly scaled problems since such mechanisms are, to a large extent, problem-specific. Nevertheless, the user can account for badly scaled problems by using her knowledge of the problem domain to rescale the independent variables, i.e. change their units so that they all have approximately the same range. See chapter 7 in Dennis-Schnabel for more details.-- [top]**Q16 -- How can I compile just the single or double variants of**`levmar`'s functions?`levmar`offers single and double precision variants of all optimization functions it includes. Starting with version 2.3, it supports the option of compiling only the single or only the double precision variants. This results in a library of smaller size. The choice of compiling the single or double precision variants is controlled by defining the macros`LM_SNGL_PREC`and`LM_DBL_PREC`, respectively. By default, both macros are defined.-- [top]**Q17 -- How can I force the linear solvers free their allocated memory before returning?**The default behavior of the linear solver routines is to retain memory between calls, aiming to increase performance by avoiding excessive free's and malloc's. If this is undesirable, it can be turned off by recompiling

`levmar`without the`LINSOLVERS_RETAIN_MEMORY`macro definition.-- [top]**Q18 -- What is the**`covar`matrix computed by the various routines?Upon return, all routines fill

`covar`with the variance-covariance matrix of the best-fit parameters, which estimates the statistical error on the latter:`covar`'s diagonal (off-diagonal) elements are estimates of the variances (covariances) of the best-fit parameters. More information can be found at the background section of this note.-- [top]**Q19 -- How can I solve a weighted non-linear least squares problem using**`levmar`?Currently

`levmar`does not directly offer support for weighted non-linear least squares problems. However, it is quite straightforward to program this for a particular problem, as follows: Let us assume that you want to minimize ||e||_C = e^T*C*e with e being your error vector (e=x-f(p)) and C a symmetric positive-definite weight matrix. Then, you can equivalently minimize s^T*s without weights, where s=W*e=W*x - W*f(p) and W is the Cholesky decomposition of C, i.e. C=W^T*W. This is because s^T*s=e^t*W^t*W*e=e^T*C*e=||e||_C. In the case that all non-diagonal elements of C are zero, then W is simply the diagonal matrix resulting after taking the sqrt of C's diagonal elements. See also the discussion here.

Functions`dlevmar_chol()`/`slevmar_chol()`included in`levmar`can be employed to compute the Cholesky decomposition of a symmetric positive-definite matrix.-- [top]**Q20 -- How can I solve a non-linear least squares problem with some fixed parameters?**Occasionally, it is required to solve problems where certain parameters are held to fixed values and the objective function is minimized with respect to the remaining ones only. This can be achieved in the following two ways:

- The fixed parameters are not included in the parameter vector. The values of the fixed parameters can be
communicated to the objective function as user-supplied arguments through the
`adata`mechanism (see Q9) or in global variables. - Columns of the Jacobian which correspond to the fixed parameters are explicitly set to zero.

- The fixed parameters are not included in the parameter vector. The values of the fixed parameters can be
communicated to the objective function as user-supplied arguments through the
-- [top]**Q21 -- How can I solve a problem involving box constraints with infinite endpoints?**To deal with non-linear least squares problems for which no lower bound inequality constraints apply to certain variables, the corresponding entries in the

`lb`argument of`dlevmar_bc_XXX`/`slevmar_bc_XXX`or`dlevmar_blec_XXX`/`slevmar_blec_XXX`should be set to`-DBL_MAX`/`-FLT_MAX`(i.e., minus infinity), respectively. Similarly, when there are no upper bound constraints, the entries in the`ub`argument should be set to`DBL_MAX`/`FLT_MAX`.-- [top]**Q22 -- What if my constrained optimization problem has an infeasible starting point?**It is strongly advised that the starting point to a constrained optimization problem is feasible, i.e. satisfies any equality, inequality and box constraints that should apply. In the opposite case,

`levmar`maps an infeasible point to a feasible one but does so in a very crude manner that might move the resulting feasible point far from the local minimum.-- [top]**Q23 -- Where can I find an example of using**`levmar`for curve fitting?An example showing how to use

`levmar`for fitting a three-parameter exponential model to a set of data measurements is in`expfit.c`; refer to the comments annotating the code for more details.-- [top]**Q24 -- Where can I find an example of a real-world problem using**`levmar`?An example of a real-word use of

`levmar`is the`homest`library which is concerned with estimating a*homography*, that is a plane projective transformation that arises in computer vision.-- [top]**Q25 -- Why am I getting syntax errors in**`lmaccess.h`with Visual Studio?This is most likely to occur when compiling an application that makes use of

`levmar`version 2.4 or earlier under Visual Studio. The problem is caused by an erroneous include path option passed to the compiler, which results in the inadvertent inclusion of an irrelevant header file. Specifically, Visual Studio comes with a header file also called`lm.h`. If a directive such as`#include <lm.h>`is included in a source file while the`/I`switch to the compiler does not specify the directory where`levmar`'s`lm.h`resides, then Visual Studio's`lm.h`gets included resulting in compilation errors. The solution is to ensure that an appropriate include path is specified to the compiler with the`/I`switch. Starting with`levmar`version 2.5,`lm.h`was renamed to`levmar.h`to avoid such conflicts. For backwards compatibility,`lm.h`has been retained as a symbolic link to`levmar.h`. However, its use in new code is discouraged.-- [top]**Q26 -- Why am I getting a link error for**`MAIN__`?When building

`lmdemo`, certain LAPACK/BLAS libraries cause the linker to fail with an error like*undefined reference to `MAIN__'*, or similar. This is due to these libraries superfluously calling a Fortran main() routine for initialization. There are several workarounds to this problem:- If using
`gcc`, pass the option -u MAIN__ to the linker. - If compiling under Windows, link against these precompiled libraries.
- Define a dummy MAIN__ function as
*int MAIN__(int argc, char **argv) { abort(); return 1; }*

See also the discussion here.

- If using
-- [top]**Q27 -- Why am I getting a link error for**`sgemm`/`dgemm`?This is a BLAS name mangling problem. Make sure that the definition of

`LM_BLAS_PREFIX`in`misc.h`matches your system:- Define it being empty if you have a F77 BLAS (default).
- Define it as f2c_ if you have an f2c'ed BLAS.
- Define it as cblas_ if you have C interface to BLAS.

For the f2c'ed/CBLAS cases, you should also define

`LM_BLAS_SUFFIX`as being empty in`misc.h`, instead of the default of being defined as an underscore. See also Q28.-- [top]**Q28 -- How can I determine what kind of BLAS library I have installed?**On unix, try

`nm libblas.a | grep gemm`. Under Windows, use`dumpbin /all blas.lib | more`. Inspect the output of those commands for prefixes like`f2c_`and`cblas_`to the`gemm`routines.-- [top]**Q29 -- Can I use**`levmar`from matlab?Yes. Starting with version 2.2,

`levmar`includes a MEX-file that can be used to interface it with matlab. See`levmar.c`in the`matlab`directory of`levmar`'s source distribution. Documentation for using`levmar`with matlab is provided in file`levmar.m`. Assuming that the MEX-file has been compiled (see also Q30), file`lmdemo.m`contains examples of its use with matlab. Try it with the command`matlab < lmdemo.m`. It has been reported that the MEX-file also works with GNU octave, but I have not tested this myself; if you succeed with it, please let me know.-- [top]**Q30 -- How can I compile**`levmar`'s MEX-file?The answer depends on your operating system/compiler. The

`Makefile`in the`matlab`directory works with GCC under Linux, whereas`Makefile.w32`works with MSVC under Windows. You might also need to run "mex -setup" from the matlab or OS command prompt to configure your system to build MEX-files. In any case, the provided makefiles can be used as a starting point for compiling on your own system. If building under Windows, you might also consider using gnumex.-- [top]**Q31 -- Can**`levmar`benefit from a multi-core architecture?Yes, in some cases. Starting with version 2.6,

`levmar`includes support for a parallel Cholesky solver from the PLASMA linear algebra package for homogeneous multi-core architectures. This parallel solver can improve the performance of large optimization problems, i.e. problems whose number of parameters is in the order of several hundreds. Since the gain in performance is platform-specific, the user is advised to do some testing for determining the speed up achieved by the parallel solver, if any. Obviously, for small optimization problems, the overheads of parallel execution will outweigh the benefits. To use the parallel Cholesky solver, the`HAVE_PLASMA`macro in`levmar.h`should be defined prior to compilation and the invocations of`AX_EQ_B_PLASMA_CHOL()`uncommented in the source code.-- [top]**Q32 -- Can I use**`levmar`from perl?Yes. John Lapeyre has written a PDL interface to

`levmar`.-- [top]**Q33 -- Can I use**`levmar`from python?Yes. Alastair Tse has written a python interface to

`levmar`.-- [top]**Q34 -- Can I use**`levmar`from haskell?Yes. Bas and Roel van Dijk have written a haskell interface to

`levmar`.-- [top]**Q35 -- Can I use**`levmar`from tcl?Yes. Andrey Nakin has written a tcl interface to

`levmar`.-- [top]**Q36 -- Can I include**`levmar`in a commercial product?`levmar`is distributed under GPL, which allows you to modify the original code into a new program and then sell copies of the modified program commercially, but only under the terms of the GPL. Thus, you must make the source code available to the users of the program as described in the GPL, and they must be allowed to redistribute and modify it as described in the GPL. Clearly, such requirements are in conflict with using`levmar`as part of a proprietary application. However, it is possible to obtain a paid license for proprietary commercial use under terms different than those of GPL. Please contact the author for further information on such licensing. More answers to questions related to the GPL license can be found here.