WNLIB/wnnlp -- a constrained non-linear optimization package in C (general optimization, not least squares). Constraints are handled by adding a penalty function.
I haven't used any of the libraries yet, but NLopt seems the most promising for me. It would be great if it had specialized interface and algorithms for (weighted) least-squares fitting.
BTW, does your note about Matlab mean that it has Lev-Mar with non-linear constraints?
I used NLopt for the optimization and the objective function was constructed to compute the squared error of the problem.
The algorithm that showed the most promising results was COBYLA (Local derivative-free optimization). It supports box constraints and non-linear constraints. The linear inequity constraints were introduced as non-linear constraints, which should be generally feasible.
Simple benchmarking shows that it does converge a little slower than a Lev-Mar approach, but speed is sacrificed due to the need for constraints.
MPFIT: A MINPACK-1 Least Squares Fitting Library in C
MPFIT uses the Levenberg-Marquardt technique to solve the least-squares problem. In its typical use, MPFIT will be used to fit a user-supplied function (the "model") to user-supplied data points (the "data") by adjusting a set of parameters. MPFIT is based upon MINPACK-1 (LMDIF.F) by More' and collaborators.
OPTIF9 can be converted to C (from Fortran) and may already have been by somebody.
If what you mean by box constraints is that it supports upper and lower limits on parameter values, I believe there is a version that does that.
That is a tricky problem, because it means whenever a parameter gets to a boundary, it effectively reduces the degrees of freedom by 1. It can get "stuck on a wall" when you didn't really want it to.
What we've found is that it's better to use an unconstrained minimizer and transform parameters, via something like a log or logit transform, so that in the search space they are unconstrained, but in the model space they are constrained.
As far as the other types of constraints, I don't know, although one option is, as part of your objective function, to make it get really bad when constraints are violated, so the optimizer avoids those areas.
I've found when I have a really flexible set of constraints, if I want a good trouble-free algorithm, I use Metropolis-Hastings. Unless I'm wrong, if it generates a sample that violates constraints, you can simply discard the sample. It takes longer, but it's simple and always works.
发布评论
评论（4）
前段时间我正在研究 C/C++ 最小二乘拟合库的状态。我记下了一些链接，包括您提供的链接以及：
ALGLIB/optimization -- Lev-Mar 与边界约束。
WNLIB/wnnlp —— C 语言的约束非线性优化包（一般优化，不是最小二乘）。约束是通过添加惩罚函数来处理的。
我还没有使用过任何库，但 NLopt 对我来说似乎是最有前途的。如果它有专门的接口和用于（加权）最小二乘拟合的算法，那就太好了。
顺便说一句，你关于 Matlab 的注释是否意味着它有带有非线性约束的 Lev-Mar ？
Some time ago I was researching the state of C/C++ least squares fitting libraries. I noted down a few links, including the ones you gave and also:
ALGLIB/optimization -- Lev-Mar with boundary constraints.
WNLIB/wnnlp -- a constrained non-linear optimization package in C (general optimization, not least squares). Constraints are handled by adding a penalty function.
I haven't used any of the libraries yet, but NLopt seems the most promising for me. It would be great if it had specialized interface and algorithms for (weighted) least-squares fitting.
BTW, does your note about Matlab mean that it has Lev-Mar with non-linear constraints?
我最终遵循的方法如下：
我使用NLopt用于优化，目标函数被构造为计算问题的平方误差。
显示出最有希望结果的算法是COBYLA（局部无导数优化）。它支持框约束和非线性约束。将线性不等式约束引入为非线性约束，这应该是普遍可行的。
简单的基准测试表明，它的收敛速度确实比 Lev-Mar 方法慢一些，但由于需要约束，速度受到了影响。
The approach I finally followed is the following:
I used NLopt for the optimization and the objective function was constructed to compute the squared error of the problem.
The algorithm that showed the most promising results was COBYLA (Local derivative-free optimization). It supports box constraints and non-linear constraints. The linear inequity constraints were introduced as non-linear constraints, which should be generally feasible.
Simple benchmarking shows that it does converge a little slower than a Lev-Mar approach, but speed is sacrificed due to the need for constraints.
MPFIT：C 中的 MINPACK-1 最小二乘拟合库
MPFIT 使用 Levenberg-Marquardt 技术来解决最小二乘问题。在其典型用途中，MPFIT 将用于通过调整一组参数来将用户提供的函数（“模型”）拟合到用户提供的数据点（“数据”）。 MPFIT 基于 More' 及其合作者的 MINPACK-1 (LMDIF.F)。
http://cow.physicals.wisc.edu/~craigm/idl/cmpfit .html
MPFIT: A MINPACK-1 Least Squares Fitting Library in C
MPFIT uses the Levenberg-Marquardt technique to solve the least-squares problem. In its typical use, MPFIT will be used to fit a user-supplied function (the "model") to user-supplied data points (the "data") by adjusting a set of parameters. MPFIT is based upon MINPACK-1 (LMDIF.F) by More' and collaborators.
http://cow.physics.wisc.edu/~craigm/idl/cmpfit.html
OPTIF9 可以转换为 C（来自 Fortran）并且可能已经被由某人。
如果框约束的意思是它支持参数值的上限和下限，我相信有一个版本可以做到这一点。
这是一个棘手的问题，因为这意味着每当参数到达边界时，它就会有效地将自由度降低 1。
当您并不真正想要它时，它可能会“卡在墙上”。
我们发现，最好通过对数或对数变换等方式使用无约束最小化器和变换参数，以便在搜索空间中它们不受约束，但在模型空间中它们受到约束。
至于其他类型的约束，我不知道，尽管作为目标函数的一部分，一种选择是在违反约束时使其变得非常糟糕，因此优化器会避免这些区域。
我发现当我有一组非常灵活的约束时，如果我想要一个好的无故障算法，我会使用 大都会-黑斯廷斯。
除非我错了，否则如果它生成违反约束的样本，您可以简单地丢弃该样本。
这需要更长的时间，但它很简单并且总是有效。
OPTIF9 can be converted to C (from Fortran) and may already have been by somebody.
If what you mean by box constraints is that it supports upper and lower limits on parameter values, I believe there is a version that does that.
That is a tricky problem, because it means whenever a parameter gets to a boundary, it effectively reduces the degrees of freedom by 1.
It can get "stuck on a wall" when you didn't really want it to.
What we've found is that it's better to use an unconstrained minimizer and transform parameters, via something like a log or logit transform, so that in the search space they are unconstrained, but in the model space they are constrained.
As far as the other types of constraints, I don't know, although one option is, as part of your objective function, to make it get really bad when constraints are violated, so the optimizer avoids those areas.
I've found when I have a really flexible set of constraints, if I want a good trouble-free algorithm, I use Metropolis-Hastings.
Unless I'm wrong, if it generates a sample that violates constraints, you can simply discard the sample.
It takes longer, but it's simple and always works.