Accuracy: Nonlinear Least Squares
Nonlinear regression is more difficult than the linear computations described above. B.D. McCullough (Journal of Applied Econometrics, 1999) surveyed numerous programs for their ability to solve nonlinear least squares problems. LIMDEP and NLOGIT were able to solve nearly all the benchmark problems using only the program default settings, and all of the rest with only minor additional effort. The example below is one of the most difficult of the set. The (correct) solution from the more difficult starting values is routine.
The NIST data and solution file
Data: 1 Response (y)
1 Predictor (x)
16 Observations
Higher Level of Difficulty
Model: Exponential Class
y = b1 * exp[b2/(x+b3)] + e
Starting values Certified Values
Start 1 Start 2 Parameter Standard Deviation
b1 = 2 0.02 5.6096364710E-03 1.5687892471E-04
b2 = 400000 4000 6.1813463463E+03 2.3309021107E+01
b3 = 25000 250 3.4522363462E+02 7.8486103508E-01
Residual Sum of Squares: 8.7945855171E+01
Residual Standard Deviation: 2.6009740065E+00
Degrees of Freedom: 13
LIMDEP and NLOGIT solution
READ ; Nobs=16 ; Nvar=2 ; Names=Ym10,Xm10$
3.478000E+04 5.000000E+01
2.861000E+04 5.500000E+01
2.365000E+04 6.000000E+01
1.963000E+04 6.500000E+01
1.637000E+04 7.000000E+01
1.372000E+04 7.500000E+01
1.154000E+04 8.000000E+01
9.744000E+03 8.500000E+01
8.261000E+03 9.000000E+01
7.030000E+03 9.500000E+01
6.005000E+03 1.000000E+02
5.147000E+03 1.050000E+02
4.427000E+03 1.100000E+02
3.820000E+03 1.150000E+02
3.307000E+03 1.200000E+02
2.872000E+03 1.250000E+02
NLSQ ; Lhs=Ym10
; Fcn=B1*EXP(B2/(Xm10+B3))
; Labels=B1,B2,B3 ; Dfc
; Start=2,400000,25000 ; Maxit=10000$
MATRIX; Peek ; b $
CALC ; Peek ; sumsqdev $
MATRIX; sd=diag(varb) ; sd=sqrt(sd) ; peek ; vecd(sd)$
-----------------------------------------------------------------------------
User Defined Optimization.........................
Nonlinear least squares regression ............
LHS=YM10 Mean = 12432.06250
Standard deviation = 9722.36427
Number of observs. = 16
Model size Parameters = 3
Degrees of freedom = 13
Residuals Sum of squares = 87.9459
Standard error of e = 2.60097
Fit R-squared = 1.00000
Adjusted R-squared = 1.00000
Model test F[ 2, 13] (prob) =********(.0000)
Diagnostic Log likelihood = -36.33608
Restricted(b=0) = -169.10165
Chi-sq [ 2] (prob) = 265.5( .0000)
Info criter. Akaike Info. Criter. = 2.07913
Not using OLS or no constant. Rsqrd & F may be < 0
--------+--------------------------------------------------------------------
| Standard Prob. 95% Confidence
UserFunc| Coefficient Error z |z|>Z* Interval
--------+--------------------------------------------------------------------
B1| .00561*** .00016 35.76 .0000 .00530 .00592
B2| 6181.35*** 23.30896 265.19 .0000 6135.66 6227.03
B3| 345.224*** .78486 439.85 .0000 343.685 346.762
--------+--------------------------------------------------------------------
Note: ***, **, * ==> Significance at 1%, 5%, 10% level.
-----------------------------------------------------------------------------
Display of all internal digits of matrix B
B [0001] = .56096364112609250D-02
B [0002] = .61813463551864850D+04
B [0003] = .34522363492429820D+03
[CALC] SUMSQDEV= .87945855170450020D+02
Display of all internal digits of matrix Result
Result [0001] = .15687857055954240D-03
Result [0002] = .23308961585643500D+02
Result [0003] = .78485908040972090D+00