Accuracy: Nonlinear Least Squares

Nonlinear regression is more difficult than the linear computations described above. B.D. McCullough (Journal of Applied Econometrics, 1999) surveyed numerous programs for their ability to solve nonlinear least squares problems. LIMDEP and NLOGIT were able to solve nearly all the benchmark problems using only the program default settings, and all of the rest with only minor additional effort. The example below is one of the most difficult of the set. The (correct) solution from the more difficult starting values is routine.

The NIST data and solution file

Data:          1 Response  (y)
               1 Predictor (x)
               16 Observations
               Higher Level of Difficulty
Model:         Exponential Class
               y = b1 * exp[b2/(x+b3)]  +  e
          Starting values             Certified Values
        Start 1     Start 2      Parameter     Standard Deviation
  b1 =        2         0.02  5.6096364710E-03  1.5687892471E-04
  b2 =   400000      4000     6.1813463463E+03  2.3309021107E+01
  b3 =    25000       250     3.4522363462E+02  7.8486103508E-01
Residual Sum of Squares:                    8.7945855171E+01
Residual Standard Deviation:                2.6009740065E+00
Degrees of Freedom:                                13

LIMDEP and NLOGIT solution

READ ; Nobs=16 ; Nvar=2 ; Names=Ym10,Xm10$
      3.478000E+04    5.000000E+01
      2.861000E+04    5.500000E+01
      2.365000E+04    6.000000E+01
      1.963000E+04    6.500000E+01
      1.637000E+04    7.000000E+01
      1.372000E+04    7.500000E+01
      1.154000E+04    8.000000E+01
      9.744000E+03    8.500000E+01
      8.261000E+03    9.000000E+01
      7.030000E+03    9.500000E+01
      6.005000E+03    1.000000E+02
      5.147000E+03    1.050000E+02
      4.427000E+03    1.100000E+02
      3.820000E+03    1.150000E+02
      3.307000E+03    1.200000E+02
      2.872000E+03    1.250000E+02
NLSQ  ; Lhs=Ym10
      ; Fcn=B1*EXP(B2/(Xm10+B3)) 
      ; Labels=B1,B2,B3 ; Dfc
      ; Start=2,400000,25000 ; Maxit=10000$
MATRIX; Peek ; b $
CALC  ; Peek ; sumsqdev $
MATRIX; sd=diag(varb) ; sd=sqrt(sd) ; peek ; vecd(sd)$

-----------------------------------------------------------------------------
User Defined Optimization.........................
Nonlinear    least squares regression ............
LHS=YM10     Mean                 =    12432.06250
             Standard deviation   =     9722.36427
             Number of observs.   =             16
Model size   Parameters           =              3
             Degrees of freedom   =             13
Residuals    Sum of squares       =        87.9459
             Standard error of e  =        2.60097
Fit          R-squared            =        1.00000
             Adjusted R-squared   =        1.00000
Model test   F[  2,    13] (prob) =********(.0000)
Diagnostic   Log likelihood       =      -36.33608
             Restricted(b=0)      =     -169.10165
             Chi-sq [  2]  (prob) = 265.5(  .0000)
Info criter. Akaike Info. Criter. =        2.07913
Not using OLS or no constant. Rsqrd & F may be < 0
--------+--------------------------------------------------------------------
        |                  Standard            Prob.      95% Confidence
UserFunc|  Coefficient       Error       z    |z|>Z*         Interval
--------+--------------------------------------------------------------------
      B1|     .00561***      .00016    35.76  .0000      .00530    .00592
      B2|    6181.35***    23.30896   265.19  .0000     6135.66   6227.03
      B3|    345.224***      .78486   439.85  .0000     343.685   346.762
--------+--------------------------------------------------------------------
Note: ***, **, * ==>  Significance at 1%, 5%, 10% level.
-----------------------------------------------------------------------------

Display of all internal digits of matrix B
B       [0001] =    .56096364112609250D-02
B       [0002] =    .61813463551864850D+04
B       [0003] =    .34522363492429820D+03
[CALC] SUMSQDEV=    .87945855170450020D+02
Display of all internal digits of matrix Result
Result  [0001] =    .15687857055954240D-03
Result  [0002] =    .23308961585643500D+02
Result  [0003] =    .78485908040972090D+00