Programming and Numerical Analysis: User Defined Optimization
Tools are provided for obtaining solutions to user specified optimization problems. These include:
- Solutions to sets of equations
- Linear programming
- Nonlinear optimization (maximize or minimize)
- User specified maximum likelihood problems
- Nonlinear least squares
- Nonlinear instrumental variables estimation
- Nonlinear GMM estimation
Optimization may be over one or more functions of a set of parameters, or may be some kind of summation or quadratic form (GMM) involving the data matrix. Components of the function to be optimized may include:
- All algebraic formats: standard operators, nested parentheses
- Standard functions: Log, Exp
- Trigonometric functions: Atn (arctan), Cos, Sin
- Gamma functions: gamma, log gamma, beta, incomplete gamma, incomplete beta, digamma, trigamma
- Probability functions: logit density, logit, logit probability, normal density, normal cdf, normal inverse Mills ratio, bivariate normal cdf, bivariate normal density
- Linear, bilinear, quadratic forms of parameters
- Gauss-Hermite and Gauss-Laguerre quadrature
- Integration by simulation - up to five simulation variables
- Maximum simulated likelihood estimation
- Weights
- Subfunctions and recursive function definition
- Optional derivatives
Optimization controls include:
- Starting values
- Maximum iterations
- Intermediate results
- Convergence criteria
Example: Heterogeneity in a probit model
To illustrate this tool, we consider a fairly intricate example. To the standard example of creating the user’s own probit estimator, we add an additional aspect, a likelihood function in which there is heterogeneity which is integrated out of the conditional density. Consider a probit model in which there is normally distributed, unobserved individual heterogeneity which multiplies one of the variables in the model,
where v is standard normally distributed. The probability that enters the log-likelihood is Prob[y = j] = Ev [Prob[ y = j | v ]], j = 0,1. The expectation is exactly equal to
The following commands estimate this model. The command uses Hermite quadrature to do the integration. Estimates using an artificial data set are shown.
SAMPLE
CREATE
CREATE
NAMELIST
CALC
PROBIT
MAXIMIZE
; 1 - 200
; z1
; v
; y
; q
; z
; kz
; Lhs
; Hrq
; Fcn
; Start
; Labels
; Output
$
= Rnn(0,1) ; z2 = Rnn(0,1)
= Rnn(0,1) ; x = Rnu(-.5,.5) $
= ( .2 + z1 + z2 + v*x + Rnn(0,1) ) > 0
= 2*y - 1 $
= one,z1,z2 $
= Col(z) $
= y ; Rhs = z $ Starting values
= u ; Hpt = 20
= Prob = Ntg (1/Sqr(pi) * Phi (q*(b1'z + t*u*x ))) | log(Prob)
= b,.1
= kz_b,t
= 3$
The following output results. (The probit results for the starting values are omitted.)
Nonlinear Estimation of Model Parameters
Method=BFGS ; Maximum iterations=100
Convergence criteria:
gtHg .1000D-05 chg.F .0000D+00 max|dB| .0000D+00
Nodes for quadrature: Laguerre=40;Hermite=20.
Replications for GHK simulator= 10
Start values: .33655D+00 .94078D+00 .90944D+00 .10000D+00
1st derivs. -.26366D-02 -.41864D-02 -.97865D-02 -.22068D-01
Parameters: .33655D+00 .94078D+00 .90944D+00 .10000D+00
Itr 1 F= .8546D+02 gtHg= .2464D-01 chg.F= .8546D+02
1st derivs. .24435D-02 -.57443D-02 .53724D-01 -.23025D-01
Parameters: .33690D+00 .94133D+00 .91072D+00 .10290D+00
Itr 2 F= .8546D+02 gtHg= .5878D-01 chg.F= .3983D-04
1st derivs. .87298D-02 .21673D-01 -.79062D-02 -.22928D-01
Parameters: .33685D+00 .94143D+00 .90976D+00 .10331D+00
(some iterations omitted)
Itr 12 F= .8503D+02 gtHg= .2383D-05 chg.F= .1082D-07
1st derivs. -.11755D-05 .64422D-06 -.72063D-06 .71775D-07
Parameters: .42442D+00 .11058D+01 .11226D+01 .34385D+01
Itr 13 F= .8503D+02 gtHg= .2052D-06 chg.F= .3283D-11
* Converged
Function= .85459845816D+02, at entry, .85030583389D+02 at exit
**************************************************
* Hermite quadrature with 20 nodes (points) *
**************************************************
+---------------------------------------------+
| User Defined Optimization |
| Dependent variable Function |
| Number of observations 200 |
| Iterations completed 13 |
| Log likelihood function 85.03058 |
| Chi-squared 170.0612 |
| Degrees of freedom 4 |
| Significance level .0000000 |
+---------------------------------------------+
+---------+--------------+----------------+--------+---------+
|Variable | Coefficient | Standard Error |b/St.Er.|P[|Z|>z] |
+---------+--------------+----------------+--------+---------+
B1 .4244209422 .20178167 2.103 .0354
B2 1.105771985 .29090246 3.801 .0001
B3 1.122614759 .34435892 3.260 .0011
T 3.438534630 2.8813847 1.193 .2327