Programming: User Defined Optimization
User Defined Optimization Tools
Tools are provided for obtaining solutions to user specified optimization problems. These include:
- Solutions to sets of equations
- Linear programming
- Nonlinear optimization (maximize or minimize) with constraints
- User specified maximum likelihood problems
- Nonlinear least squares
- Nonlinear instrumental variables and GMM estimation
Components of the Optimized Function
Optimization may be over one or more functions of a set of parameters, or may be some kind of summation or quadratic form (GMM) involving the data matrix. Components of the function to be optimized may include:
- All algebraic formats: standard operators, nested parentheses
- Standard functions: Log, Exp
- Trigonometric functions: Atn (arctan), Cos, Sin
- Gamma functions: gamma, log gamma, beta, incomplete gamma, incomplete beta, digamma, trigamma
- Probability functions: logit density, logit, logit probability, normal density, normal cdf, normal inverse Mills ratio, bivariate normal cdf, bivariate normal density
- Linear, bilinear, quadratic forms of parameters
- Gauss-Hermite and Gauss-Laguerre quadrature
- Integration by simulation - up to five simulation variables
- Maximum simulated likelihood estimation
- Weights
- Subfunctions and recursive function definition
- Optional derivatives
Example: Heterogeneity in a Probit Model
To illustrate this tool, we consider a fairly intricate example. To the standard example of creating the user’s own probit estimator, we add an additional aspect, a likelihood function in which there is heterogeneity which is integrated out of the conditional density. Consider a probit model in which there is normally distributed, unobserved individual heterogeneity which multiplies one of the variables in the model,
y = 0 or 1
Prob(y=1|x,v) = F(b’x + cxv)
where v is standard normally distributed. The probability that enters the log-likelihood is Prob[y = j] = Ev [Prob[ y = j | v ]], j = 0,1. The expectation is exactly equal to
P(y=j|x) = sqr(1/2pi) * Integral(-inf,inf)exp(-.5*v^2)F[(2j-1)(b’x + cxv)dv
The following commands estimate this model. The command uses Hermite quadrature to do the integration. Estimates using an artificial data set are shown.
SAMPLE
CREATE
CREATE
NAMELIST
CALC
PROBIT
MAXIMIZE
; 1 - 200
; z1
; v
; y
; q
; z
; kz
; Lhs
; Hrq
; Fcn
; Start
; Labels
; Output
$
= Rnn(0,1) ; z2 = Rnn(0,1)
= Rnn(0,1) ; x = Rnu(-.5,.5) $
= ( .2 + z1 + z2 + v*x + Rnn(0,1) ) > 0
= 2*y - 1 $
= one,z1,z2 $
= Col(z) $
= y ; Rhs = z $ Starting values
= u ; Hpt = 20
= Prob = Ntg (1/Sqr(pi) * Phi (q*(b1'z + t*u*x ))) | log(Prob)
= b,.1
= kz_b,t
= 3$
The following output results. (The probit results for the starting values are omitted.)
Nonlinear Estimation of Model Parameters Method=BFGS ; Maximum iterations=100 Convergence criteria: gtHg .1000D-05 chg.F .0000D+00 max|dB| .0000D+00 Nodes for quadrature: Laguerre=40;Hermite=20. Replications for GHK simulator= 10 Start values: .33655D+00 .94078D+00 .90944D+00 .10000D+00 1st derivs. -.26366D-02 -.41864D-02 -.97865D-02 -.22068D-01 Parameters: .33655D+00 .94078D+00 .90944D+00 .10000D+00 Itr 1 F= .8546D+02 gtHg= .2464D-01 chg.F= .8546D+02 1st derivs. .24435D-02 -.57443D-02 .53724D-01 -.23025D-01 Parameters: .33690D+00 .94133D+00 .91072D+00 .10290D+00 Itr 2 F= .8546D+02 gtHg= .5878D-01 chg.F= .3983D-04 1st derivs. .87298D-02 .21673D-01 -.79062D-02 -.22928D-01 Parameters: .33685D+00 .94143D+00 .90976D+00 .10331D+00 (some iterations omitted) Itr 12 F= .8503D+02 gtHg= .2383D-05 chg.F= .1082D-07 1st derivs. -.11755D-05 .64422D-06 -.72063D-06 .71775D-07 Parameters: .42442D+00 .11058D+01 .11226D+01 .34385D+01 Itr 13 F= .8503D+02 gtHg= .2052D-06 chg.F= .3283D-11 * Converged Function= .85459845816D+02, at entry, .85030583389D+02 at exit ************************************************** * Hermite quadrature with 20 nodes (points) * ************************************************** +---------------------------------------------+ | User Defined Optimization | | Dependent variable Function | | Number of observations 200 | | Iterations completed 13 | | Log likelihood function 85.03058 | | Chi-squared 170.0612 | | Degrees of freedom 4 | | Significance level .0000000 | +---------------------------------------------+ +---------+--------------+----------------+--------+---------+ |Variable | Coefficient | Standard Error |b/St.Er.|P[|Z|>z] | +---------+--------------+----------------+--------+---------+ B1 .4244209422 .20178167 2.103 .0354 B2 1.105771985 .29090246 3.801 .0001 B3 1.122614759 .34435892 3.260 .0011 T 3.438534630 2.8813847 1.193 .2327