Programming and Numerical Analysis: User Defined Optimization

User Defined Optimization Tools

Tools are provided for obtaining solutions to user specified optimization problems. These include:

  • Solutions to sets of equations
  • Linear programming
  • Nonlinear optimization (maximize or minimize) with constraints
  • User specified maximum likelihood problems
  • Nonlinear least squares
  • Nonlinear instrumental variables and GMM estimation

Components of the Optimized Function

Optimization may be over one or more functions of a set of parameters, or may be some kind of summation or quadratic form (GMM) involving the data matrix. Components of the function to be optimized may include:

  • All algebraic formats: standard operators, nested parentheses
  • Standard functions: Log, Exp
  • Trigonometric functions: Atn (arctan), Cos, Sin
  • Gamma functions: gamma, log gamma, beta, incomplete gamma, incomplete beta, digamma, trigamma
  • Probability functions: logit density, logit, logit probability, normal  density, normal cdf, normal inverse Mills ratio, bivariate normal cdf, bivariate normal density
  • Linear, bilinear, quadratic forms of parameters
  • Gauss-Hermite and Gauss-Laguerre quadrature
  • Integration by simulation - up to five simulation variables
  • Maximum simulated likelihood estimation
  • Weights
  • Subfunctions and recursive function definition
  • Optional derivatives

Example: Heterogeneity in a Probit Model

To illustrate this tool, we consider a fairly intricate example. To the standard example of creating the user’s own probit estimator, we add an additional aspect, a likelihood function in which there is heterogeneity which is integrated out of the conditional density. Consider a probit model in which there is normally distributed, unobserved individual heterogeneity which multiplies one of the variables in the model,

y = 0 or 1
Prob(y=1|x,v) = F(b’x + cxv)

where v is standard normally distributed. The probability that enters the log-likelihood is Prob[y = j] = Ev [Prob[ y = j | v ]], j = 0,1. The expectation is exactly equal to

P(y=j|x) = sqr(1/2pi) * Integral(-inf,inf)exp(-.5*v^2)F[(2j-1)(b’x + cxv)dv

The following commands estimate this model. The command uses Hermite quadrature to do the integration. Estimates using an artificial data set are shown.

SAMPLE
CREATE

CREATE

NAMELIST
CALC
PROBIT
MAXIMIZE
; 1 - 200
; z1
; v
; y
; q
; z
; kz
; Lhs
; Hrq
; Fcn
; Start
; Labels
; Output
$
= Rnn(0,1) ; z2 = Rnn(0,1)
= Rnn(0,1) ; x = Rnu(-.5,.5) $
= ( .2 + z1 + z2 + v*x + Rnn(0,1) ) > 0
= 2*y - 1 $
= one,z1,z2 $
= Col(z) $
= y ; Rhs = z $ Starting values
= u ; Hpt = 20
= Prob = Ntg (1/Sqr(pi) * Phi (q*(b1'z + t*u*x ))) | log(Prob)
= b,.1
= kz_b,t
= 3$

The following output results. (The probit results for the starting values are omitted.)

Nonlinear Estimation of Model Parameters
Method=BFGS  ; Maximum iterations=100
Convergence criteria:
gtHg   .1000D-05 chg.F   .0000D+00 max|dB|   .0000D+00
Nodes for quadrature: Laguerre=40;Hermite=20.
Replications for GHK simulator=  10
Start values:   .33655D+00   .94078D+00   .90944D+00   .10000D+00
1st derivs.    -.26366D-02  -.41864D-02  -.97865D-02  -.22068D-01
Parameters:     .33655D+00   .94078D+00   .90944D+00   .10000D+00
Itr  1 F=  .8546D+02 gtHg=  .2464D-01 chg.F=  .8546D+02 
1st derivs.     .24435D-02  -.57443D-02   .53724D-01  -.23025D-01
Parameters:     .33690D+00   .94133D+00   .91072D+00   .10290D+00
Itr  2 F=  .8546D+02 gtHg=  .5878D-01 chg.F=  .3983D-04
1st derivs.     .87298D-02   .21673D-01  -.79062D-02  -.22928D-01
Parameters:     .33685D+00   .94143D+00   .90976D+00   .10331D+00
(some iterations omitted)
Itr 12 F=  .8503D+02 gtHg=  .2383D-05 chg.F=  .1082D-07
1st derivs.    -.11755D-05   .64422D-06  -.72063D-06   .71775D-07
Parameters:     .42442D+00   .11058D+01   .11226D+01   .34385D+01
Itr 13 F=  .8503D+02 gtHg=  .2052D-06 chg.F=  .3283D-11
                        * Converged
Function=  .85459845816D+02, at entry,  .85030583389D+02 at exit
**************************************************
* Hermite  quadrature with  20 nodes (points)    *
**************************************************
+---------------------------------------------+
| User Defined Optimization                   |
| Dependent variable             Function     |
| Number of observations              200     |
| Iterations completed                 13     |
| Log likelihood function        85.03058     |
| Chi-squared                    170.0612     |
| Degrees of freedom                    4     |
| Significance level             .0000000     |
+---------------------------------------------+
+---------+--------------+----------------+--------+---------+
|Variable | Coefficient  | Standard Error |b/St.Er.|P[|Z|>z] |
+---------+--------------+----------------+--------+---------+
 B1           .4244209422       .20178167    2.103   .0354
 B2           1.105771985       .29090246    3.801   .0001
 B3           1.122614759       .34435892    3.260   .0011
 T            3.438534630       2.8813847    1.193   .2327