Utilities and Helpers
This page aggregates functionality from the Types, Parsing, and Tools submodules (src/SubModules/Types/, src/SubModules/Parsing/, src/SubModules/Tools/) and the pre-processing routines (src/Preprocessor/).
This page documents internal utilities, types, and helper functions used across the package. Functions are grouped by their mathematical or functional domain rather than alphabetically.
Pre-processing
Pre-processing prepares user-supplied model inputs and steady-state specifications for the solver and linearization stages. It stitches together template files with user .mod content, validates equation-variable consistency, and writes generated functions used during solution and estimation.
- Purpose: Convert high-level aggregate model and steady-state definitions into executable Julia functions under
bld_example/Preprocessor/generated_fcns/. - Inputs: Paths and names set by the example model (e.g.,
Model/input_aggregate_model.mod,Model/input_aggregate_names.jl,Model/input_aggregate_steady_state.mod). - Outputs: Generated files such as
FSYS_agg_generated.jlandprepare_linearization_generated.jl, plus diagnostics printed to the console.
Workflow Overview
- Initialization and paths: Creates output folders (e.g.,
bld_example/Preprocessor/generated_fcns) and loads template files fromsrc/Preprocessor/template_fcns/. - Aggregate model generation:
- Reads
input_aggregate_model.modand injects its content into theFSYS_agg.jltemplate at the special marker. - Supports “replication” via a magic comment
@Rto instantiate multiple sectors/economies by symbol substitution. - Writes
FSYS_agg_generated.jlwith a header indicating auto-generation.
- Reads
- Consistency checks:
- Loads variable names from
input_aggregate_names.jl. - Parses
FSYS_agg_generated.jlto identify variables that actually appear in error equations (F[indexes.<var>]). - Reports missing equations or variables and warns if the number of equations does not match expectations.
- Ensures all household problem inputs (
args_hh_prob_names) are included among aggregate names.
- Loads variable names from
- Steady-state preparation:
- Reads
input_aggregate_steady_state.modand injects intoprepare_linearization.jltemplate at its marker. - Sets
n_par.n_agg_eqnfrom counted equations to keep solver dimensions consistent. - Supports the same
@Rreplication logic for multi-sector steady-state definitions. - Writes
prepare_linearization_generated.jlto be used by downstream linearization routines.
- Reads
- Finalization: Closes files and prints “Preprocessing Inputs… Done.” on success.
Replication with @R
When @R is present in a .mod file, pre-processing will:
- Read the replication symbol and number (e.g.,
@R S 3), - Replace occurrences of the symbol with
"", "2", "3"across sectors/economies, - Insert sector demarcation comments to improve readability in generated files.
Diagnostics and Common Issues
- Missing error equations: The pre-processor checks that every aggregate variable has a corresponding model equation; otherwise it prints the missing names.
- Mismatched counts: If the number of variables does not equal the number of equations (excluding distributional names), a warning highlights inconsistencies and the offending sets.
- Household inputs: If a variable listed as input to the household problem is absent from aggregate names, pre-processing errors out.
Generated Artifacts
FSYS_agg_generated.jl: Aggregate model equations assembled from the template and user inputs.prepare_linearization_generated.jl: Steady-state and linearization preparation code, including dimension settings.
Tools
The Tools submodule provides core numerical building blocks used throughout calibration, solution, and post-estimation. Below are the most crucial ones with short explanations and references to their API docs.
Discretization and Interpolation
Tauchen: Discretizes a Gaussian AR(1) process into a Markov chain with equi-probability bins. Bounds are picked from the Normal quantiles and transition probabilities are integrated with Gauss–Chebyshev nodes; used for income or productivity shocks.ExTransition: Alternative discretizer that keeps user-provided bounds and scales variance viariskscale; relies on Gauss–Chebyshev quadrature and is what the steady state/perturbation code calls when constructing income-transition matrices.myinterpolate3: Fast trilinear interpolation on structured grids; supports performance-critical lookups in HANK state spaces.
BASEforHANK.Tools.Tauchen — Function
Tauchen(rho::Float64, N::Int; sigma::Float64 = 1.0, mue::Float64 = 0.0)Generate a discrete approximation to an AR(1) process, following Tauchen (1987).
Uses importance sampling: each bin has probability 1/N to realize.
Authored by Christian Bayer, Uni Bonn, 03.05.2010.
Arguments
rho::Float64: autocorrelation coefficientN::Int: number of gridpointssigma::Float64: long-run variancemue::Float64: mean of the AR(1) process
Returns
grid_vec::Vector: state vector grid of dimensionNP::Array{Float64, 2}: transition matrix of dimensionN x Nbounds::Vector: bin bounds of dimensionN + 1#, transtype::Symbol = :importance)
BASEforHANK.Tools.ExTransition — Function
ExTransition(rho::Number, bounds::Array{Float64, 1}, riskscale::Number)Calculates the transition probability matrix for a discretized state space model, similar to the Tauchen method, using importance sampling and numerical integration with Gauss-Chebyshev nodes.
Arguments
rho::Number: The autoregressive parameter of the process.bounds::Array{Float64, 1}: A 1D array specifying the grid boundaries of the state space.riskscale::Number: A scaling factor that adjusts the short-run risk (variance) in the model.
Returns
P::Array{Float64, 2}: The transition probability matrix, whereP[i, j]represents the probability of transitioning from stateito statej.
Notes
- The grid
boundsmust be sorted in ascending order. #similar to TAUCHEN
BASEforHANK.Tools.myinterpolate3 — Function
myinterpolate3(
xgrd1::AbstractVector,
xgrd2::AbstractVector,
xgrd3::AbstractVector,
ygrd::AbstractArray,
model::AbstractModel,
xeval1::AbstractVector,
xeval2::AbstractVector,
xeval3::AbstractVector)
Trilinearly project ygrd on (xgrd1, xgrd2, xgrd3) and use it to interpolate values at (xeval1, xeval2, xeval3).
Arguments
xgrd1,xgrd2,xgrd3::AbstractVector: Grid points along each of the three dimensions.ygrd::AbstractArray: Array of function values evaluated at the grid points defined by (xgrd1,xgrd2,xgrd3).model::AbstractModel: Model object, which can be of typeOneAsset,TwoAsset, orCompleteMarkets.xeval1,xeval2,xeval3::AbstractVector: Points at which the interpolation is performed along each dimension.
Returns
- An array of interpolated values at the specified evaluation points (
xeval1,xeval2,xeval3).
Notes
- It assumes that
xgrd1,xgrd2, andxgrd3are sorted.
Root Finding and Solvers
CustomBrent: Robust 1D root finder (Brent’s method) with package-specific defaults.broyden: Quasi-Newton method for multi-dimensional fixed-point problems; effective for equilibrium conditions.Fastroot: Optimized wrapper for vector root solving in common model setups.
BASEforHANK.Tools.CustomBrent — Function
CustomBrent(f::Function, a::Real, b::Real; tol = 1e-14)Find the root of a function using a customized version of Brent's method. This implementation adapts Brent's method by incorporating initial guesses for value functions and distributions, based on linear interpolations from the previous two iterations.
The function is designed for cases where the root-finding process needs to account for additional parameters beyond the function's value at the endpoints.
Arguments
f::Function: The function for which the root is being found. It should return a tuple, where the first element (f(z)[1]) represents the function value atz, and the subsequent elements represent additional parameters used for interpolation.a::Real: The lower bound of the interval where the root is located.b::Real: The upper bound of the interval where the root is located.tol::Real: The tolerance for convergence. The default is1e-14.
Returns
b::Real: The estimated root of the function.iter::Int: The number of iterations used to find the root.fb::Tuple: The function value at the root, along with any interpolated parameters.
Errors
- Throws an error if the function values at
aandbhave the same sign, indicating no root exists in the specified interval. # Implementation of Brent's method to find a root of a function (as on wikipedia)
BASEforHANK.Tools.broyden — Function
broyden(f, x0, critF, critX, maxiter, tau, tauS)Solve for the root of a function f using the "good" Broyden algorithm, an iterative method to approximate the root of a nonlinear system of equations. This method is an approximation to Newton's method that uses updates to the Jacobian, improving efficiency in solving for the root.
Arguments
f::Function: The function for which the root is being found. It should return a vector of values.x0::AbstractVector: The starting guess for the root, provided as a column vector.critF::Real: The precision required for the function values to converge.critX::Real: The precision required for the change inxto converge.maxiter::Int: The maximum number of iterations allowed for the algorithm.tau::Real: A scaling factor used to adjust the update step.tauS::Real: A scaling factor for adjusting the step size dynamically with each iteration.
Returns
xstar::AbstractVector: The estimated root of the function.fval::AbstractVector: The value of the function at the root.iter::Int: The number of iterations performed.distF::AbstractVector: The distance from zero at each iteration, representing the convergence of the function.
BASEforHANK.Tools.Fastroot — Function
Fastroot(xgrid::Vector, fx::AbstractArray)Used for fast root finding using linear interpolation and a simplified Newton step.
This function performs a fast root-finding algorithm on a given grid (xgrid) and function values (fx). It applies a single Newton step at the largest negative function value.
Arguments
xgrid::Vector: The grid of x values.fx::AbstractArray: Function values evaluated at the points inxgrid. The array should be reshaped to match the dimensions ofxgrid.
Returns
roots::Vector: A vector of roots found through interpolation for each column offx. The length of the vector corresponds to the number of columns infx.
Linear Algebra Helpers
real_schur: Wrapper for real Schur decomposition useful in linear solution routines.stepwiseRSKron/stepwiseLSKron: Structured right/left stepwise Kronecker products to accelerate large linear operations.
BASEforHANK.Tools.real_schur — Function
real_schur(A, B)Computes the real generalized Schur decomposition of the matrix pair (A, B). Selects the eigenvalues related to the state variables.
Arguments
A: A square matrix.B: A square matrix of the same size asA.
Returns
F: The Schur decomposition of(A, B), return ofLinearAlgebra.schur.select_ev: A boolean vector indicating which eigenvalues have a modulus greater than or equal to 1, i.e. which eigenvalues and/or eigenvectors are related to the state variables.nk: The number of eigenvalues with a modulus greater than or equal to 1, i.e. the number of state variables.λ: The vector of all generalized eigenvalues.
BASEforHANK.Tools.stepwiseRSKron — Function
stepwiseRSKron(M,A,B)Calculate M*kron(A,B) stepwise, without explicit computation of kron(A,B). Can use that B,M is sparse.
BASEforHANK.Tools.stepwiseLSKron — Function
stepwiseLSKron(M,A,B)Calculate kron(A,B)*M stepwise, without explicit computation of kron(A,B). Can use that B,M is sparse.
Sylvester / Doubling Solvers
doublingGxx: Main doubling algorithm for generalized Sylvester equations; used in perturbation and post-estimation to recover higher-order solution tensors.doublingSimple: Lightweight variant for linear Sylvester equations of the formA*X + X*Fkron = -B; helpful in unconditional moment calculations.
BASEforHANK.Tools.doublingGxx — Function
doublingGxx(iAB, hx, iAD; tol=1e-14, maxit=20)Solve a Sylvester-like equation using the doubling algorithm (Kim–Kim–Sims, JEDC 2008). Computes X satisfying X = reshape(M*hx, size(iAD)) + X_old with powers of hx and iAB. Returns the solution matrix X. Prints convergence diagnostics; warns if not converged.
BASEforHANK.Tools.doublingSimple — Function
doublingSimple(A, Fkron, B; tol=eps(), maxit=20)Simple doubling iteration X = X_old + A^p * X_old * Fkron^p to solve A*X*Fkron + B = 0. Returns X; prints iteration divergence and warns on non-convergence.
Probability and Transforms
pdf_to_cdf/cdf_to_pdf: Numerical transforms between density and distribution functions; handy for distributional IRFs and diagnostics.
BASEforHANK.Tools.pdf_to_cdf — Function
pdf_to_cdf(pdf)Computes the cumulative distribution function (CDF) from a given probability density function (PDF) by taking cumulative sums along all dimensions.
Arguments
pdf: An array representing the probability density function. Can be one-dimensional or higher-dimensional.
Returns
Y: An array of the same size aspdf, representing the computed cumulative distribution function.
BASEforHANK.Tools.cdf_to_pdf — Function
cdf_to_pdf(cdf)Computes the probability density function (PDF) from a given cumulative distribution function (CDF) by taking finite differences along all dimensions.
Arguments
cdf: An array representing the cumulative distribution function. Can be one-dimensional or higher-dimensional.
Returns
Y: An array of the same size ascdf, representing the computed probability density function.
Notes
Allows for one-asset and two-asset case by only taking differences where there are at least two elements.
Derivatives and Sensitivity
centralderiv: Central finite-difference derivatives for scalar/vector functions; useful when auto-diff is not applicable.
BASEforHANK.Tools.centralderiv — Function
centralderiv(y, x, dims)Calculate the approximate derivative of the array y with respect to the array x along the specified dimension(s) dims using central differences (cf. BASEforHANK.Tools.centraldiff()).
Arguments
y::AbstractArray: The array for which the derivative is computed.x::AbstractArray: The array with respect to which the derivative ofyis calculated.dims::Union{Int, Tuple{Vararg{Int}}}: The dimension(s) along which to calculate the derivative.
Returns
AbstractArray: The approximate derivative ofywith respect tox.
BASEforHANK.Tools.centraldiff — Function
centraldiff(x)Calculate the central difference of a vector x. The central difference is a numerical approximation to the derivative of a function, computed using the values of the function at adjacent points. The first and last elements of the vector are computed using forward and backward differences, respectively, because a central difference cannot be computed at the boundaries.
Arguments
x::AbstractVector: The input vector for which the central difference is computed.
Returns
a::AbstractVector: The central difference approximation of the input vectorx. The output vector has the same length asx, with the first and last elements computed using forward and backward differences, respectively.
Errors:
- Throws an error if the input x is not a 1-dimensional vector.
Structure ↔ Vector Mapping
struc_to_vec/vec_to_struct: Flatten and reconstruct parameter/variable structs for optimizers and estimators.
BASEforHANK.Tools.struc_to_vec — Function
struc_to_vec(s::S) -> Vector{Any}Return a Vector{Any} with one entry per field of s. Works when fields have different concrete types (vectors, matrices, scalars). Efficiently preallocates the result and preserves each field's value.
BASEforHANK.Tools.vec_to_struct — Function
vec_to_struct(::Type{S}, vals::AbstractVector) -> SInverse of struc_to_vec / struc_to_vec_same_type. Construct an instance of S by splatting the elements of vals into S's constructor. Asserts that the number of entries in vals matches the number of fields of S.
Logging and Quiet Execution
quiet_call,@silent,unmute_println,unmute_printf: Control logging and stdout/stderr for clean console output or benchmarking without noise.
BASEforHANK.Tools.quiet_call — Function
quiet_call(f, args...; kwargs...)Run f(args...; kwargs...) while suppressing all text written to Base.stdout and Base.stderr by redirecting them to the system null device. The function returns the value produced by f (or propagates any exception thrown by f).
Arguments
f: A callable to invoke.args...: Positional arguments forwarded tof.kwargs...: Keyword arguments forwarded tof.
Returns
- Whatever
freturns.
Notes
- Only suppresses writes to
Base.stdoutandBase.stderr(i.e., low-level printing). It does not change the active logger; logging macros that use the global logger may still emit output unless the logger itself is changed. - Implementation uses the OS null device path (
"/dev/null"), so behavior is platform-dependent (on Windows, use"NUL"instead). - Any exceptions raised by
fpropagate out ofquiet_call; the redirections are properly restored when the call exits.
BASEforHANK.Tools.@silent — Macro
@silent exprEvaluate expr while silencing both logging macros (by using a Logging.NullLogger()) and any writes to Base.stdout/Base.stderr (by redirecting them to devnull). The macro returns the value of expr.
Usage
@silent some_expression— runssome_expressionwith logging and standard output/stderr suppressed.
Returns
- The result of evaluating
expr.
Notes
- Suppresses messages produced by logging macros such as
@info,@warn, etc., by temporarily installing aLogging.NullLogger(). - Also redirects
Base.stdoutandBase.stderrtodevnull, suppressing plainprint/printlnoutput. - Evaluation happens in the caller's scope (macro hygiene preserved where appropriate); side effects of
exprstill occur, except for emitted text. - The suppression is lexically limited to the execution of
expr; the original logger and standard streams are restored afterwards. - Exceptions raised by
exprare propagated; suppression is still undone on exit.
BASEforHANK.Tools.unmute_println — Function
unmute_println(xs...)Print the given arguments to the original stdout (Tools.ORIG_STDOUT), bypassing any redirection of Base.stdout that may be in effect.
Arguments
xs...: Zero or more values to print; forwarded toprintln.
Returns
nothing. Side effect: writes a newline-terminated line toTools.ORIG_STDOUT.
Notes
- This is useful when output has been redirected (e.g., via
redirect_stdout) but you still want to emit a message to the original terminal or log sink that was captured inTools.ORIG_STDOUT. - Does not change the current stdout; only writes directly to
Tools.ORIG_STDOUT.
BASEforHANK.Tools.unmute_printf — Function
unmute_printf(fmt::AbstractString, args...)Write a formatted string to the original stdout (Tools.ORIG_STDOUT) using Printf-style formatting, bypassing any redirection of Base.stdout.
Arguments
fmt::AbstractString: APrintf-style format string.args...: Values to be formatted according tofmt.
Returns
nothing. Side effect: writes the formatted output (without adding an extra newline) toTools.ORIG_STDOUT.
Notes
- Equivalent in purpose to
@printforprintbut ensures the output goes to the preserved original stdout regardless of current redirection. - Formatting is performed via
Printf.Format(fmt)semantics; invalid format strings or argument mismatches propagate the usualPrintferrors.
Types
The Types submodule provides the abstract type hierarchy used to describe models, state representations, and indexing across the package. Most structures in Parsing/Structs inherit some type. Using types allows us to establish interfaces and dispatch points (see Multiple Dispatch, an alternative to object-oriented programming in terms of structuring code, that is better suited for the Julia programming language).
- Model families:
AbstractMacroModelwith typesOneAsset,TwoAsset, andCompleteMarketsdefine the degree of market incompleteness (market completeness vs uninsured idiosynratic risk vs additional trading friction). They determine many downstream choices (state dimensionality, policy/value structures, and solution routines). - Transition of distribution:
TransitionTypewithNonLinearTransitionandLinearTransitionmarks whether using the Young/lottery-method for modelling the transition of the distribution, which is linear in optimal policies, or the DEGM-method, which allows for nonlinearities and is thus the better choice for higher-order solutions. - Functions and matrices: Abstract containers
PolicyFunctions,ValueFunctions, andTransitionMatricesparameterized by array types define interfaces for storing objects like grids of policy/value functions or Markov/state transition matrices. - Distribution representation:
DistributionValuesstores the type of how the distribution is represented, includingCopulaOneAsset,CopulaTwoAssets,CDFandRepAgent. - Distribution states:
DistributionStateTypewithCDFStatesandCopulaStatesspecifies whether distributions are stored in joint, cumulative form or with Copula and marginals. - Indexing:
Indexes,DistributionIndexes, andValueFunctionIndexesunify index handling for variables, equations, and grids to avoid hard-coded positions and to enable robust parsing/preprocessing. - Transformations:
Transformationsis an abstract anchor for elements of function transformations, for example matrices of the Discrete Cosine Transformation.
These types are used throughout the methods in Parsing, Preprocessor, PerturbationSolution, Estimation, and PostEstimation so that code remains generic across model classes and distribution/state encodings.
Parsing
The Parsing submodule translates user and example model inputs into internal, typed representations and auto-generates boilerplate code used throughout the package. It provides:
- Typed containers: parameter sets, results structs, and index structs to reference variables, shocks, distributions, and function grids without hard-coding positions.
- Name registries:
state_names,control_names,shock_names,aggr_names, and their ASCII variants to ensure consistent mapping from.modcontent to code. - Prior/settings helpers:
priorbuilds priors;EstimationSettingsand the conveniencee_setencapsulate estimation-specific knobs. - Macros for codegen: generate functions and structs for indices and equations from compact specifications, keeping model authorship ergonomic.
Workflow
- Documentation mode: If
pathsis not defined, it automatically includes the baseline example inputs to make docs and examples self-contained. - Struct and index generation:
@make_structand@make_struct_aggrcreate index structs;@make_fnand@make_fnaggrgenerate functions likeproduce_indexesthat map names → positions. - Macro utilities:
@generate_equationsexpands compact model equations; argument macros like@write_args_hh_prob*and@read_args_hh_prob*manage household problem I/O signatures. - Priors and settings: Includes
prior.jl, sets upEstimationSettings, and providese_setconfigured with availableshock_names.
Key macros and helpers
BASEforHANK.Parsing.@writeXSS — Macro
@writeXSS()Write all single steady state variables into vectors XSS / XSSaggr.
Requires
(module) globals state_names, control_names, aggr_names
BASEforHANK.Parsing.@make_fn — Macro
@make_fn(fn_name)Create function fn_name that returns an instance of IndexStruct (created by @make_struct), mapping states and controls to indexes inferred from numerical parameters and compression indexes.
Requires
(module) global state_names, control_names
BASEforHANK.Parsing.@make_fnaggr — Macro
@make_fnaggr(fn_name)Create function fn_name that returns an instance of IndexStructAggr (created by @make_struct_aggr), mapping aggregate states and controls to values 1 to length(aggr_names) (both steady state and deviation from it).
Requires
(module) global aggr_names
BASEforHANK.Parsing.@make_struct — Macro
@make_struct(struct_name)Generate a struct named struct_name that contains index fields for state and control variables and associated distribution/value-function indexes. For each state_names element the macro creates <name>SS and <name> integer fields; for each control_names element it creates <name>SS and <name> integer fields. The resulting struct also includes fields distrSS, valueFunctionSS, distr, and valueFunction to hold index containers.
Notes
- Relies on module-level
state_namesandcontrol_namesvariables.
BASEforHANK.Parsing.@make_struct_aggr — Macro
@make_struct_aggr(struct_name)Generate a lightweight struct named struct_name that contains integer index fields for each name in the global aggr_names vector. For every aggregate name this macro creates two integer fields: one named <name>SS for the steady-state index and one named <name> for the deviation index.
Notes
- Relies on a module-level
aggr_namesvariable (array of names).
BASEforHANK.Parsing.@generate_equations — Macro
@generate_equations()Write out the expansions around steady state for all variables in aggr_names, i.e. generate code that reads aggregate states/controls from steady state deviations.
Equations take the form of (with variable r as example):
r = exp.(XSS[indexes.rSS] .+ X[indexes.r])rPrime = exp.(XSS[indexes.rSS] .+ XPrime[indexes.r])
Requires
(module) global aggr_names
BASEforHANK.Parsing.@write_args_hh_prob_ss — Macro
@write_args_hh_prob_ss()See @write_args_hh_prob, however, for the case where the variables are called with a suffix "SS".
Requires
(module) global args_hh_prob_names
BASEforHANK.Parsing.@write_args_hh_prob — Macro
@write_args_hh_prob()Write all variables defined in the global string vector args_hh_prob_names into a vector args_hh_prob based on the variables with the according names in local scope. The type of the vector is inferred from the first variable in args_hh_prob_names.
Requires
(module) global args_hh_prob_names
BASEforHANK.Parsing.@read_args_hh_prob — Macro
@read_args_hh_prob()Read all variables defined in the global string vector args_hh_prob_names into local scope based on the variables with the according names in the vector args_hh_prob.
Requires
(module) global args_hh_prob_names, args_hh_prob
BASEforHANK.Parsing.@read_args_hh_prob_ss — Macro
@read_args_hh_prob_ss()See @read_args_hh_prob, however, for the case where the variables are called with a suffix "SS".
Requires
(module) global args_hh_prob_names, args_hh_prob
BASEforHANK.Parsing.@asset_vars — Macro
@asset_vars(model_type)Generate asset variable tuple based on model type, only accessing variables that exist.
Additional core entries (overview):
ModelParameters,NumericalParameters: Core parameter containers referenced across modules.EstimationSettings,e_set: Holds estimation control knobs (proposal scales, ME caps, horizons, etc.).IndexStruct,IndexStructAggr: Programmatic indices that map named objects to positions in vectors/matrices.produce_indexes,produce_indexes_aggr: Functions generated by macros to build index maps from name lists.prior: Constructs distributions for parameter priors consistent withModelParameters.- Name lists:
state_names,control_names,shock_names,aggr_names,distr_namesused in parsing and preprocessing.