Utilities and Helpers

Note

This page aggregates functionality from the Types, Parsing, and Tools submodules (src/SubModules/Types/, src/SubModules/Parsing/, src/SubModules/Tools/) and the pre-processing routines (src/Preprocessor/).

This page documents internal utilities, types, and helper functions used across the package. Functions are grouped by their mathematical or functional domain rather than alphabetically.

Pre-processing

Pre-processing prepares user-supplied model inputs and steady-state specifications for the solver and linearization stages. It stitches together template files with user .mod content, validates equation-variable consistency, and writes generated functions used during solution and estimation.

  • Purpose: Convert high-level aggregate model and steady-state definitions into executable Julia functions under bld_example/Preprocessor/generated_fcns/.
  • Inputs: Paths and names set by the example model (e.g., Model/input_aggregate_model.mod, Model/input_aggregate_names.jl, Model/input_aggregate_steady_state.mod).
  • Outputs: Generated files such as FSYS_agg_generated.jl and prepare_linearization_generated.jl, plus diagnostics printed to the console.

Workflow Overview

  1. Initialization and paths: Creates output folders (e.g., bld_example/Preprocessor/generated_fcns) and loads template files from src/Preprocessor/template_fcns/.
  2. Aggregate model generation:
    • Reads input_aggregate_model.mod and injects its content into the FSYS_agg.jl template at the special marker.
    • Supports “replication” via a magic comment @R to instantiate multiple sectors/economies by symbol substitution.
    • Writes FSYS_agg_generated.jl with a header indicating auto-generation.
  3. Consistency checks:
    • Loads variable names from input_aggregate_names.jl.
    • Parses FSYS_agg_generated.jl to identify variables that actually appear in error equations (F[indexes.<var>]).
    • Reports missing equations or variables and warns if the number of equations does not match expectations.
    • Ensures all household problem inputs (args_hh_prob_names) are included among aggregate names.
  4. Steady-state preparation:
    • Reads input_aggregate_steady_state.mod and injects into prepare_linearization.jl template at its marker.
    • Sets n_par.n_agg_eqn from counted equations to keep solver dimensions consistent.
    • Supports the same @R replication logic for multi-sector steady-state definitions.
    • Writes prepare_linearization_generated.jl to be used by downstream linearization routines.
  5. Finalization: Closes files and prints “Preprocessing Inputs… Done.” on success.

Replication with @R

When @R is present in a .mod file, pre-processing will:

  • Read the replication symbol and number (e.g., @R S 3),
  • Replace occurrences of the symbol with "", "2", "3" across sectors/economies,
  • Insert sector demarcation comments to improve readability in generated files.

Diagnostics and Common Issues

  • Missing error equations: The pre-processor checks that every aggregate variable has a corresponding model equation; otherwise it prints the missing names.
  • Mismatched counts: If the number of variables does not equal the number of equations (excluding distributional names), a warning highlights inconsistencies and the offending sets.
  • Household inputs: If a variable listed as input to the household problem is absent from aggregate names, pre-processing errors out.

Generated Artifacts

  • FSYS_agg_generated.jl: Aggregate model equations assembled from the template and user inputs.
  • prepare_linearization_generated.jl: Steady-state and linearization preparation code, including dimension settings.

Tools

The Tools submodule provides core numerical building blocks used throughout calibration, solution, and post-estimation. Below are the most crucial ones with short explanations and references to their API docs.

Discretization and Interpolation

  • Tauchen: Discretizes a Gaussian AR(1) process into a Markov chain with equi-probability bins. Bounds are picked from the Normal quantiles and transition probabilities are integrated with Gauss–Chebyshev nodes; used for income or productivity shocks.
  • ExTransition: Alternative discretizer that keeps user-provided bounds and scales variance via riskscale; relies on Gauss–Chebyshev quadrature and is what the steady state/perturbation code calls when constructing income-transition matrices.
  • myinterpolate3: Fast trilinear interpolation on structured grids; supports performance-critical lookups in HANK state spaces.
BASEforHANK.Tools.TauchenFunction
Tauchen(rho::Float64, N::Int; sigma::Float64 = 1.0, mue::Float64 = 0.0)

Generate a discrete approximation to an AR(1) process, following Tauchen (1987).

Uses importance sampling: each bin has probability 1/N to realize.

Authored by Christian Bayer, Uni Bonn, 03.05.2010.

Arguments

  • rho::Float64: autocorrelation coefficient
  • N::Int: number of gridpoints
  • sigma::Float64: long-run variance
  • mue::Float64: mean of the AR(1) process

Returns

  • grid_vec::Vector: state vector grid of dimension N
  • P::Array{Float64, 2}: transition matrix of dimension N x N
  • bounds::Vector: bin bounds of dimension N + 1 #, transtype::Symbol = :importance)
source
BASEforHANK.Tools.ExTransitionFunction
ExTransition(rho::Number, bounds::Array{Float64, 1}, riskscale::Number)

Calculates the transition probability matrix for a discretized state space model, similar to the Tauchen method, using importance sampling and numerical integration with Gauss-Chebyshev nodes.

Arguments

  • rho::Number: The autoregressive parameter of the process.
  • bounds::Array{Float64, 1}: A 1D array specifying the grid boundaries of the state space.
  • riskscale::Number: A scaling factor that adjusts the short-run risk (variance) in the model.

Returns

  • P::Array{Float64, 2}: The transition probability matrix, where P[i, j] represents the probability of transitioning from state i to state j.

Notes

  • The grid bounds must be sorted in ascending order. #similar to TAUCHEN
source
BASEforHANK.Tools.myinterpolate3Function
myinterpolate3(
    xgrd1::AbstractVector,
    xgrd2::AbstractVector,
    xgrd3::AbstractVector,
    ygrd::AbstractArray,
    model::AbstractModel,
    xeval1::AbstractVector,
    xeval2::AbstractVector,
    xeval3::AbstractVector

)

Trilinearly project ygrd on (xgrd1, xgrd2, xgrd3) and use it to interpolate values at (xeval1, xeval2, xeval3).

Arguments

  • xgrd1, xgrd2, xgrd3::AbstractVector: Grid points along each of the three dimensions.
  • ygrd::AbstractArray: Array of function values evaluated at the grid points defined by (xgrd1, xgrd2, xgrd3).
  • model::AbstractModel: Model object, which can be of type OneAsset, TwoAsset, or CompleteMarkets.
  • xeval1, xeval2, xeval3::AbstractVector: Points at which the interpolation is performed along each dimension.

Returns

  • An array of interpolated values at the specified evaluation points (xeval1, xeval2, xeval3).

Notes

  • It assumes that xgrd1, xgrd2, and xgrd3 are sorted.
source

Root Finding and Solvers

  • CustomBrent: Robust 1D root finder (Brent’s method) with package-specific defaults.
  • broyden: Quasi-Newton method for multi-dimensional fixed-point problems; effective for equilibrium conditions.
  • Fastroot: Optimized wrapper for vector root solving in common model setups.
BASEforHANK.Tools.CustomBrentFunction
CustomBrent(f::Function, a::Real, b::Real; tol = 1e-14)

Find the root of a function using a customized version of Brent's method. This implementation adapts Brent's method by incorporating initial guesses for value functions and distributions, based on linear interpolations from the previous two iterations.

The function is designed for cases where the root-finding process needs to account for additional parameters beyond the function's value at the endpoints.

Arguments

  • f::Function: The function for which the root is being found. It should return a tuple, where the first element (f(z)[1]) represents the function value at z, and the subsequent elements represent additional parameters used for interpolation.
  • a::Real: The lower bound of the interval where the root is located.
  • b::Real: The upper bound of the interval where the root is located.
  • tol::Real: The tolerance for convergence. The default is 1e-14.

Returns

  • b::Real: The estimated root of the function.
  • iter::Int: The number of iterations used to find the root.
  • fb::Tuple: The function value at the root, along with any interpolated parameters.

Errors

  • Throws an error if the function values at a and b have the same sign, indicating no root exists in the specified interval. # Implementation of Brent's method to find a root of a function (as on wikipedia)
source
BASEforHANK.Tools.broydenFunction
broyden(f, x0, critF, critX, maxiter, tau, tauS)

Solve for the root of a function f using the "good" Broyden algorithm, an iterative method to approximate the root of a nonlinear system of equations. This method is an approximation to Newton's method that uses updates to the Jacobian, improving efficiency in solving for the root.

Arguments

  • f::Function: The function for which the root is being found. It should return a vector of values.
  • x0::AbstractVector: The starting guess for the root, provided as a column vector.
  • critF::Real: The precision required for the function values to converge.
  • critX::Real: The precision required for the change in x to converge.
  • maxiter::Int: The maximum number of iterations allowed for the algorithm.
  • tau::Real: A scaling factor used to adjust the update step.
  • tauS::Real: A scaling factor for adjusting the step size dynamically with each iteration.

Returns

  • xstar::AbstractVector: The estimated root of the function.
  • fval::AbstractVector: The value of the function at the root.
  • iter::Int: The number of iterations performed.
  • distF::AbstractVector: The distance from zero at each iteration, representing the convergence of the function.
source
BASEforHANK.Tools.FastrootFunction
Fastroot(xgrid::Vector, fx::AbstractArray)

Used for fast root finding using linear interpolation and a simplified Newton step.

This function performs a fast root-finding algorithm on a given grid (xgrid) and function values (fx). It applies a single Newton step at the largest negative function value.

Arguments

  • xgrid::Vector: The grid of x values.
  • fx::AbstractArray: Function values evaluated at the points in xgrid. The array should be reshaped to match the dimensions of xgrid.

Returns

  • roots::Vector: A vector of roots found through interpolation for each column of fx. The length of the vector corresponds to the number of columns in fx.
source

Linear Algebra Helpers

  • real_schur: Wrapper for real Schur decomposition useful in linear solution routines.
  • stepwiseRSKron / stepwiseLSKron: Structured right/left stepwise Kronecker products to accelerate large linear operations.
BASEforHANK.Tools.real_schurFunction
real_schur(A, B)

Computes the real generalized Schur decomposition of the matrix pair (A, B). Selects the eigenvalues related to the state variables.

Arguments

  • A: A square matrix.
  • B: A square matrix of the same size as A.

Returns

  • F: The Schur decomposition of (A, B), return of LinearAlgebra.schur.
  • select_ev: A boolean vector indicating which eigenvalues have a modulus greater than or equal to 1, i.e. which eigenvalues and/or eigenvectors are related to the state variables.
  • nk: The number of eigenvalues with a modulus greater than or equal to 1, i.e. the number of state variables.
  • λ: The vector of all generalized eigenvalues.
source

Sylvester / Doubling Solvers

  • doublingGxx: Main doubling algorithm for generalized Sylvester equations; used in perturbation and post-estimation to recover higher-order solution tensors.
  • doublingSimple: Lightweight variant for linear Sylvester equations of the form A*X + X*Fkron = -B; helpful in unconditional moment calculations.
BASEforHANK.Tools.doublingGxxFunction
doublingGxx(iAB, hx, iAD; tol=1e-14, maxit=20)

Solve a Sylvester-like equation using the doubling algorithm (Kim–Kim–Sims, JEDC 2008). Computes X satisfying X = reshape(M*hx, size(iAD)) + X_old with powers of hx and iAB. Returns the solution matrix X. Prints convergence diagnostics; warns if not converged.

source
BASEforHANK.Tools.doublingSimpleFunction
doublingSimple(A, Fkron, B; tol=eps(), maxit=20)

Simple doubling iteration X = X_old + A^p * X_old * Fkron^p to solve A*X*Fkron + B = 0. Returns X; prints iteration divergence and warns on non-convergence.

source

Probability and Transforms

  • pdf_to_cdf / cdf_to_pdf: Numerical transforms between density and distribution functions; handy for distributional IRFs and diagnostics.
BASEforHANK.Tools.pdf_to_cdfFunction
pdf_to_cdf(pdf)

Computes the cumulative distribution function (CDF) from a given probability density function (PDF) by taking cumulative sums along all dimensions.

Arguments

  • pdf: An array representing the probability density function. Can be one-dimensional or higher-dimensional.

Returns

  • Y: An array of the same size as pdf, representing the computed cumulative distribution function.
source
BASEforHANK.Tools.cdf_to_pdfFunction
cdf_to_pdf(cdf)

Computes the probability density function (PDF) from a given cumulative distribution function (CDF) by taking finite differences along all dimensions.

Arguments

  • cdf: An array representing the cumulative distribution function. Can be one-dimensional or higher-dimensional.

Returns

  • Y: An array of the same size as cdf, representing the computed probability density function.

Notes

Allows for one-asset and two-asset case by only taking differences where there are at least two elements.

source

Derivatives and Sensitivity

  • centralderiv: Central finite-difference derivatives for scalar/vector functions; useful when auto-diff is not applicable.
BASEforHANK.Tools.centralderivFunction
centralderiv(y, x, dims)

Calculate the approximate derivative of the array y with respect to the array x along the specified dimension(s) dims using central differences (cf. BASEforHANK.Tools.centraldiff()).

Arguments

  • y::AbstractArray: The array for which the derivative is computed.
  • x::AbstractArray: The array with respect to which the derivative of y is calculated.
  • dims::Union{Int, Tuple{Vararg{Int}}}: The dimension(s) along which to calculate the derivative.

Returns

  • AbstractArray: The approximate derivative of y with respect to x.
source
BASEforHANK.Tools.centraldiffFunction
centraldiff(x)

Calculate the central difference of a vector x. The central difference is a numerical approximation to the derivative of a function, computed using the values of the function at adjacent points. The first and last elements of the vector are computed using forward and backward differences, respectively, because a central difference cannot be computed at the boundaries.

Arguments

  • x::AbstractVector: The input vector for which the central difference is computed.

Returns

  • a::AbstractVector: The central difference approximation of the input vector x. The output vector has the same length as x, with the first and last elements computed using forward and backward differences, respectively.

Errors:

  • Throws an error if the input x is not a 1-dimensional vector.
source

Structure ↔ Vector Mapping

  • struc_to_vec / vec_to_struct: Flatten and reconstruct parameter/variable structs for optimizers and estimators.
BASEforHANK.Tools.struc_to_vecFunction
struc_to_vec(s::S) -> Vector{Any}

Return a Vector{Any} with one entry per field of s. Works when fields have different concrete types (vectors, matrices, scalars). Efficiently preallocates the result and preserves each field's value.

source
BASEforHANK.Tools.vec_to_structFunction
vec_to_struct(::Type{S}, vals::AbstractVector) -> S

Inverse of struc_to_vec / struc_to_vec_same_type. Construct an instance of S by splatting the elements of vals into S's constructor. Asserts that the number of entries in vals matches the number of fields of S.

source

Logging and Quiet Execution

  • quiet_call, @silent, unmute_println, unmute_printf: Control logging and stdout/stderr for clean console output or benchmarking without noise.
BASEforHANK.Tools.quiet_callFunction
quiet_call(f, args...; kwargs...)

Run f(args...; kwargs...) while suppressing all text written to Base.stdout and Base.stderr by redirecting them to the system null device. The function returns the value produced by f (or propagates any exception thrown by f).

Arguments

  • f: A callable to invoke.
  • args...: Positional arguments forwarded to f.
  • kwargs...: Keyword arguments forwarded to f.

Returns

  • Whatever f returns.

Notes

  • Only suppresses writes to Base.stdout and Base.stderr (i.e., low-level printing). It does not change the active logger; logging macros that use the global logger may still emit output unless the logger itself is changed.
  • Implementation uses the OS null device path ("/dev/null"), so behavior is platform-dependent (on Windows, use "NUL" instead).
  • Any exceptions raised by f propagate out of quiet_call; the redirections are properly restored when the call exits.
source
BASEforHANK.Tools.@silentMacro
@silent expr

Evaluate expr while silencing both logging macros (by using a Logging.NullLogger()) and any writes to Base.stdout/Base.stderr (by redirecting them to devnull). The macro returns the value of expr.

Usage

  • @silent some_expression — runs some_expression with logging and standard output/stderr suppressed.

Returns

  • The result of evaluating expr.

Notes

  • Suppresses messages produced by logging macros such as @info, @warn, etc., by temporarily installing a Logging.NullLogger().
  • Also redirects Base.stdout and Base.stderr to devnull, suppressing plain print/println output.
  • Evaluation happens in the caller's scope (macro hygiene preserved where appropriate); side effects of expr still occur, except for emitted text.
  • The suppression is lexically limited to the execution of expr; the original logger and standard streams are restored afterwards.
  • Exceptions raised by expr are propagated; suppression is still undone on exit.
source
BASEforHANK.Tools.unmute_printlnFunction
unmute_println(xs...)

Print the given arguments to the original stdout (Tools.ORIG_STDOUT), bypassing any redirection of Base.stdout that may be in effect.

Arguments

  • xs...: Zero or more values to print; forwarded to println.

Returns

  • nothing. Side effect: writes a newline-terminated line to Tools.ORIG_STDOUT.

Notes

  • This is useful when output has been redirected (e.g., via redirect_stdout) but you still want to emit a message to the original terminal or log sink that was captured in Tools.ORIG_STDOUT.
  • Does not change the current stdout; only writes directly to Tools.ORIG_STDOUT.
source
BASEforHANK.Tools.unmute_printfFunction
unmute_printf(fmt::AbstractString, args...)

Write a formatted string to the original stdout (Tools.ORIG_STDOUT) using Printf-style formatting, bypassing any redirection of Base.stdout.

Arguments

  • fmt::AbstractString: A Printf-style format string.
  • args...: Values to be formatted according to fmt.

Returns

  • nothing. Side effect: writes the formatted output (without adding an extra newline) to Tools.ORIG_STDOUT.

Notes

  • Equivalent in purpose to @printf or print but ensures the output goes to the preserved original stdout regardless of current redirection.
  • Formatting is performed via Printf.Format(fmt) semantics; invalid format strings or argument mismatches propagate the usual Printf errors.
source

Types

The Types submodule provides the abstract type hierarchy used to describe models, state representations, and indexing across the package. Most structures in Parsing/Structs inherit some type. Using types allows us to establish interfaces and dispatch points (see Multiple Dispatch, an alternative to object-oriented programming in terms of structuring code, that is better suited for the Julia programming language).

  • Model families: AbstractMacroModel with types OneAsset, TwoAsset, and CompleteMarkets define the degree of market incompleteness (market completeness vs uninsured idiosynratic risk vs additional trading friction). They determine many downstream choices (state dimensionality, policy/value structures, and solution routines).
  • Transition of distribution: TransitionType with NonLinearTransition and LinearTransition marks whether using the Young/lottery-method for modelling the transition of the distribution, which is linear in optimal policies, or the DEGM-method, which allows for nonlinearities and is thus the better choice for higher-order solutions.
  • Functions and matrices: Abstract containers PolicyFunctions, ValueFunctions, and TransitionMatrices parameterized by array types define interfaces for storing objects like grids of policy/value functions or Markov/state transition matrices.
  • Distribution representation: DistributionValues stores the type of how the distribution is represented, including CopulaOneAsset, CopulaTwoAssets, CDF and RepAgent.
  • Distribution states: DistributionStateType with CDFStates and CopulaStates specifies whether distributions are stored in joint, cumulative form or with Copula and marginals.
  • Indexing: Indexes, DistributionIndexes, and ValueFunctionIndexes unify index handling for variables, equations, and grids to avoid hard-coded positions and to enable robust parsing/preprocessing.
  • Transformations: Transformations is an abstract anchor for elements of function transformations, for example matrices of the Discrete Cosine Transformation.

These types are used throughout the methods in Parsing, Preprocessor, PerturbationSolution, Estimation, and PostEstimation so that code remains generic across model classes and distribution/state encodings.

Parsing

The Parsing submodule translates user and example model inputs into internal, typed representations and auto-generates boilerplate code used throughout the package. It provides:

  • Typed containers: parameter sets, results structs, and index structs to reference variables, shocks, distributions, and function grids without hard-coding positions.
  • Name registries: state_names, control_names, shock_names, aggr_names, and their ASCII variants to ensure consistent mapping from .mod content to code.
  • Prior/settings helpers: prior builds priors; EstimationSettings and the convenience e_set encapsulate estimation-specific knobs.
  • Macros for codegen: generate functions and structs for indices and equations from compact specifications, keeping model authorship ergonomic.

Workflow

  1. Documentation mode: If paths is not defined, it automatically includes the baseline example inputs to make docs and examples self-contained.
  2. Struct and index generation: @make_struct and @make_struct_aggr create index structs; @make_fn and @make_fnaggr generate functions like produce_indexes that map names → positions.
  3. Macro utilities: @generate_equations expands compact model equations; argument macros like @write_args_hh_prob* and @read_args_hh_prob* manage household problem I/O signatures.
  4. Priors and settings: Includes prior.jl, sets up EstimationSettings, and provides e_set configured with available shock_names.

Key macros and helpers

BASEforHANK.Parsing.@writeXSSMacro
@writeXSS()

Write all single steady state variables into vectors XSS / XSSaggr.

Requires

(module) globals state_names, control_names, aggr_names

source
BASEforHANK.Parsing.@make_fnMacro
@make_fn(fn_name)

Create function fn_name that returns an instance of IndexStruct (created by @make_struct), mapping states and controls to indexes inferred from numerical parameters and compression indexes.

Requires

(module) global state_names, control_names

source
BASEforHANK.Parsing.@make_fnaggrMacro
@make_fnaggr(fn_name)

Create function fn_name that returns an instance of IndexStructAggr (created by @make_struct_aggr), mapping aggregate states and controls to values 1 to length(aggr_names) (both steady state and deviation from it).

Requires

(module) global aggr_names

source
BASEforHANK.Parsing.@make_structMacro
@make_struct(struct_name)

Generate a struct named struct_name that contains index fields for state and control variables and associated distribution/value-function indexes. For each state_names element the macro creates <name>SS and <name> integer fields; for each control_names element it creates <name>SS and <name> integer fields. The resulting struct also includes fields distrSS, valueFunctionSS, distr, and valueFunction to hold index containers.

Notes

  • Relies on module-level state_names and control_names variables.
source
BASEforHANK.Parsing.@make_struct_aggrMacro
@make_struct_aggr(struct_name)

Generate a lightweight struct named struct_name that contains integer index fields for each name in the global aggr_names vector. For every aggregate name this macro creates two integer fields: one named <name>SS for the steady-state index and one named <name> for the deviation index.

Notes

  • Relies on a module-level aggr_names variable (array of names).
source
BASEforHANK.Parsing.@generate_equationsMacro
@generate_equations()

Write out the expansions around steady state for all variables in aggr_names, i.e. generate code that reads aggregate states/controls from steady state deviations.

Equations take the form of (with variable r as example):

  • r = exp.(XSS[indexes.rSS] .+ X[indexes.r])
  • rPrime = exp.(XSS[indexes.rSS] .+ XPrime[indexes.r])

Requires

(module) global aggr_names

source
BASEforHANK.Parsing.@write_args_hh_probMacro
@write_args_hh_prob()

Write all variables defined in the global string vector args_hh_prob_names into a vector args_hh_prob based on the variables with the according names in local scope. The type of the vector is inferred from the first variable in args_hh_prob_names.

Requires

(module) global args_hh_prob_names

source
BASEforHANK.Parsing.@read_args_hh_probMacro
@read_args_hh_prob()

Read all variables defined in the global string vector args_hh_prob_names into local scope based on the variables with the according names in the vector args_hh_prob.

Requires

(module) global args_hh_prob_names, args_hh_prob

source

Additional core entries (overview):

  • ModelParameters, NumericalParameters: Core parameter containers referenced across modules.
  • EstimationSettings, e_set: Holds estimation control knobs (proposal scales, ME caps, horizons, etc.).
  • IndexStruct, IndexStructAggr: Programmatic indices that map named objects to positions in vectors/matrices.
  • produce_indexes, produce_indexes_aggr: Functions generated by macros to build index maps from name lists.
  • prior: Constructs distributions for parameter priors consistent with ModelParameters.
  • Name lists: state_names, control_names, shock_names, aggr_names, distr_names used in parsing and preprocessing.