ACEfit

Documentation for ACEfit.

Scikit-learn solvers

To use Python based Scikit-learn solvers you need to load PythonCall in addition to ACEfit.

using ACEfit
using PythonCall

MLJ solvers

To use MLJ solvers you need to load MLJ in addition to ACEfit

using ACEfit
using MLJ

After that you need to load an appropriate MLJ solver. Take a look on available MLJ solvers. Note that only MLJScikitLearnInterface.jl and MLJLinearModels.jl have extension available. To use other MLJ solvers please file an issue.

You need to load the solver and then create a solver structure

# Load ARD solver
ARDRegressor = @load ARDRegressor pkg=MLJScikitLearnInterface

# Create the solver itself and give it parameters
solver = ARDRegressor(
    max_iter = 300,
    tol = 1e-3,
    threshold_lambda = 10000
)

After this you can use the MLJ solver like any other solver.

Index

ACEfit.ASP — Type

ASP : Active Set Pursuit solver

Solves the lasso optimization problem.

\[\max_{y} \left( b^T y - \frac{1}{2} λ y^T y \right)\]

subject to

\[ \|A^T y\|_{\infty} \leq 1.\]

Constructor Keyword arguments

ACEfit.ASP(; P = I, select = (:byerror, 1.0), tsvd = false, nstore=100, 
            params...)
  • select : Selection criterion for the final solution (required)
    • :final : final solution (largest computed basis)
    • (:byerror, q) : solution with error within q times the minimum error along the path; if training error is used and q == 1.0, then this is equivalent to to :final.
    • (:bysize, n) : best solution with at most n non-zero features; if training error is used, then it will be the solution with exactly n non-zero features.
  • P = I : prior / regularizer (optional)

The remaining kwarguments to ASP are parameters for the ASP homotopy solver.

  • actMax : Maximum number of active constraints.
  • min_lambda : Minimum value for λ. (defaults to 0)
  • loglevel : Logging level.
  • itnMax : Maximum number of iterations.

Extended syntax for solve

solve(solver::ASP, A, y, Aval=A, yval=y)
  • A : m-by-n design matrix. (required)
  • b : m-vector. (required)
  • Aval = nothing : p-by-n validation matrix
  • bval = nothing : p- validation vector

If independent Aval and yval are provided (instead of detaults A, y), then the solver will use this separate validation set instead of the training set to select the best solution along the model path.

source
ACEfit.AbstractData — Type

ACEfit users should define a type of the form: UserData <: AbstractData

Several functions acting on such a type should be implemented: countobservations featurematrix targetvector weightvector

source
ACEfit.BLR — Type

struct BLR : Bayesian linear regression

Refer to bayesianlinear.jl (for now) for kwarg definitions.
source
ACEfit.OMP — Type

OMP : Orthogonal Matching Pursuit solver

Solves the lasso optimization problem.

\[\max_{y} \left( b^T y - \frac{1}{2} λ y^T y \right)\]

subject to

\[ \|A^T y\|_{\infty} \leq 1.\]

Constructor Keyword arguments

ACEfit.ASP(; P = I, select = (:byerror, 1.0), tsvd = false, nstore=100, 
            params...)
  • select : Selection criterion for the final solution (required)
    • :final : final solution (largest computed basis)
    • (:byerror, q) : solution with error within q times the minimum error along the path; if training error is used and q == 1.0, then this is equivalent to to :final.
    • (:bysize, n) : best solution with at most n non-zero features; if training error is used, then it will be the solution with exactly n non-zero features.
  • P = I : prior / regularizer (optional)

The remaining kwarguments to ASP are parameters for the ASP homotopy solver.

  • actMax : Maximum number of active constraints.
  • min_lambda : Minimum value for λ. (defaults to 0)
  • loglevel : Logging level.
  • itnMax : Maximum number of iterations.

Extended syntax for solve

solve(solver::ASP, A, y, Aval=A, yval=y)
  • A : m-by-n design matrix. (required)
  • b : m-vector. (required)
  • Aval = nothing : p-by-n validation matrix
  • bval = nothing : p- validation vector

If independent Aval and yval are provided (instead of detaults A, y), then the solver will use this separate validation set instead of the training set to select the best solution along the model path.

source
ACEfit.QR — Type

struct QR : linear least squares solver, using standard QR factorisation; this solver computes

\[ θ = \arg\min \| A \theta - y \|^2 + \lambda \| P \theta \|^2\]

Constructor

ACEfit.QR(; lambda = 0.0, P = nothing)

where

  • λ : regularisation parameter
  • P : right-preconditioner / tychonov operator
source
ACEfit.RRQR — Type

struct RRQR : linear least squares solver, using rank-revealing QR factorisation, which can sometimes be more robust / faster than the standard regularised QR factorisation. This solver first transforms the parameters $\theta_P = P \theta$, then solves

\[ θ = \arg\min \| A P^{-1} \theta_P - y \|^2\]

where the truncation tolerance is given by the rtol parameter, and finally reverses the transformation. This uses the pqrfact of LowRankApprox.jl; For further details see the documentation of LowRankApprox.jl.

Crucially, note that this algorithm is not deterministic; the results can change slightly between applications.

Constructor

ACEfit.RRQR(; rtol = 1e-15, P = I)

where

  • rtol : truncation tolerance
  • P : right-preconditioner / tychonov operator
source
ACEfit.TruncatedSVD — Type

struct TruncatedSVD : linear least squares solver for approximately solving

\[ θ = \arg\min \| A \theta - y \|^2 \]

  • transform $\tilde\theta = P \theta$
  • perform svd on $A P^{-1}$
  • truncate svd at rtol, i.e. keep only the components for which $\sigma_i \geq {\rm rtol} \max \sigma_i$
  • Compute $\tilde\theta$ from via pseudo-inverse
  • Reverse transformation $\theta = P^{-1} \tilde\theta$

Constructor

ACEfit.TruncatedSVD(; rtol = 1e-9, P = I)

where

  • rtol : relative tolerance
  • P : right-preconditioner / tychonov operator
source
ACEfit.basis_size — Method
basis_size(model)

Return the length of the basis, assuming that model is a linear model, or when interpreted as a linear model. The returned integer must match the size of the feature matrix that will be assembled for the given model.

It defaults to Base.length but can be overloaded if needed.

source