| Title: | Parametric Simplex Method for Sparse Learning |
|---|---|
| Description: | Implements a unified framework of parametric simplex method for a variety of sparse learning problems (e.g., Dantzig selector (for linear regression), sparse quantile regression, sparse support vector machines, and compressive sensing) combined with efficient hyper-parameter selection strategies. The core algorithm is implemented in C++ with Eigen3 support for portable high performance linear algebra. For more details about parametric simplex method, see Haotian Pang (2017) <https://papers.nips.cc/paper/6623-parametric-simplex-method-for-sparse-learning.pdf>. |
| Authors: | Zichong Li, Qianli Shen |
| Maintainer: | Zichong Li <[email protected]> |
| License: | GPL (>= 2) |
| Version: | 1.0.2 |
| Built: | 2025-03-06 04:48:25 UTC |
| Source: | https://github.com/cran/PRIMAL |
A package for parametric simplex method for sparse learning
| Package: | PRIMAL |
| Type: | Package |
| Version: | 1.0.0 |
| Date: | 2019-08-15 |
The package "PRIMAL" provides 5 main functions:
(1) The dantzig selector solver applying simplex method. Please refer to Dantzig_solver.
(2) The sparse SVM solver applying simplex method. Please refer to SparseSVM_solver.
(3) The compressed sensing solver. Please refer to CompressedSensing_solver.
(4) The quantile regression solver. Please refer to QuantileRegression_solver.
(5) The solver for standard formulation of parametric simplex method. Please refer to PSM_solver.
Qianli Shen, Zichong Li
plot.primal, print.primal, coef.primal
Print the estimated solution correspond to a specific parameter.
## S3 method for class 'primal' coef(object, n = NULL, ...)## S3 method for class 'primal' coef(object, n = NULL, ...)
object |
An object with S3 class |
n |
The index of the wanted parameter. |
... |
System reserved (No specific usage) |
Dantzig_solver, SparseSVM_solver
Solve given compressed sensing problem in parametric simplex method
CompressedSensing_solver(X, y, max_it = 50, lambda_threshold = 0.01)CompressedSensing_solver(X, y, max_it = 50, lambda_threshold = 0.01)
X |
|
y |
|
max_it |
This is the number of the maximum path length one would like to achieve. The default length is |
lambda_threshold |
The parametric simplex method will stop when the calculated parameter is smaller than lambda. The default value is |
An object with S3 class "primal" is returned:
data |
The |
response |
The length |
beta |
A matrix of regression estimates whose columns correspond to regularization parameters for parametric simplex method. |
df |
The degree of freedom (number of nonzero coefficients) along the solution path. |
value |
The sequence of optimal value of the object function corresponded to the sequence of lambda. |
iterN |
The number of iteration in the program. |
lambda |
The sequence of regularization parameters |
type |
The type of the problem, such as |
primal-package, Dantzig_solver
## Compressed Sensing ## We set X to be standard normal random matrix and generate Y using gaussian noise. ## Generate the design matrix and coefficient vector n = 100 # sample number d = 250 # sample dimension c = 0.5 # correlation parameter s = 20 # support size of coefficient set.seed(1024) X = scale(matrix(rnorm(n*d),n,d)+c*rnorm(n))/sqrt(n-1)*sqrt(n) beta = c(rnorm(s), rep(0, d-s)) ## Generate response using Gaussian noise, and solve the solution path noise = rnorm(n) Y = X%*%beta + noise ## Compressed Sensing solved with parametric simplex method fit.compressed = CompressedSensing_solver(X, Y, max_it = 100, lambda_threshold = 0.01) ###lambdas used print(fit.compressed$lambda) ## number of nonzero coefficients for each lambda print(fit.compressed$df) ## Visualize the solution path plot(fit.compressed)## Compressed Sensing ## We set X to be standard normal random matrix and generate Y using gaussian noise. ## Generate the design matrix and coefficient vector n = 100 # sample number d = 250 # sample dimension c = 0.5 # correlation parameter s = 20 # support size of coefficient set.seed(1024) X = scale(matrix(rnorm(n*d),n,d)+c*rnorm(n))/sqrt(n-1)*sqrt(n) beta = c(rnorm(s), rep(0, d-s)) ## Generate response using Gaussian noise, and solve the solution path noise = rnorm(n) Y = X%*%beta + noise ## Compressed Sensing solved with parametric simplex method fit.compressed = CompressedSensing_solver(X, Y, max_it = 100, lambda_threshold = 0.01) ###lambdas used print(fit.compressed$lambda) ## number of nonzero coefficients for each lambda print(fit.compressed$df) ## Visualize the solution path plot(fit.compressed)
Solve given Dantzig selector problem in parametric simplex method
Dantzig_solver(X, y, max_it = 50, lambda_threshold = 0.01)Dantzig_solver(X, y, max_it = 50, lambda_threshold = 0.01)
X |
|
y |
|
max_it |
This is the number of the maximum path length one would like to achieve. The default length is |
lambda_threshold |
The parametric simplex method will stop when the calculated parameter is smaller than lambda. The default value is |
An object with S3 class "primal" is returned:
data |
The |
response |
The length |
beta |
A matrix of regression estimates whose columns correspond to regularization parameters for parametric simplex method. |
df |
The degree of freedom (number of nonzero coefficients) along the solution path. |
value |
The sequence of optimal value of the object function corresponded to the sequence of lambda. |
iterN |
The number of iteration in the program. |
lambda |
The sequence of regularization parameters |
type |
The type of the problem, such as |
## Dantzig ## We set X to be standard normal random matrix and generate Y using gaussian noise. ## Generate the design matrix and coefficient vector n = 100 # sample number d = 250 # sample dimension c = 0.5 # correlation parameter s = 20 # support size of coefficient set.seed(1024) X = scale(matrix(rnorm(n*d),n,d)+c*rnorm(n))/sqrt(n-1)*sqrt(n) beta = c(rnorm(s), rep(0, d-s)) ## Generate response using Gaussian noise, and solve the solution path noise = rnorm(n) Y = X%*%beta + noise ## Dantzig selection solved with parametric simplex method fit.dantzig = Dantzig_solver(X, Y, max_it = 100, lambda_threshold = 0.01) ###lambdas used print(fit.dantzig$lambda) ## number of nonzero coefficients for each lambda print(fit.dantzig$df) ## Visualize the solution path plot(fit.dantzig)## Dantzig ## We set X to be standard normal random matrix and generate Y using gaussian noise. ## Generate the design matrix and coefficient vector n = 100 # sample number d = 250 # sample dimension c = 0.5 # correlation parameter s = 20 # support size of coefficient set.seed(1024) X = scale(matrix(rnorm(n*d),n,d)+c*rnorm(n))/sqrt(n-1)*sqrt(n) beta = c(rnorm(s), rep(0, d-s)) ## Generate response using Gaussian noise, and solve the solution path noise = rnorm(n) Y = X%*%beta + noise ## Dantzig selection solved with parametric simplex method fit.dantzig = Dantzig_solver(X, Y, max_it = 100, lambda_threshold = 0.01) ###lambdas used print(fit.dantzig$lambda) ## number of nonzero coefficients for each lambda print(fit.dantzig$df) ## Visualize the solution path plot(fit.dantzig)
Plot regularization path and parameter obtained from the algorithm.
## S3 method for class 'primal' plot(x, n = NULL, ...)## S3 method for class 'primal' plot(x, n = NULL, ...)
x |
An object with S3 class |
n |
If |
... |
System reserved (No specific usage) |
Dantzig_solver, SparseSVM_solver
Print the information about the model usage, the parameter path, degree of freedom of the solution path.
## S3 method for class 'primal' print(x, ...)## S3 method for class 'primal' print(x, ...)
x |
An object with S3 class |
... |
System reserved (No specific usage) |
Dantzig_solver, SparseSVM_solver
Solve given problem in parametric simplex method
PSM_solver(A, b, b_bar, c, c_bar, B_init = NULL, max_it = 50, lambda_threshold = 0.01)PSM_solver(A, b, b_bar, c, c_bar, B_init = NULL, max_it = 50, lambda_threshold = 0.01)
A |
|
b |
|
b_bar |
|
c |
|
c_bar |
|
B_init |
|
max_it |
This is the number of the maximum path length one would like to achieve. The default length is |
lambda_threshold |
The parametric simplex method will stop when the calculated parameter is smaller than lambda. The default value is |
An object with S3 class "primal" is returned:
data |
The |
response |
The length |
beta |
A matrix of regression estimates whose columns correspond to regularization parameters for parametric simplex method. |
beta0 |
A vector of regression estimates whose index correspond to regularization parameters for parametric simplex method. |
df |
The degree of freecom (number of nonzero coefficients) along the solution path. |
value |
The sequence of optimal value of the object function corresponded to the sequence of lambda. |
iterN |
The number of iteration in the program. |
lambda |
The sequence of regularization parameters |
type |
The type of the problem, such as |
## This example show how to use PSM_solver() to solve dantzig problem. ## Generate the design matrix and coefficient vector n = 100 # sample number d = 250 # sample dimension c = 0.5 # correlation parameter s = 20 # support size of coefficient set.seed(1024) X = scale(matrix(rnorm(n*d),n,d)+c*rnorm(n))/sqrt(n-1)*sqrt(n) beta = c(rnorm(s), rep(0, d-s)) ## Generate response using Gaussian noise, and solve the solution path noise = rnorm(n) Y = X%*%beta + noise ## Define parameters for dantzig problem XtX = t(X)%*%X A = cbind(cbind(rbind(XtX,-XtX),-rbind(XtX,-XtX)),diag(rep(1,2*d))) b = rbind(t(X)%*%Y,-t(X)%*%Y) c = c(rep(-1,2*d),rep(0,2*d)) c_bar = rep(0,4*d) b_bar = rep(1,2*d) B_init = seq(2*d,4*d-1) ## Dantzig selection solved with parametric simplex method fit.dantzig = PSM_solver(A, b, b_bar, c, c_bar, B_init, max_it = 50, lambda_threshold = 0.01) ###lambdas used print(fit.dantzig$lambda) ## number of nonzero coefficients for each lambda print(fit.dantzig$df) ## Visualize the solution path plot(fit.dantzig)## This example show how to use PSM_solver() to solve dantzig problem. ## Generate the design matrix and coefficient vector n = 100 # sample number d = 250 # sample dimension c = 0.5 # correlation parameter s = 20 # support size of coefficient set.seed(1024) X = scale(matrix(rnorm(n*d),n,d)+c*rnorm(n))/sqrt(n-1)*sqrt(n) beta = c(rnorm(s), rep(0, d-s)) ## Generate response using Gaussian noise, and solve the solution path noise = rnorm(n) Y = X%*%beta + noise ## Define parameters for dantzig problem XtX = t(X)%*%X A = cbind(cbind(rbind(XtX,-XtX),-rbind(XtX,-XtX)),diag(rep(1,2*d))) b = rbind(t(X)%*%Y,-t(X)%*%Y) c = c(rep(-1,2*d),rep(0,2*d)) c_bar = rep(0,4*d) b_bar = rep(1,2*d) B_init = seq(2*d,4*d-1) ## Dantzig selection solved with parametric simplex method fit.dantzig = PSM_solver(A, b, b_bar, c, c_bar, B_init, max_it = 50, lambda_threshold = 0.01) ###lambdas used print(fit.dantzig$lambda) ## number of nonzero coefficients for each lambda print(fit.dantzig$df) ## Visualize the solution path plot(fit.dantzig)
Solve given quantile regression problem in parametric simplex method
QuantileRegression_solver(X, y, max_it = 50, lambda_threshold = 0.01, tau = 0.5)QuantileRegression_solver(X, y, max_it = 50, lambda_threshold = 0.01, tau = 0.5)
X |
|
y |
|
max_it |
This is the number of the maximum path length one would like to achieve. The default length is |
lambda_threshold |
The parametric simplex method will stop when the calculated parameter is smaller than lambda. The default value is |
tau |
The quantile number you want. The default quantile is |
An object with S3 class "primal" is returned:
data |
The |
response |
The length |
beta |
A matrix of regression estimates whose columns correspond to regularization parameters for parametric simplex method. |
df |
The degree of freedom (number of nonzero coefficients) along the solution path. |
value |
The sequence of optimal value of the object function corresponded to the sequence of lambda. |
iterN |
The number of iteration in the program. |
lambda |
The sequence of regularization parameters |
type |
The type of the problem, such as |
primal-package, Dantzig_solver
## Quantile Regression ## We set X to be standard normal random matrix and generate Y using gaussian noise ## with default quantile number to be 0.5. ## Generate the design matrix and coefficient vector n = 100 # sample number d = 250 # sample dimension c = 0.5 # correlation parameter s = 20 # support size of coefficient set.seed(1024) X = scale(matrix(rnorm(n*d),n,d)+c*rnorm(n))/sqrt(n-1)*sqrt(n) beta = c(rnorm(s), rep(0, d-s)) ## Generate response using Gaussian noise, and solve the solution path noise = rnorm(n) Y = X%*%beta + noise ## Quantile Regression problem solved with parametric simplex method fit.quantile = QuantileRegression_solver(X, Y, max_it = 100, lambda_threshold = 0.01) ###lambdas used print(fit.quantile$lambda) ## number of nonzero coefficients for each lambda print(fit.quantile$df) ## Visualize the solution path plot(fit.quantile)## Quantile Regression ## We set X to be standard normal random matrix and generate Y using gaussian noise ## with default quantile number to be 0.5. ## Generate the design matrix and coefficient vector n = 100 # sample number d = 250 # sample dimension c = 0.5 # correlation parameter s = 20 # support size of coefficient set.seed(1024) X = scale(matrix(rnorm(n*d),n,d)+c*rnorm(n))/sqrt(n-1)*sqrt(n) beta = c(rnorm(s), rep(0, d-s)) ## Generate response using Gaussian noise, and solve the solution path noise = rnorm(n) Y = X%*%beta + noise ## Quantile Regression problem solved with parametric simplex method fit.quantile = QuantileRegression_solver(X, Y, max_it = 100, lambda_threshold = 0.01) ###lambdas used print(fit.quantile$lambda) ## number of nonzero coefficients for each lambda print(fit.quantile$df) ## Visualize the solution path plot(fit.quantile)
Solve given Sparse SVM problem in parametric simplex method
SparseSVM_solver(X, y, max_it = 50, lambda_threshold = 0.01)SparseSVM_solver(X, y, max_it = 50, lambda_threshold = 0.01)
X |
|
y |
|
max_it |
This is the number of the maximum path length one would like to achieve. The default length is |
lambda_threshold |
The parametric simplex method will stop when the calculated parameter is smaller than lambda. The default value is |
An object with S3 class "primal" is returned:
data |
The |
response |
The length |
beta |
A matrix of regression estimates whose columns correspond to regularization parameters for parametric simplex method. |
beta0 |
A vector of regression estimates whose index correspond to regularization parameters for parametric simplex method. |
df |
The degree of freecom (number of nonzero coefficients) along the solution path. |
value |
The sequence of optimal value of the object function corresponded to the sequence of lambda. |
iterN |
The number of iteration in the program. |
lambda |
The sequence of regularization parameters |
type |
The type of the problem, such as |
## SparseSVM ## We set the X matrix to be normal random matrix and Y is a vector consists of -1 and 1 ## with the number of iteration to be 1000. ## Generate the design matrix and coefficient vector n = 200 # sample number d = 100 # sample dimension c = 0.5 # correlation parameter s = 20 # support size of coefficient set.seed(1024) X = matrix(rnorm(n*d),n,d)+c*rnorm(n) ## Generate response and solve the solution path Y <- sample(c(-1,1),n,replace = TRUE) ## Sparse SVM solved with parametric simplex method fit.SVM = SparseSVM_solver(X, Y, max_it = 1000, lambda_threshold = 0.01) ## lambdas used print(fit.SVM$lambda) ## Visualize the solution path plot(fit.SVM)## SparseSVM ## We set the X matrix to be normal random matrix and Y is a vector consists of -1 and 1 ## with the number of iteration to be 1000. ## Generate the design matrix and coefficient vector n = 200 # sample number d = 100 # sample dimension c = 0.5 # correlation parameter s = 20 # support size of coefficient set.seed(1024) X = matrix(rnorm(n*d),n,d)+c*rnorm(n) ## Generate response and solve the solution path Y <- sample(c(-1,1),n,replace = TRUE) ## Sparse SVM solved with parametric simplex method fit.SVM = SparseSVM_solver(X, Y, max_it = 1000, lambda_threshold = 0.01) ## lambdas used print(fit.SVM$lambda) ## Visualize the solution path plot(fit.SVM)