How to Optimize SVMs

Support Vector Machines (SVMs) are very elegant classical machine learning models before the era of deep learning. I still remember... Continue

How to Optimize Multinomial Logistic Regression

I randomly came to scikit-learn LogisticRegression page and found they might support different regularization terms such as $\ell$-1 norm, squared... Continue

Scalable Low-Rank Tensor Completion

Tensor completion asks us to recover a multidimensional array from only a subset of its entries. A hyperspectral image is... Continue

Robust Structured Multi-Task Regression

Multi-task regression is useful when several prediction problems are related, but not identical. Instead of fitting one independent model for... Continue

Sparse Orthogonality-Constrained Optimization via ADPMM

Optimization problems with orthogonality constraints arise throughout machine learning, including dimensionality reduction, clustering, and representation learning. These problems require the... Continue

Simplex-Constrained Incoherent Matrix Factorization with Hybrid Mirror Descent

Matrix factorization is a fundamental tool in data mining and machine learning, with applications in clustering, topic modeling, recommender systems,... Continue

FISTA

“Fast iterative shrinkage-thresholding algorithm”(FISTA) is a proximal gradient method that aims to solve convex optimization problems of the form: \[\min_x... Continue