1
$\begingroup$

I am solving an L1 regularized least squares of the form like:

$$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} + \lambda {\left\| \boldsymbol{x} \right\|}_{1} $$

I saw that for the L2 norm case there are several methods to obtain the magnitude of $\lambda$ as seen in the study Comparing parameter choice methods for regularization of ill-posed problems . However, I didn't find anything for the L1 case. What should I be doing besides trial and error to efficiently determine $\lambda$ for this case?

Improve this question
$\endgroup$
4
  • $\begingroup$ You can use AIC/BIC or other merit, e.g. adjusted R2, scikit-learn.org/stable/auto_examples/linear_model/… $\endgroup$
    – I.M.
    Commented Dec 9, 2022 at 3:32
  • $\begingroup$ thank you for the info, i am not sure how to implement this. do I for loop over different values of $\lambda$, then for each iteration calculate the log likelihood by $|A\bf{x}-\bf{y}|^2$ and for number of model parameters count the non-zero elements in $\bf{x}$? also for num of object \ sample size, should I use the distention of $\bf{x}$? I understand that calculating the AIC\BIC given for each $\lambda$ gives me a number and the minimum of the number is the $\lambda$ I should use? $\endgroup$
    – yourds
    Commented Dec 12, 2022 at 21:09
  • $\begingroup$ Yes, you can scan over different values of lamda hyperparameter. LARS Lasso might be also of interest to you. scikit-learn.org/stable/modules/… $\endgroup$
    – I.M.
    Commented Dec 13, 2022 at 1:51
  • $\begingroup$ but the minimization that is in my question is lasso, so I dont understand. $\endgroup$
    – yourds
    Commented Dec 13, 2022 at 20:23

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.