Incremental feature selection and l1 regularization for relaxed maximum-entropy modeling
The paper presents a combined feature selection and regularization technique for maximum-entropy models that incorporates l1-regularization into gradient-based, incremental feature selection. Our method is based on the Grafting framework of Perkins et al. (2003) which it extends by n-best feature selection. We present experimental results showing the advantage of n-best feature selection using l1-regularization over l0-, l1-, and l2-regularization for the task of maximum-entropy parsing, both in terms of improved computational complexity and generalization performance.
Riezler, S ; Vasserman, A. Incremental feature selection and l1 regularization for relaxed maximum-entropy modeling. Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing; 2004 July 25-26; Barcelona; Spain.