Bellet, Aurélien; Habrard, Amaury; Morvant, EmilieISTA ; Sebban, Marc
Weighted majority votes allow one to combine the output of several classifiers or voters. MinCq is a recent algorithm for optimizing the weight of each voter based on the minimization of a theoretical bound over the risk of the vote with elegant PAC-Bayesian generalization guarantees. However, while it has demonstrated good performance when combining weak classifiers, MinCq cannot make use of the useful a priori knowledge that one may have when using a mixture of weak and strong voters. In this paper, we propose P-MinCq, an extension of MinCq that can incorporate such knowledge in the form of a constraint over the distribution of the weights, along with general proofs of convergence that stand in the sample compression setting for data-dependent voters. The approach is applied to a vote of k-NN classifiers with a specific modeling of the voters' performance. P-MinCq significantly outperforms the classic k-NN classifier, a symmetric NN and MinCq using the same voters. We show that it is also competitive with LMNN, a popular metric learning algorithm, and that combining both approaches further reduces the error.
This work was funded by the French project SoLSTiCe ANR-13-BS02-01 of the ANR.
129 - 154
Bellet A, Habrard A, Morvant E, Sebban M. Learning a priori constrained weighted majority votes. Machine Learning. 2014;97(1-2):129-154. doi:10.1007/s10994-014-5462-z
Bellet, A., Habrard, A., Morvant, E., & Sebban, M. (2014). Learning a priori constrained weighted majority votes. Machine Learning. Springer. https://doi.org/10.1007/s10994-014-5462-z
Bellet, Aurélien, Amaury Habrard, Emilie Morvant, and Marc Sebban. “Learning a Priori Constrained Weighted Majority Votes.” Machine Learning. Springer, 2014. https://doi.org/10.1007/s10994-014-5462-z.
A. Bellet, A. Habrard, E. Morvant, and M. Sebban, “Learning a priori constrained weighted majority votes,” Machine Learning, vol. 97, no. 1–2. Springer, pp. 129–154, 2014.
Bellet A, Habrard A, Morvant E, Sebban M. 2014. Learning a priori constrained weighted majority votes. Machine Learning. 97(1–2), 129–154.
Bellet, Aurélien, et al. “Learning a Priori Constrained Weighted Majority Votes.” Machine Learning, vol. 97, no. 1–2, Springer, 2014, pp. 129–54, doi:10.1007/s10994-014-5462-z.
All files available under the following license(s):
This Item is protected by copyright and/or related rights. [...]
Link(s) to Main File(s)