Learning a priori constrained weighted majority votes

article Learning a priori constrained weighted majority votes published yes Aurélien Bellet author Amaury Habrard author Emilie Morvant author 4BAC2A72-F248-11E8-B48F-1D18A9856A870000-0002-8301-7240 Marc Sebban author ChLa department Lifelong Learning of Visual Scene Understanding project Weighted majority votes allow one to combine the output of several classifiers or voters. MinCq is a recent algorithm for optimizing the weight of each voter based on the minimization of a theoretical bound over the risk of the vote with elegant PAC-Bayesian generalization guarantees. However, while it has demonstrated good performance when combining weak classifiers, MinCq cannot make use of the useful a priori knowledge that one may have when using a mixture of weak and strong voters. In this paper, we propose P-MinCq, an extension of MinCq that can incorporate such knowledge in the form of a constraint over the distribution of the weights, along with general proofs of convergence that stand in the sample compression setting for data-dependent voters. The approach is applied to a vote of k-NN classifiers with a specific modeling of the voters' performance. P-MinCq significantly outperforms the classic k-NN classifier, a symmetric NN and MinCq using the same voters. We show that it is also competitive with LMNN, a popular metric learning algorithm, and that combining both approaches further reduces the error. Springer2014 eng Machine Learning 00034143130000710.1007/s10994-014-5462-z 971-2129 - 154 Bellet A, Habrard A, Morvant E, Sebban M. Learning a priori constrained weighted majority votes. Machine Learning. 2014;97(1-2):129-154. doi:<a href="https://doi.org/10.1007/s10994-014-5462-z">10.1007/s10994-014-5462-z</a> A. Bellet, A. Habrard, E. Morvant, M. Sebban, Machine Learning 97 (2014) 129–154. Bellet A, Habrard A, Morvant E, Sebban M. 2014. Learning a priori constrained weighted majority votes. Machine Learning. 97(1–2), 129–154. Bellet, Aurélien, et al. “Learning a Priori Constrained Weighted Majority Votes.” Machine Learning, vol. 97, no. 1–2, Springer, 2014, pp. 129–54, doi:<a href="https://doi.org/10.1007/s10994-014-5462-z">10.1007/s10994-014-5462-z</a>. A. Bellet, A. Habrard, E. Morvant, and M. Sebban, “Learning a priori constrained weighted majority votes,” Machine Learning, vol. 97, no. 1–2. Springer, pp. 129–154, 2014. Bellet, A., Habrard, A., Morvant, E., & Sebban, M. (2014). Learning a priori constrained weighted majority votes. Machine Learning. Springer. <a href="https://doi.org/10.1007/s10994-014-5462-z">https://doi.org/10.1007/s10994-014-5462-z</a> Bellet, Aurélien, Amaury Habrard, Emilie Morvant, and Marc Sebban. “Learning a Priori Constrained Weighted Majority Votes.” Machine Learning. Springer, 2014. <a href="https://doi.org/10.1007/s10994-014-5462-z">https://doi.org/10.1007/s10994-014-5462-z</a>. 21802018-12-11T11:56:10Z2025-09-29T11:35:24Z