Alex Wozniakowski

[Alex Wozniakowski @airwoz]

Our paper generalizes gradient boosting in the setting of regression.

@MLSTjournal publication:
https://t.co/PlgMQzvhAX

@github repository:
https://t.co/TxcJz5nZH5

[Alex Wozniakowski @airwoz]

@rasbt @tdietterich @MuhammadAnas707 Gronlund's conclusion seems to be: "While not final proof that this is the real or only explanation, it suggests that the success of gradient boosters, despite having poor margins, may be explained by the many small predictions made by the base learner trees."

[Alex Wozniakowski @airwoz]

@rasbt @tdietterich @MuhammadAnas707 Gronlund et al. and Cortes et al. have recent works on gradient boosting, regularization, and margin bounds: https://t.co/vwbFO2PacQ and https://t.co/gC4qpLn1sn

[Alex Wozniakowski @airwoz]

@rasbt @Sidsharmaa22 On the regression front, there's a more general formulation: https://t.co/PlgMQzvhAX… that allows gradient boosting to regard predictions from any regression model as an inductive bias.

[Alex Wozniakowski @airwoz]

@rasbt @Sidsharmaa22 Around the same time, there is AnyBoost by Mason et al. Then, SAMME came later and the paper formulates it with forward stagewise additive modeling.

[Alex Wozniakowski @airwoz]

@Sidsharmaa22 @rasbt The multiclass classification version is found in their textbook.

[Alex Wozniakowski @airwoz]

@Sidsharmaa22 @rasbt For binary classification, it traces back to Friedman, Hastie, and Tibshirani, when they rederived adaptive boosting in terms of the additive model and maximum likelihood estimation. Then, Friedman reformulates the results in terms of the gradient boosting machine.

[Machine Learning: Science and Technology @MLSTjournal]

'A new formulation of #gradientboosting' by @airwoz @MileGu_ @FelixBinder and Jayne Thompson @NTUsg @quantumlah @NUSingapore @iqoqivienna @tu_wien hits 2000 downloads! https://t.co/ekFJ02egNV #quantum #machinelearning #quantumcomputing #algorithms #openscience #AI #qubits #device

[Machine Learning: Science and Technology @MLSTjournal]

'A new formulation of #gradientboosting' by @airwoz @MileGu_ @FelixBinder and Jayne Thompson @NTUsg @quantumlah @NUSingapore @iqoqivienna @tu_wien hits 1500 downloads! https://t.co/ekFJ02vRFt #quantum #machinelearning #opendata #quantumcomputing #algorithms #scikit #AI #HPC

[Machine Learning: Science and Technology @MLSTjournal]

'A new formulation of #gradientboosting' by @airwoz @MileGu_ @FelixBinder and Jayne Thompson @NTUsg @quantumlah @NUSingapore @iqoqivienna @tu_wien hits 1000 downloads! https://t.co/ekFJ02vRFt #quantum #machinelearning #opendata #quantumcomputing #algorithms #openscience

[Machine Learning: Science and Technology @MLSTjournal]

'A new formulation of gradient boosting' by @airwoz @MileGu_ @FelixBinder and Jayne Thompson @NTUsg @quantumlah @NUSingapore @iqoqivienna @tu_wien hits 500 downloads! https://t.co/ekFJ02vRFt #quantum #machinelearning #opendata #quantumcomputing #algorithms #Qubits #AI

[Alex Wozniakowski @airwoz]

Buhlmann and Yu, then others, such as @docmilanfar, pointed out connections between gradient boosting and Tukey's method of reroughing.

This paper fully merges the two methods, and a comparison with LightGBM suggests its viability.

[Alex Wozniakowski @airwoz]

Our paper generalizes gradient boosting in the setting of regression.

@MLSTjournal publication:
https://t.co/PlgMQzvhAX

@github repository:
https://t.co/TxcJz5nZH5

[Machine Learning: Science and Technology @MLSTjournal]

Great new work by @airwoz @MileGu_ @FelixBinder and Jayne Thompson @NTUsg @quantumlah @NUSingapore @iqoqivienna @tu_wien - 'A new formulation of gradient boosting' - https://t.co/ekFJ02vRFt #quantum #machinelearning #opendata #quantumcomputing #algorithms #datascience

[Alex Wozniakowski @airwoz]

@stevenstrogatz Simmons is the MIT OCW book (so there are some online lectures from David Jerison). It describes radians from scratch, illustrates the differential (below), etc., where the underlying theory is kept to the Appendix. https://t.co/SGGYD7htv9

[Alex Wozniakowski @airwoz]

@nhigham The shapes of X and Y prohibit multiplication.

[Alex Wozniakowski @airwoz]

The plots compare non-nested and nested cross-validation in a @GoogleAI regression task. As the score is mean absolute error (nonnegative), the non-nested procedure exhibits a performance evaluation bias.

Code: https://t.co/TxcJz5FB5F

[Alex Wozniakowski @airwoz]

Scikit-physlearn is now available open-source: https://t.co/TxcJz5FB5F

It provides a unified regressor (@scikit_learn, LightGBM, XGBoost, @CatBoostML, Mlxtend), and it supports base boosting.

Paper: https://t.co/lOjixDo6os

[Alex Wozniakowski @airwoz]

@carrasqu In general, eschewing prior scientific knowledge in machine learning appears counter to science, which builds on previous discoveries. In contrast, conventional applications, such as NLP, computer vision, etc., emanated from the lack of an explicit model of the domain.

[Alex Wozniakowski @airwoz]

@carrasqu Here's a recent survey on the few-shot learning problem, which addresses the incorporation of prior domain knowledge in scenarios with a paucity of labeled data: https://t.co/6L78zaU6wn

[Alex Wozniakowski @airwoz]

@carrasqu In the low data regime (which frequently occurs in quantum experiments), we definitely want to leverage prior domain knowledge: https://t.co/lOjixDFHg0