Kaj Björklund

Software engineer

[TensorFlow @TensorFlow]

IPS-weighting or two-tower model? 🤔

Watch this use case video on unbiased LTR and debiasing clicks to learn to solve the problem of position bias 🧡 ↓

[Valeriy M., PhD, MBA, CQF @predict_addict]

Still using using XGBoost (substitute your favourite classifier) and think it outputs probabilities? Read on.

Probabilistic Prediction in scikit-learn.

'Adding confidence measures to predictive models should increase the trustworthiness, but only if the models are well-calibrated.

Historically, some algorithms like logistic regression, but also neural networks, have been considered to produce well-calibrated probability estimates off-the-shelf. Other techniques, like decision trees and Naive Bayes, on the other hand, are infamous for being significantly overconfident in their probabilistic predictions. In this paper, a large experimental study is conducted to investigate how well-calibrated models produced by a number of algorithms in the scikit-learn library are out-of-the-box, but also if either the built-in calibration techniques Platt scaling and isotonic regression, or Venn-Abers, can be used to improve the calibration.

The results show that of the seven algorithms evaluated, the only one obtaining well-calibrated models without the external calibration is logistic regression. All other algorithms, i.e., decision trees, adaboost, gradient boosting, kNN, naive Bayes and random forest benefit from using any of the calibration techniques. In particular, decision trees, Naive Bayes and the boosted models are substantially improved using external calibration.

From a practitioner’s perspective, the obvious recommendation becomes to incorporate calibration when using probabilistic prediction.'

https://t.co/uI5i4OgLte

#algorithms #neuralnetworks #uncertainty #machinelearning #conformalprediction

[Russ Cox @_rsc]

“Coroutines for Go”
https://t.co/HEEL9ntWQf

[Luke Gessler @LukeGessler]

this paper's nuts. for sentence classification on out-of-domain datasets, all neural (Transformer or not) approaches lose to good old kNN on representations generated by.... gzip https://t.co/6eZiXlJxOX

[Valeriy M., PhD, MBA, CQF @predict_addict]

Deep learning neural networks are notoriously miscalibrated and for several years people thought they can fix them using using Bayesian methods to quantify uncertainty.

And now there is complete disillusionment including by some of the leading bayesians. The way to fix it is not by creating methods that don’t have validity guarantees though but by using #conformalprediction that can be deployed as a reliable and safe UQ layer on top of any model including deep learning without any modification of the model itself. A powerful incentive that no other methods can ever offer.

”Assessing the predictive uncertainty of deep neural networks is crucial for safety- related applications of deep learning. Although Bayesian deep learning offers a principled framework for estimating model uncertainty, the approaches that are commonly used to approximate the posterior often fail to deliver reliable estimates of predictive uncertainty.”

#deeplearning #safety #neuralnetworks #conformalprediction #machinelearning

[George E. Dahl @GeorgeEDahl]

We've just released the first version of our Deep Learning Tuning Playbook! This is our attempt to distill our process for actually getting good results with deep learning. We emphasize hyperparameter tuning since it has been a large pain point. https://t.co/PjeJVWeOzS

[Accepted papers at TMLR @TmlrPub]

ZerO Initialization: Initializing Neural Networks with only Zeros and Ones

Jiawei Zhao, Florian Tobias Schaefer, Anima Anandkumar

https://t.co/FwExkRvshm

[Hacker News Bot @newsycombinator]

Maximum Flow and Minimum-Cost Flow in Almost-Linear Time https://t.co/2sfQV2319Z

[Kaj Björklund @kbjorklu]

@gynvael Censoring? Consider for example whether a person will live to be 100 years old.

[James Long @jlongster]

🎉 It's finally ready! I've been hacking on something for the past couple months and it's absurdly great. It's time for more powerful web apps.

Announcing "A future for SQL on the web": https://t.co/TrXcfMoHa6

[Steeve Morin @steeve]

This article made me dig a bit more into Zig's ability to cross compile C and C++. Honestly, it is hands down the best. The list of targets is _insane_. I think @andy_kelley did ship Clang/LLVM with all possible os/arch combinations. I'm really impressed. https://t.co/3Gi8aRDK80

[Kaj Björklund @kbjorklu]

@SebAaltonen There's also the corrected summation algorithm for exactly this purpose: https://t.co/QXpFONG0UL

[Unity for Games @unitygames]

Full screen. Sound on.

[timorous bestie 😷 @vnfrombucharest]

New blog post: introduction to gradient-based optimization with constraints: reparametrization, projected gradient, cool connections in between.

🔮Geometry insights show that 🪞mirror descent = reparametrization with a "straight-through" sigmoid!

https://t.co/KQcq6mkM1i

[Postgres Weekly 🐘 @PostgresWeekly]

PostgreSQL 12 Released: https://t.co/Fr8Kp1QVES

[Kaj Björklund @kbjorklu]

Verifying myself: I am kbjorklu on https://t.co/cqddctMqcU. BajU1g3UEKjacbLUrj0ko2_Jy-fG9bx_Liz7 / https://t.co/DmXozZwGNB

[Go @golang]

🥳 Go 1.13 is released!

📝 Release notes: https://t.co/B5cODBfZHv

⬇️ Download: https://t.co/StazJXa19M

#golang

[Go @golang]

Go Module Mirror and Checksum Database officially launched! 😄🎉🎆
Learn more at
https://t.co/IbgVM9EMAK

[Aras Pranckevičius 🇺🇦🇱🇹 @aras_p]

QuickJS javascript engine by Fabrice Bellard: https://t.co/6NZTqASWuj

[Kaj Björklund @kbjorklu]

@Sol_HSA See: https://t.co/mWjlJS2Me4 - that calls it round-robin iterative algorithm until a fixpoint. A variation is the worklist algorithm, where changed nodes are added back to the worklist.