dlindelof

Aspiring polymath, technical lead, data scientist, researcher, agile coach, blogger, and public speaker.

[dlindelof @dlindelof]

Spending one hour this morning trying to fix a bug in SQLite taught me more about good programming than anything I could have read in books.

[dlindelof @dlindelof]

Top-down design is a great way to redesign a program you already know how to write. --Unknown

[dlindelof @dlindelof]

Working on a side project, bit of a citizen science thing. Can’t reveal too much at this point but was gratified that my LLM assistant eventually asked “so to which local newspaper do you want to pitch this story?”

[dlindelof @dlindelof]

Had a conversation with the boss yesterday and I think I managed to convince him how pointless it is to set the random number generator seed in analysis notebooks.

[dlindelof @dlindelof]

"If you're worried that your current job is rotting your brain, it probably is." (Joel Spolsky, The Best Software Writing I)

[dlindelof @dlindelof]

I’m always very nervous when I see an SQL query soi my a cross join on a set of “dimensions”. The opportunities for over counting are rife.

[dlindelof @dlindelof]

Another year, another attempt at getting the future package to work for me.

[dlindelof @dlindelof]

Currently playing with more modern alternatives to traditional R functions. skimr::skim() instead of summary() and the gtsummary package instead of the venerable tableone.

[dlindelof @dlindelof]

"There is an enemy. There is an intelligent, active, malign force working against us." (Steven Pressfield, Do the Work)

[dlindelof @dlindelof]

Today's lesson: always include an interaction between a treatment effect and the biggest predictor of the outcome.

[dlindelof @dlindelof]

Data catalogs like Alation are phenomenally useful to understand your data landscape but should NOT be part of your analysis code. Your queries, including possibly cached results, should be part of your main analysis code.

[dlindelof @dlindelof]

Most (all?) statistics textbooks that deal with online experiments seem to assume that the experimental unit is the user, but it's hard to believe this scales well.

[dlindelof @dlindelof]

Still working my way through Jennison & Turnbull’s textbook on group sequential testing. Got to the point where they describe tests that can accept the null early, which is critically important for me.

[dlindelof @dlindelof]

Asking a data scientist to size an experiment always feels like asking an R2 unit to prepare a hyperspace jump, but really it shouldn’t.

[dlindelof @dlindelof]

My current understanding is that traditional sequential testing like SPRT takes one measurement at a time and allows for accepting the null early, but group sequential methods like Pocock take groups of measurements but don’t seem to have a mechanism for accepting the null early

[dlindelof @dlindelof]

Reading up on group sequential designs and how they may relate to online controlled experiments.

[dlindelof @dlindelof]

Last week I’ve been exploring ways to break my bad habit of doing data science in two steps: 1) prepare data in one tool 2) work with data in another. I’m convinced it’s possible to do everything in one tool.

[dlindelof @dlindelof]

This is one interview question you won't have to worry about anymore: https://t.co/AVbvkxHElr

[Towards Data Science @TDataScience]

Precision alone can't give you a true estimate of prevalence, but recall and specificity can help adjust the predicted prevalence. Learn how to make this adjustment and get unbiased results from @dlindelof's article.

https://t.co/DlZR7MQMws

[Towards Data Science @TDataScience]

Even when a classifier isn’t perfect, we can still get useful prevalence estimates. By using recall and specificity, we can adjust the predicted prevalence without relying on precision, which can be biased by the actual prevalence. Check out how this works in @dlindelof's latest article.

https://t.co/DUsaBzhipE

[dlindelof @dlindelof]

Even imperfect binary classifiers can be used to estimate the true prevalence of some condition in a population. Here's how.

https://t.co/QtGMyJ81hs