Stephen Macke

Large-scale ML infra and tools for interactive data analytics | Maintaining and | Previously @IllinoisCS, @ucbrise

[Databricks @databricks]

#DBRX, a new general-purpose LLM, is now available on Databricks Marketplace!

Enterprises can use DBRX in RAG apps or to build a DBRX-style MoE model with their own data. Check it out to advance your AI initiatives. https://t.co/BPe8oFcsHz

[Chris Holdgraf @choldgraf]

Wow, @ProjectJupyter is the winner of the White House OSTP "Technical Advancement to Enable Open Science" category.

Amazing to see recognition of the project at a national level! Makes me proud of the ecosystem of interconnected tools we've built.

https://t.co/EQapI37In7

[Stephen Macke @stephen_macke]

@akshaykagrawal This is a pretty old issue, I believe stemming from changes to support top-level await (every module level statement now runs in it's own function). Also ran into it when trying to improve support for pdb in ipyflow. It doesn't manifest for breakpoints inside of functions.

[Stephen Macke @stephen_macke]

@sh_reya You should do it. Totally valid as long as it's defined somewhere (e.g. Definition 1.2).

[Stephen Macke @stephen_macke]

Excited that interactive dataflow and reactivity continue to gain traction with interfaces for data and scientific computing! Amazing how fast the folks at @marimo_io have made their tool one of the best out there for Python; lots of lessons to bring to https://t.co/XTvzI4fm6d

[Akshay Agrawal @akshaykagrawal]

We've added an inspiration section to our README: https://t.co/6JFLGHS8hc

@marimo_io was inspired by @PlutoJL, which in turn was inspired by @observablehq. We've also remixed ideas from other reactive programming tools like @stephen_macke's ipyflow and @streamlit.

[Stephen Macke @stephen_macke]

@davidbrochart I like the black background and incorporating brackets and/or an execution counter somewhere, but I think filling the space at the top and bottom adds too much noise. What do you think about [jp]term?

[Aditya Parameswaran @adityagp]

Also in the same session at 10:30, @sh_reya and @stephen_macke will present their work on making computational notebooks more production-ready, by applying program slicing techniques. Now you don’t need to compromise between experimentation and rigor. Take note, notebook cos.!

[Stephen Macke @stephen_macke]

@thorstenball I have spent a long time trying to address this with https://t.co/XTvzI4fTVL

[Will McGugan @willmcgugan]

I have a feeling these three sentences are not enough to prevent some nasty bugs.

If you don't save a reference to an asyncio Task it could disappear at any time. Your code would stop running, but likely only when your app is under real load.

Good luck figuring that out.

[Ponder @ponderdata]

📣 Introducing Ponder: Run #pandas on 1TB+ DIRECTLY in your data warehouse 🚀

Learn more below! 🧵[1/N]
https://t.co/LgowCiNQId

#python #datascience #AI #database

[Stephen Macke @stephen_macke]

@sh_reya @drewmikehead @sarahchasins @adityagp @sh_reya I still remember when you noticed the opportunity to improve notebook slicing by leveraging the runtime session information and how mind blown I was by the initial results. A simple idea with fantastic execution, and I couldn't be prouder to have been part of it.

[Shreya Shankar @sh_reya]

Notebook ML workflows have a lot of exploration & data wrangling, mixed w/modeling. Relevant artifacts (dataframe, model) are only a sliver of the workflow. So @stephen_macke & I developed nbslicer, a dynamic slicer for Python notebooks, built on ipyflow (https://t.co/BRiKz5DyfL)

[Stephen Macke @stephen_macke]

ipyflow 💖 ipywidgets

Just put the finishing touches on the integration and really happy about how it turned out.

Always felt like ipywidgets should be able to trigger cell execution in response to, e.g., slider adjustments -- with https://t.co/XTvzI4fTVL, it's finally here!

[Hamel Husain @HamelHusain]

This demo https://t.co/KjlEFQ6BM2 blows my mind:

It stores the entire execution/dependency graph of variables in a notebook and snapshots it according to your specifications.

It think that this could be used to build a better notebook that allows for instant time travel etc.

[Stephen Macke @stephen_macke]

Probably will have to try running this again with a larger sample :)

[Stephen Macke @stephen_macke]

For computational notebooks with reactivity, the ordering of reactively executed cells should proceed

[Stephen Macke @stephen_macke]

@untitled01ipynb Can confirm, am going to hell for perpetuating with https://t.co/XTvzI4fTVL

[Stephen Macke @stephen_macke]

@benjamin_bolte Nope but definitely agree that we're long overdue for a copilot-like experience for open source Jupyter notebooks / lab. A direction I'm really interested is to include the runtime dataflow state in the LM prompt to see if it improves code suggestions...

[Stephen Macke @stephen_macke]

Reactive execution in https://t.co/XTvzI4fTVL with ctrl+shift+enter / cmd+shift+enter is 🔥🔥🔥

[Stephen Macke @stephen_macke]

How to read a micro SD card in 2023