Marco Tulio Ribeiro

[Marco Tulio Ribeiro @marcotcr]

Testing LLMs (and prompts) like we test software: https://t.co/bZlIhgZEFh
TL;DR: (1) You should, (2) How to test: specific properties, evaluate these with LLMs (perception is easier than generation), (3) What to test: get the LLM to help you figure it out.

[Daniel Gross @danielgross]

Microsoft open-sources a new AI library that connects to open-source GPTs, not just OpenAI. https://t.co/wxrBtHCmIC

[Marco Tulio Ribeiro @marcotcr]

@sean_lynch We're just writing stuff WE would want to use, and I guess we probably count as 'real developers' :)

[Andrej Karpathy @karpathy]

Also highly relevant: guidance from microsoft
"Guidance programs allow you to interleave generation, prompting, and logical control"
Also internally handles subtle but important tokenization-related issues, e.g. "token healing".
https://t.co/eEc1rywuWP

[Clive Chan @itsclivetime]

been reading the readme for https://t.co/v1qsPSXiXJ, kind of galaxy brain

tl;dr they made a whole prompt engineering language

[Marco Tulio Ribeiro @marcotcr]

Blog post: playing with Vicuna-13B, ChatGPT (3.5), MPT-7B-Chat on harder stuff https://t.co/u2YQEuP6rV
TL;DR: We think ChatGPT is still way ahead, but sometimes the extra control from open source models is worth it.

[Marco Tulio Ribeiro @marcotcr]

My intern is close to writing a paper, so I wrote her this blog post on writing (part 1 of 2):
https://t.co/AoMgl4YHLi

[Marco Tulio Ribeiro @marcotcr]

I never tweet, but here is a blog post I wrote for an intern, may be useful for others too...
Part 1: https://t.co/DhE55I7ie7
Part 2: https://t.co/ncpJTb4otF

[Marco Tulio Ribeiro @marcotcr]

@jacswork My guess is you mean LIME : ). I don't know exactly what you mean, but we have follow up work coming out soon!

[Marco Tulio Ribeiro @marcotcr]

@BecomingDataSci Twitter is too hard : )

[Marco Tulio Ribeiro @marcotcr]

@BecomingDataSci Would love to hear what went wrong if you are willing to share in detail. Would you email me at my handle @gmail.com?

[Marco Tulio Ribeiro @marcotcr]

@BecomingDataSci I can share a few additional text or tabular examples if you like, but they all require specific datasets. We have a ton.

[Marco Tulio Ribeiro @marcotcr]

@BecomingDataSci Which ones didn't work? The first three should work, so please let me know if there are bugs =]

[Marco Tulio Ribeiro @marcotcr]

"Why Should I Trust You?" Explaining the Predictions of Any Classifier.
Promo video: https://t.co/Eu4lwhFZrU #kdd2016 @guestrin @sameer_

[Marco Tulio Ribeiro @marcotcr]

@fmailhot Heh, sorry for opaqueness of my replies. Twitter is not really my thing. Feel free to email me further questions or comments: )

[Marco Tulio Ribeiro @marcotcr]

@fmailhot Good point about opaque features. If the classifier uses stopwords, LIME should reflect it, so I don't think LIME should remove it

[Marco Tulio Ribeiro @marcotcr]

@fmailhot Probably don't need LIME for that though, if it's only 5 tokens. Things change in a longer sentence (not even a long document).

[Marco Tulio Ribeiro @marcotcr]

@fmailhot That is true (only 32 data points), but you may still want to tease out the contribution of each token. e.g 'I do not like that.'

[Marco Tulio Ribeiro @marcotcr]

@fmailhot It works with documents of any size. We also just added support for tabular (numerical + categorical) data. Maybe images soon.

[Tianqi Chen @tqchenml]

Great work from @marcotcr , @sameer_ on explaining any machine learning model (20 newsgroup, deep net). https://t.co/OClODgse0z @guestrin