Want to implement chain-of-thought reasoning in your text-to-SQL tool to help guide the SQL query generation? One way is to provide few-shot examples demonstrating CoT. @langchain makes this a piece of cake. (1/4)
Need help finding something! It's a cartoon drawing of a monster (representing LLMs) and I think a lasso or maybe a window that the monster was being squeezed through (representing prompt engineering). I'm pretty sure I saw it on Twitter a few weeks ago? Anyone remember this?
With this, each prompt can be individually adjusted as needed for each SQL dialect without affecting the others. PR’s and suggestions very welcome from everyone!
@JonZLuo added specific prompts for MSSQL, MYSQL, MariaDB, Oracle, Postgres, and SQLite
PR: https://t.co/228OSlWYZf
Huge shout out to @JonZLuo - he's already done a ton to push the capabilities of the SQL chains, and this just continues that 💪💪💪
Announcing the third batch of AI chatbot experts sharing insights during the upcoming "Build a ChatGPT Chatbot For Your Data" program
AI SQL experts:
NLP Practitioner, @fpingham
Bioinformatics Analyst, @JonZLuo
Semantic search expert:
@transitive_bs
Learn more below...
Really excited to publicly announce that:
1⃣ @ankush_gola11 and I have started a company around @langchain
2⃣We raised a seed round from @benchmark
We wrote a bit about the journey so far, and where we want to take it:
https://t.co/SoCfGwEcWm
Want to implement chain-of-thought reasoning in your text-to-SQL tool to help guide the SQL query generation? One way is to provide few-shot examples demonstrating CoT. @langchain makes this a piece of cake. (1/4)
@jxnlco@langchain Sorry for the late reply - not that I know of for CoT. Also thinking about the best way to go about it. I think we need to be deliberate in selecting the subset of questions/queries from spider for benchmarking.
@monitus@langchain Sorry for the late reply! SQLGeneratorChain is just a chain I made that generates SQL code but does not execute it. The SQLDatabaseChain in LangChain executes the query after generating it.
@socialtippers@langchain I think they're complementary. What they've demoed so far is very cool but you need to upload or connect your data, which may be undesirable. Generally with your own tools, you can make more bespoke solutions and also work without ever sending your data to a third party.
@socialtippers@langchain Hey, thanks for the kind words and sorry for the late reply. I think a lot of the heavy lifting in understanding complex joins comes from providing the few-shot examples. Zero-shot doesn't work as well - however, I was using Codex which doesn't do great zero-shot txt2SQL anyway
While it may underperform in zero-shot tasks (as seen in @ekzhu's cool Spider evals), I found it to be excellent in the few-shot setting for text-to-sql, although I don't have a formal eval. I'm needing to wrestle a lot more with the other models for my app. RIP Codex😢(2/2)
An underappreciated thing about Codex imo was its context length. With 8k context len, you could provide many more few-shot examples than other models. In addition to being able to feed it more examples, it had stronger in-context learning ability: https://t.co/l7ltS5fS7D (1/2)
Farewell to OpenAI Codex. Codex's code‑davinci‑002 was the best performing model in the GPT-3/3.5 line for many tasks. Despite its name, it excelled in both code and natural language. It will be missed.
Left: OpenAI email. Right: From Yao Fu 2022 https://t.co/iLs2eKGpGi
@mayowaoshin@langchain sky's the limit! If appropriate for your data, you could have the chain/agent execute the query to interpret the results or to decide on the next action, use a tool to generate a plot, etc
@mayowaoshin@langchain Sorry - examples are injected into the prompt which isn’t shown in the output here. Green is the generated SQL for the input question (orange). Blue is from a chain that selects only the tables needed to answer an input question so we don’t clog the prompt w/ unneeded table DDL
...and that's all there is to it. This can help the LLM navigate unruly queries and is more interpretable for the reader. There is also ongoing work to implement zero-shot CoT for SQL in @langchain. Exciting times! (4/4)
Annotate your examples with CoT, demonstrated here in a block comment as mentioned in the prompt. I manage my examples in a YAML file. Provide these examples to the prompt as described in the docs: https://t.co/yunT4MzPQD (3/4)