Siavash Ghaffari

Data scientist | Single-Cell Omics, AI/ML, Computer Vision | PhD from @McGillu | OPINIONS ARE MY OWN

[Siavash Ghaffari @scSiavash]

@amitisinvesting @saylor 😂

[Nature Methods @naturemethods]

Introducing guided sparse factor analysis (GSFA), a statistical framework to detect changes in gene expression resulting from perturbations in single-cell CRISPR screening. @xinhe2 @MengjieChen6

OA paper: https://t.co/94lDl8VZQC

[Aurimas Griciūnas @Aurimas_Gr]

Did you know that you can visualize YAML, JSON and Data of many other formats in comprehensive and easy to navigate graphs directly in your favorite 𝗜𝗗𝗘 or in an 𝗼𝗻𝗹𝗶𝗻𝗲 𝗲𝗱𝗶𝘁𝗼𝗿?

Check out PlantUML.

Here is how you can take advantage of the tool:

➡️ Install PlantUML extension in your favorite 𝗜𝗗𝗘 - it supports 𝗣𝘆𝗖𝗵𝗮𝗿𝗺, 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗝, 𝗩𝗦𝗖𝗼𝗱𝗲 and many more.
➡️ Create a file with .𝗽𝘂𝗺𝗹 extension.
➡️ Add 𝗬𝗔𝗠𝗟/𝗝𝗦𝗢𝗡 content or one of many other supported formats enclosed in a specific decorators. e.g. @𝘀𝘁𝗮𝗿𝘁𝘆𝗮𝗺𝗹 <𝗰𝗼𝗻𝘁𝗲𝗻𝘁> @𝗲𝗻𝗱𝘆𝗮𝗺𝗹 for yaml.
➡️ See your diagram created and changed in real time as you type!

𝗧𝗵𝗲𝗿𝗲 𝗶𝘀 𝗺𝗼𝗿𝗲:

➡️ The graphs are interactive, each time you press on an object in the graph you get the source code representing the object highlighted and ready to be edited!
➡️ You can style the output graph using code!
➡️ You can save the graph as a picture in different formats making it easy to be used as part of documentation!
➡️ There is an online editor as well, link in the comment section.

𝗡𝗲𝘃𝗲𝗿 𝗴𝗲𝘁 𝗹𝗼𝘀𝘁 𝗶𝗻 𝘁𝗵𝗲 𝗬𝗮𝗺𝗹 𝗝𝘂𝗻𝗴𝗹𝗲 𝗮𝗴𝗮𝗶𝗻!

--------

Follow me to upskill in #MLOps, #MachineLearning, #DataEngineering, #DataScience and overall #Data space.

Also hit 🔔to stay notified about new content.
𝗗𝗼𝗻’𝘁 𝗳𝗼𝗿𝗴𝗲𝘁 𝘁𝗼 𝗹𝗶𝗸𝗲 💙, 𝘀𝗵𝗮𝗿𝗲 𝗮𝗻𝗱 𝗰𝗼𝗺𝗺𝗲𝗻𝘁!

Join a growing community of Data Professionals by subscribing to my 𝗡𝗲𝘄𝘀𝗹𝗲𝘁𝘁𝗲𝗿: https://t.co/qgNCnGtF4A

[Santiago @svpino]

StackOverflow is fighting back!

Here is everything you need to know about the release of OverflowAI, their new generative AI initiative.

StackOverflow traffic has been declining for several years, but after the release of ChatGPT, things took a turn for the worse:

Posting activity on the site fell the equivalent of five years in just six months! People stopped visiting the site and asking questions. Many thought this was a slow and painful death, and there was no coming back.

Today, their CEO announced OverflowAI.

If you can't beat them, join them, right?

But what's OverflowAI, and is this enough for StackOverflow to become relevant again?

Before I get into it, I want to thank Insta360 and their AI-powered smartphone stabilizer, Flow, for sponsoring this post. Flow is a gimbal with everything you need built-in, including a tripod and a selfie stick. Flow offers smart features like Deep Track, which remembers your subject even if they're obstructed or out of frame, or Insta360's automatic AI editing for reels made in seconds. It's also got an all-day battery life and can even charge your phone. To support my content, check Insta360's Flow here: https://t.co/0UkYt9euHx.

Here is StackOverflow's statement from seven months ago:

"All use of generative AI (e.g., ChatGPT and other LLMs) is banned when posting content on Stack Overflow. This includes asking the question to an AI generator then copy-pasting its output as well as using an AI generator to reword your answers."

OverflowAI is not a tool but a group of different generative-AI initiatives. StackOverflow went from banning AI to embracing it in over six months.

Some of the features they announced will be open to everyone:

First, they will replace their search functionality with one powered by generative AI. You can ask a question, and the site will search and summarize existing answers for you.

To power this feature, they use a vector database and embeddings of their 58 million questions and answers. It's a smart move, and it should improve the quality of the solutions we get.

Answers will cite sources, so authors get the credit, and users can always trace back the results.

Second, the assistant will help you ask a question if you don't find an answer. This should help people keep a consistent format and include as much relevant information as necessary.

The third interesting feature is a Visual Studio plugin to ask questions from your IDE. You can select a snippet of code, fire the plugin, ask a question, and get an answer without opening a browser.

I'm having difficulty seeing how a plugin that answers questions will fit next to Copilot. Will they complement each other?

I'm glad StackOverflow is fighting back. They promised the new search functionality by August.

But is this enough to keep them alive?

Do you think they have a fighting chance against ChatGPT and Copilot?

[Stack Overflow @StackOverflow]

Stack Overflow’s Visual Studio Code IDE extension will pull in validated content from both the public platform and a user’s private Stack Overflow for Teams instance to provide developers with a personalized summary of how to solve problems efficiently and effectively.

[Santiago @svpino]

Another deep learning breakthrough:

Deep TDA, a new algorithm using self-supervised learning, overcomes the limitations of traditional dimensionality reduction algorithms.

t-SNE and UMAP have long been the favorites. Deep TDA might change that forever.

Here are the details:

[Inflection AI @inflectionAI]

It’s a big week! We’ve raised $1.3 billion and are building the world’s largest AI cluster (22k H100s).

We’re grateful for our investors and new funding that will help us accelerate our mission to make personal AI available to every person in the world. https://t.co/l2MPlhgqVl

[Nature Medicine @NatureMedicine]

A machine learning algorithm developed to detect #occlusion #myocardial infarction with no-ST elevation from #ECG outperforms clinicians in diagnostic assessments. #MyocardialInfarction #Cardiology #DigitalMedicine
https://t.co/JnhHV284qV

[DAIR.AI @dair_ai]

Top ML Papers of the Week (June 19 - 25):

- RoboCat
- LMFlow
- AudioPaLM
- ClinicalGPT
- SequenceMatch
- Textbooks Are All You Need
...

[Nature Reviews Drug Discovery @NatRevDrugDisc]

Small-molecule discovery through DNA-encoded libraries https://t.co/xTB27Irfl2

This new review discusses small molecules discovered by using DELs, covering initial identification, optimization and validation of biological properties such as suitability for clinical applications

[Andrew Ng @AndrewYNg]

Had an insightful conversation with @geoffreyhinton about AI and catastrophic risks. Two thoughts we want to share:
(i) It's important that AI scientists reach consensus on risks-similar to climate scientists, who have rough consensus on climate change-to shape good policy.
(ii) Do AI models understand the world? We think they do. If we list out and develop a shared view on key technical questions like this, it will help move us toward consensus on risks.

I learned a lot speaking with Geoff. Let’s all of us in AI keep having conversations to learn from each other!

[Nature Reviews Genetics @NatureRevGenet]

RNA barcoding: the catalyst for the single-cell revolution https://t.co/mOFMGTzjnT #JournalClub by @VallotCeline @institut_curie

[Aurimas Griciūnas @Aurimas_Gr]

How is 𝗖𝗜/𝗖𝗗 𝗽𝗿𝗼𝗰𝗲𝘀𝘀 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁 𝗳𝗼𝗿 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗣𝗿𝗼𝗷𝗲𝗰𝘁𝘀 compared to 𝗥𝗲𝗴𝘂𝗹𝗮𝗿 𝘀𝗼𝗳𝘁𝘄𝗮𝗿𝗲?

The important difference that the Machine Learning aspect of the projects brings to the CI/CD process is the treatment of the Machine Learning Training pipeline as a first class citizen of the software world.

➡️ CI/CD pipeline is a separate entity from Machine Learning Training pipeline. There are frameworks and tools that provide capabilities specific to Machine Learning pipelining needs (e.g. KubeFlow Pipelines, Sagemaker Pipelines etc.).
➡️ ML Training pipeline is an artifact produced by Machine Learning project and should be treated in the CI/CD pipelines as such.

What does it mean? Let’s take a closer look:

Regular CI/CD pipelines will usually be composed of at-least three main steps. These are:

𝗦𝘁𝗲𝗽 𝟭: Unit Tests - you test your code so that the functions and methods produce desired results for a set of predefined inputs.

𝗦𝘁𝗲𝗽 𝟮: Integration Tests - you test specific pieces of the code for ability to integrate with systems outside the boundaries of your code (e.g. databases) and between the pieces of the code itself.

𝗦𝘁𝗲𝗽 𝟯: Delivery - you deliver the produced artifact to a pre-prod or prod environment depending on which stage of GitFlow you are in.

What does it look like when ML Training pipelines are involved?

𝗦𝘁𝗲𝗽 𝟭: Unit Tests - in mature MLOps setup the steps in ML Training pipeline should be contained in their own environments and Unit Testable separately as these are just pieces of code composed of methods and functions.

𝗦𝘁𝗲𝗽 𝟮: Integration Tests - you test if ML Training pipeline can successfully integrate with outside systems, this includes connecting to a Feature Store and extracting data from it, ability to hand over the ML Model artifact to the Model Registry, ability to log metadata to ML Metadata Store etc. This CI/CD step also includes testing the integration between each of the Machine Learning Training pipeline steps, e.g. does it succeed in passing validation data from training step to evaluation step.

𝗦𝘁𝗲𝗽 𝟯: Delivery - the pipeline is delivered to a pre-prod or prod environment depending on which stage of GitFlow you are in. If it is a production environment, the pipeline is ready to be used for Continuous Training. You can trigger the training or retraining of your ML Model ad-hoc, periodically or if the deployed model starts showing signs of Feature/Concept Drift.

Let me know your thoughts. 👇

--------

Follow me to upskill in #MLOps, #MachineLearning, #DataEngineering, #DataScience and overall #Data space.

Also hit 🔔to stay notified about new content.
𝗗𝗼𝗻’𝘁 𝗳𝗼𝗿𝗴𝗲𝘁 𝘁𝗼 𝗹𝗶𝗸𝗲 💙, 𝘀𝗵𝗮𝗿𝗲 𝗮𝗻𝗱 𝗰𝗼𝗺𝗺𝗲𝗻𝘁!

Join a growing community of Data Professionals by subscribing to my 𝗡𝗲𝘄𝘀𝗹𝗲𝘁𝘁𝗲𝗿.

[Andrew Ng @AndrewYNg]

1/Thrilled to announce: 3 new Generative AI courses!
* Building Systems with the ChatGPT API, with OpenAI’s @isafulf
* LangChain for LLM Application Development, with LangChain’s @hwchase17
* How Diffusion Models Work, by @realSharonZhou
Check them out: https://t.co/IN454k1Wz6

[Rowan Cheung @rowancheung]

RIP online job interviews.

This AI tool enables real-time transcriptions for your microphone input AND speaker output.

It then generates a response for the user to answer questions based on the live conversation:

[DAIR.AI @dair_ai]

Top ML Papers of the Week (May 15 - 21):

- DragGAN
- CodeT5+
- StructGPT
- Med-PaLM 2
- Symbol Tuning
- Evidence of Meaning in LLMs
...

[Rowan Cheung @rowancheung]

This week in AI just changed everything.

Massive announcements from OpenAI, Warren Buffet, Microsoft, AMD, Palantir, Tesla, Meta, Humane, IBM, Wendy's, Google, HuggingFace, Scale AI, Synthesia, Anthropic, and Stability AI.

EVERYTHING you need to know and why it's important:

[National Human Genome Research Institute @genome_gov]

When Human Genome Project researchers announced they had successfully completed sequencing the human genome, it was actually only about 92% complete. Now, researchers have finally got that last 8%! https://t.co/zZBpKbDKHd

#Maythe4thBeWithYou

[Modular @Modular]

The future of AI development starts here. Two incredible breakthroughs: the world’s fastest unified inference engine, and Mojo 🔥 a new programming language for all of AI announced by @clattner_llvm &amp; @iamtimdavis

[Andrew Ng @AndrewYNg]

1/ Thrilled to announce: Our new course ChatGPT Prompt Engineering for Developers, created together with @OpenAI, is available now for free! Access it here: https://t.co/OaIpa6L2jn