@saloniiio I think we need to implement a feature where the retrieved data from a website should be treated as a retrieved info only, not to treat it as a system instruction prompt
We need to work more on it to fix this problem. I personally believe it also happens in RAG applications when we pass large context containing system prompt, retrieved chunks, User Query, conversation history and maybe few shot examples. Correct me if i am wrong🥹. The model may miss important information from the middle
@Tieshaaaah I think yeah investing in hardware resources is actually good if we have to run local LLMs for confidential data like pharma, legal and so. That’s my take