Top Tweets for #rageval

RAGEval Proposes a simple framework to automatically generate evaluation datasets to assess knowledge usage of different LLM under different scenarios. It defines a schema from seed documents and then generates diverse documents which leads to question-answering pairs. The QA pairs are based on both the articles and configurations. It seems that this benchmark can help with more reliably evaluating the knowledge usage ability of LLMs and avoids confusion regarding the source of knowledge (parameterized or retrieval).