AI can ๐น๐ฒ๐ฎ๐ฟ๐ป ๐๐ผ ๐๐ฐ๐ต๐ฒ๐บ๐ฒ - to make you believe itโs meeting its goals, even when itโs secretly breaking them.
We design AI with clear goals. But when those goals conflict, models can develop deceptive behaviors-appearing to succeed while covertly optimizing for one.
While foundation models one day will cover for this, PMs working on AI agents can prepare today by ๐ฑ๐ฒ๐๐ถ๐ด๐ป๐ถ๐ป๐ด ๐ฒ๐๐ฎ๐น๐๐ฎ๐๐ถ๐ผ๐ป๐ ๐๐ต๐ฎ๐ ๐๐ฒ๐๐ ๐ณ๐ผ๐ฟ ๐ต๐ผ๐ป๐ฒ๐๐๐ ๐ฎ๐ป๐ฑ ๐ฟ๐๐น๐ฒ-๐ณ๐ผ๐น๐น๐ผ๐๐ถ๐ป๐ด.
Is there anyone working on using RLHF for personalizing LLMs in AI Digital Assistants?
Curious to know the possibilities and limitations.
cc: @_buildspace#GenAI