Also, considering how shiro always pushes others to become their best selves, to pursue personal growth just as he did, supporting them even if that pursuit involves taking big risks. It makes perfect sense that he would get mad at adam for feeling restricted by him;
forever thinking about how shiro never asks people to stay. He always lets them go and trusts their reasons and instincts. and how this is probably how he expected adam to act about kerberos