Embryos form nipple buds at 4-6 weeks—before the Y chromosome triggers testosterone and male development. By then the structures are already wired in. Evolution doesn't delete harmless leftovers with zero cost. Male nipples are basically developmental spandrels. They even have nerves.
Nice chest.