Why would he say this?

Usually it's those so called "nice guys" that try to tell you that every guy is an ******, player, etc. If you trust your boyfriend, don't listen to him or anyone else. He'd never be able to tell just by looking how your boyfriend was. No one can tell that. Don't read anything into what he's saying other than he's probably insecure and jealous.