There is really no physical way to tell if a boy is a virgin.
At this point I also need to say that it is also not always possible to tell with finality if a lady is a virgin. There are many daily activities that could result in breakage of the hymen despite not having had sexual intercourse. They include use of wrong size tampons, certain physical activity like horse ridding or even competitive bicycle riding.
At the end of the day for either gender it comes down to just trust that either party is telling the truth about their sexual experience history.