The Earl of Wessex has explained why he has not shaken hands with the public