Man says he gave his wife permission to sleep with other men - and immediately regretted it