Man says he refuses to give up his seat for pregnant women because of long working hours