The motivation makes a difference. If my wife think I'm intentionally punishing her she'll react differently than if she thinks that I don't want to be affectionate because I feel hurt or angry.