You would totally hear it, as a small click. I had a bug in some audio software I was working with that did almost exactly this, and it sounded terrible.