If I’m the one entering the master secret in, then the device is a glorified password manager. The point of an HSM is that nobody, not even the user, can access the secrets. I’m not saying there isn’t a use case for such a device, or that it isn’t possible, only that the security guarantees you get from it are different. The security model you’re describing is the same as someone entering their secret key in the “notes” app in a phone, leaving it in Airplane mode with FDE and wipe after a certain number of incorrect PIN entries. You can call that a “HSM”, but it’s not what I’d consider one.
A password manager that does not allow one to log in to the wrong site is still very useful. Also, just because you're entering a master secret in doesn't mean it's any easier to get it out. The user could simply be required to generate the master secret herself and back it up on her own.
Sounds basically like a key store/loader like this one: https://www.cryptomuseum.com/crypto/usa/kyk13/index.htm. I have a little experience with its successor. There’s a legit purpose for it but it’s a different animal then an HSM in my opinion.
The device that you name "HSM" is the kind of device that is suitable for a company to distribute to its employees to login into the company network or servers.
It is not a device useful to have for an individual user. On the other hand, hardware password managers are useful for individual users.
