Wow, as cool as this is, it's kind of a shame that we need to say "use coords to show where the mouse should click" instead of designing interfaces that keep pointing-device-free users in mind.
With Windows in particular, you absolutely can navigate Windows + Office keyboard only. I do it every day.
Now, third party software, is always going to be all over the place. Stuff that was largely built on Win32 components works fine, but "modern" stylized applications rarely have strong support.
You’re right that lots of Windows apps were designed with Keyboard only workflows in mind. It’s a shame that MacOS has so many points where if you don’t have a mouse you’re out of luck.
There is one major improvement you can do on Mac, at least for menus:
I really feel like this used to be the default. That's how I always did it in macOS going back to the early 2000's.
Only in the last two versions or so did I notice it was no longer the default. I'm glad to see here that I can now re-enable it.
Edit: I see that I do have it enabled. But for some reason there are a lot of programs where it doesn't seem to work anymore, no matter what the settings. Off the top of my head: Half the Adobe programs I use for work.
I get that this might annoy you, but there is a direct trace all the way back to the original Mac in 1984 that required a mouse. As time went on and the two other OSes we still have gained mouse support (Windows, Linux) from their keyboard roots, they brought forward their ethos of keyboard navigation. Mac OS resolutely stayed attached to its mouse only roots.
If you're interested in keyboard navigation of websites, consider a browser or extension with link hinting support! It worked really well in my experience a few years ago, although I've since became much more of a mouse guy and stopped using it.
Qutebrowser was my favorite browser for keyboard navigation but firefox, chrome, etc. have extensions for this as well.
I remember using TweakUI to enable "always show underline for shortcut key" because that genuinely felt like it should be a default for better usability.
I wouldn't say they're dead, just more hidden (e.g. GTK4 only shows them when you hold Alt). AFAIK most toolkits still support them, but app developers also have to actually define them.
On macOS with "Full Keyboard Control" you can navigate the system and most any official app from the keyboard. It's not an efficient experience though.
> designing interfaces that keep pointing-device-free users in mind.
Agreed. Using keyboard keys to emulate a mouse cursor seems like it ought to be a last resort for graphical applications that lack proper accessibility affordances.
Contrast that with command palettes, accessibility controls, syntax tree navigation, and other approaches that rely on the names, content, and document structure that users already know rather than a special mode that displays two letter codes that must be read each time or memorized. Many of these other approaches also allow users to activate buttons, menu items, and links that are outside the current viewport or hidden in menus which reduces the overall number of "clicks" required to perform those actions. The downside is that they can take longer to type than a two-letter code. Still, my guess is that for most people it would be overall more efficient to optimize for cognitive load than pure speed.
(Though in the long run, I suspect that improvements in eye-tracking will lead to hybrid systems that are both lower cognitive load and faster than any of these.)
I just hit tab 1 to N times and hope for the best. I wonder if VIM style search on elements with a new HTML tag attribute would work (at least for browsers).
It would be great if they built Vim style shortcuts into the spec and browser like you suggested but in the time being we have the Vimium extensions for other browsers. Personally I am not a fan of extensions unless I write them myself.
I'm curious if there's a program that uses a simple detection model for UX components to locate clickable areas. This would allow for global navigation similar to VimiumC
I think it's ok that hardware and software are designed with the 99% in mind. After that you probably run into competing interests/trade-offs anyway (a system built for ergonomics probably looks different from a system built for speed).
I think it's ok that hardware and software are designed with the 99% in mind.
That's called mob rule. We don't act like cavemen anymore. We build entire civilizations to prevent that sort of thing. You may have read in a history book once "We hold these truths to be self-evident, that all men are created equal."
I think the controller interfaces for FFXIV is worth a study in this. They designed an interface that is workable for an MMORPG with both mouse and controller (in this case, the controller can act as a proxy for our keyboard).
