As a teenager I took a drawing class (mostly so I could learn to draw Pokemon) and I remember doing a study on hands at one point based on some characters from Dragon Ball Z.
And man, it was the hardest thing it that class. With faces, once you get a face “right”, you can make small adjustments to make the mouth/eyes open/close, but hands… if your character makes any sort of gesture BOOM now you’re drawing a completely different shape.
Between the number of joints, their range of possible rotations, and the angles they can be seen from, hands are probably the most complicated parts of our bodies that are visible from the outside. It’s completely unsurprising to me that these networks have trouble encoding them.
I was trained in classical animation so I've drawn a lot of hands. It's difficult for me to understand how any AI can produce real hand images.
It's not the number of joints, it's not the articulation... it's the relationship of the hand to the skeleton, to the gesture, and to the objects with which the hand interacts.
It's great that midjourney can now draw raised hands doing nothing or anime girls holding their hands in a mannerist pose but that doesn't address the real issue. Hands are intentional and laden with tiny muscular efforts that we're primed to perceive.
When AI draws a tree we aren't expecting each branch to interact perfectly with a cradled object. It's all arbitrary.
I wouldn't be surprised if "AI hand touch-up" becomes a specialist skill for the next five years or so. I don't think the hand issue can be addressed until new models are devised that invest more semantic consideration into a scene.
I pose 3D characters in Daz 3D and completely agree. Getting a rigged hand to hold an object such as a wine glass or mobile phone is virtually impossible to do realistically by guesswork. I usually have to hold a real object myself to understand what is going on. With experience I am learning some common patterns but I find there is no substitute for the 'hold-it-yourself' principle.
What's really interesting is that it's something both extremely hard to get right, and also super easy to diagnose as "wrong" when not done properly: you need a lot of training to design convincing hands but anyone can judge you for bad hands.
Yes. This is actually why life drawing of models is such a good exercise. You can draw a tree or a house badly in the sense that it doesn't (say) have the proportions of the original but it can still look okay to someone who hasn't seen the original. But draw a person with their eyes too close to the top of the head and it immediately screams 'wrong'
It would be useful if the AI could identify the figure and impose a predefined 3D framework or skeleton as constraints for drawing. Like if one wanted to generate a human or animal there would be such a rig in place. That skeleton would in turn constrain joint rotation, proportions, and obviously the number of fingers.
I can only speak for SD but I've had some success using img2img on a CG or hand drawn figure to get the correct pose. The downside of that is that you have to use a low strength value to ensure that it actually follows your image.
>we aren't expecting each branch to interact perfectly with a cradled object.
We are. Sometimes if its subtle and camouflaged it slips past us.
>It's all arbitrary.
Its not. You are right there are probably fault modes of AI we don't notice most the time, and fault modes that bother us a lot. But its not arbitrary. We are better at noticing certain things aims more than others because that's what we evolved to see.
It’s not for naught that there’s a common meme around AI hands https://imgur.io/tf43ecd?r (Alt text: Human asks robot “can an AI draw hands?”, robot counters “can you?”.)
I remember during the original AI art arguments, an artist friend semi-jokingly remarked to me that AI can’t draw hands because there aren’t enough examples in the training set, because artists go to great lengths to choose poses that hide the hands since they can’t draw hands either.
I heard a great quip about the way people misreason about AI capabilities.
Announcement: "AI can play chess!"
Public: "People can play chess. And AI can play chess. People can also X. Therefore AI can also X."
... Ignoring the underlying nature of the chess problem and how it was different than other problems' structures.
Hands are the same.
You can image-bash together faces from examples and get something mostly-right simply through pattern copying.
You cannot do the same with hands, because rendering them plausibly requires at least intuition and approximation of inverse kinematics -- something the recent set of image generative AI didn't include.
Which isn't to say it can't, simply that the "hands problem" is unlike "the face problem."
“ You can image-bash together faces from examples and get something mostly-right simply through pattern copying. … You cannot do the same with hands, because rendering them plausibly requires at least intuition and approximation of inverse kinematics”
Not sure I agree. I think hands probably just require ~1 OOM more image-bashing than faces, and training sets had ~1 OOM less samples of hands than faces. E.g. faces need 1 trillion cumulative Tflops while hands need 10 trillion cumulative Tflops, and because there were more faces than hands in the training set, by the time we reached 1T on faces we had reached 0.1T on hands. (numbers made up)
yep, drawing hands well requires an understanding of the underlying anatomy that is more apparent than most other salient features
hands are obvious to most people, but there would be many features that an AI would require a vast training set to completely capture, but that humans would also miss most of the time
for instance look at Michelangelo's Moses, it was sculpted with models and with a very thorough knowledge of anatomy by the artist, and includes details like the muscle of the forearm that contracts when someone lifts the pinky finger:
what are the chances that, for instance, the average person would notice that detail missing without being told about it? let alone reproduce it generatively, representing an imaginary person
More importantly, there are so many examples of drawn art which intentionally have the wrong amount of fingers that it would make all sense for a model to learn that non-photographic humans may easily have less fingers.
Yeah. Unfortunately the only way I found to do it convincingly and reliably was the manga way: draw a pentagonal shape for the "wireframe" of the palm, then draw five lines for the wireframes of the fingers (pose them as you need and make sure to place the thumb on the side of the palm, please), then flesh them out. Try not to make them look like sausages (i.e. draw them tapering towards the ends). Draw nails if you really must. I like to draw little lines on the knuckles and inside the palm a little "Y" shape.
That fudges a lot of complexity, but I was only interested in cartoon-like hands, more expressive and evocative, than anatomically correct. So, you know. Manga hands.
(like Jazz hands, but with bigger eyes).
It gets more complicated if you want the hands to be doing stuff. For some reason, one of my favourite themes was someone tapping furiously at a keyboard. Go figure.
I also did some classical animation like StrictDabbler below but I never had to draw any hands, let alone animate them. That would take special training I reckon, you need technique to do that, you can't just intuit it or it'll look horrible.
I don't know what all this has to do with image generators though. They do not crate images like we do. I don't understand why they 're not good with hands, to be honest.
I guess the human form is much more ah, surprising and irregular, than we realise. I think, to an alien, we'd look pretty freaky just like we imagine cephalopod-like aliens to be. "AAAAH what are those tendril-like appendages sticking out of them?!!"
Anyway, for me the hardest part of all was perspective. Now that is some tough shit. But that, too, you can learn, if you're shown the right technique. Allegedly.
It must be the combination of the complex, dynamic shape, and our high sensitivity to hands that look ‘off’. There are many other things that are hard to draw accurately but where we are completely convinced by very cartoonish representations.
That gives hands a very wide uncanny valley that is hard to cross.
Surely this is because hands are one of the most versatile and useful parts of the human body? We probably have a lot of brain cycles dedicated to modeling them.
Just under half your bones are in your hands + feet alone, adding in the degrees of freedom provided by the wrist/elbow/shoulder on your hand position and I can see why it would be difficult to get right.
way way back when I was in art classes, the hardest part for me was hair. everything else for me would look acceptable slightly better than a 5 year old, but the hair was never better than stick figure at best. i remember trying to draw a portrait of Robert Smith from the Cure. the hair, ugh
As a teenager I took a drawing class (mostly so I could learn to draw Pokemon) and I remember doing a study on hands at one point based on some characters from Dragon Ball Z.
And man, it was the hardest thing it that class. With faces, once you get a face “right”, you can make small adjustments to make the mouth/eyes open/close, but hands… if your character makes any sort of gesture BOOM now you’re drawing a completely different shape.
Between the number of joints, their range of possible rotations, and the angles they can be seen from, hands are probably the most complicated parts of our bodies that are visible from the outside. It’s completely unsurprising to me that these networks have trouble encoding them.