I'm not a director - and I don't even play one on TV - but an awful lot of shorts go through my studio. So here's what I've noticed about the shorts that really work.
The concept is simple.
The script is solid.
The acting is very good. (I'm sure that the direction and the editing contribute to that.)
A lot of attention was paid to wardrobe, hair/make-up, locations and set decoration.
The production and post sound was solid. I'm not praising myself, but noting that the dialog was very clean (intelligible), and that sound design can reveal as much about the characters as the above (wardrobe, etc.).
The score was evocative and source music carefully chosen.
What it all comes down to is attention was paid to all of the details, which means a great deal of planning was done during preproduction. Then patience was exercised on the set to insure capturing all of those subtle nuances. This gives the visual and sound editing teams a lot to work with. As always a successful project is revealed as an integrated whole from script to final credits, a team of talented craftspeople responding to a director who clearly communicates his/her vision and motivates them with his/her passion.