The biggest risk from handholding is camera shake – you inadvertently move the camera fractionally while exposing and image sharpness is affected. The two main factors that influence this are how big/heavy the lens is and which shutter speed you use. The rule-of-thumb to avoid camera shake is to ensure your shutter speed at least matches the focal length of the lens – 1/60sec for 50mm, 1/125sec for 100mm, 1/250sec for 200mm, and so on. This doesn’t apply so much at smaller focal lengths (shooting with a zoom at 10mm doesn’t mean it’s safe to handhold at 1/10sec) but it’s a good rule for telezooms and telephoto lenses. The recommended shutter speeds also relate to the effective focal length, so for example, if you use a 24-70mm zoom at 70mm on
a DSLR with a crop factor of 1.5x, the effective focal length is 105mm so you need to use a shutter speed of 1/100sec or, even better, 1/125sec.
Obviously, we all differ as individuals – some are able to hold the camera rock steady at slower shutter speeds than others.