Well, I'm glad I'm not the only one who finds this to be a bit of a conundrum. I can take my hand off the handlebars and signal with it so people know what I'm doing (haha, like anyone understands hand signals anymore) or I can maintain full control of my moped and brake levers by not signaling.
Mostly I compromise. When there aren't other cars around, I don't signal. If I think I will improve the safety of the situation, I signal, but only if I'm not going to have to stop in the next 20 feet and there aren't potholes or obstructions in the road.