Just press or hold A when you walk past people, as has already been said, however if you want to make life a little easier start doing this right from the start of the game.
In the early stages you don't have a "notoriety meter" to worry about so there is no penalty for picking pockets. It'll earn around 10 to 20 florins per pocket which adds up pretty quickly if you concentrate on groups and spend a little more time on the ground than the rooftops.
I'd hit this before the end of sequence 1 without any trouble.
[Edit] If you've already got too far in the game use N0T0RI0USnFaMuZ's solution (most likely above) for bribing and pickpocketing Heralds @ 500f a time.