Gems are found in the levels as well as by breaking pots, killing enemies and opening chests. This achievement requires you to collect a total of 10,000 over all playthroughs so no need to try and get 10,000 at once.
Make sure you start upgrading the Extra Gems from Kills skill as soon as you unlock it. This will help increase the gems you get and speed this up. You will need a total of 50,000 for a different achievement so this will come naturally along the way.
I unlocked this before even completing the 2nd World so you will probably find it fairly quick to get. My successful run of World 1 netted me over 4,000 gems so if you prefer to farm it by redoing the easier levels you can do that easily enough. Happy hunting!