When I finished the game, I had only around 20 gems found. Clad master gave me a tip that seems to be by far the best solution to farm gems:
Start level 1-5, it's the one that has a giant axe swinging right at the start of the level. There's a chance for a gem just before the axe. A second gem can be found after going bellow the axe and to the left, right above a small poll of lava.
After that, just hit start and restart the level.
Gems are completely random. You might try 10 times and not find a single gem on those two spots and you might find 3 in a row. Just keep at it and you'll get the achievement in around an hour or so.