This is a little luck based but I have a couple of tips to help get this.
Firstly, always try and keep all the gems in one big group. If gems get isolated then you have little or no chance of matching them up.
Secondly, matching groups of 4 or more gems is helpful for clearing them quickly but this can leave isolated gems so do so carefully.
Finally always try and be mindful as to which direction gems will move once you make a match and try and plan a couple of moves ahead.
It took me several attempts and I ended up with 7 gems left a lot of the time but a little perseverance and luck should get you there. :)