Before going for this achievement, I recommend creating a new save file that you can return to after you have unlocked it. But, that's completely your call.
To be able to spend money on crime bribes, you need to have a guard approach you after you committed a crime. Crimes include looting items that are red, pickpocketing and striking innocents/guards.
Once you have accumulated 10,000 gold (or a little extra) return to Gorhart and swing one of your weapons at one of the guards standing around. When you do, you will get the option to enable Aggressive Mode by hitting

. Then, strike a guard. That guard will then immediately confront you and give you the option to (a) go to jail, (b) resist arrest or (c) pay a fine. Choosing to go to jail will let you get:
But, for the Crime Doesn't Pay achievement you are going to select to pay the fine, which should be around 900 gold. Once you've paid, the guard will start to walk away and you will still be in Aggressive Mode. Hit the guard again, pay the fine, rinse and repeat until you've spent 10,000 gold on bribes.
Depending on your level, eventually you may kill the guard, which will result in a roughly 3000 gold bribe, which is why it is best to have a little extra as a cushion.
If you have extra gold (35,000 - 40,000) you can keep doing this on repeat to also get: