In the interest of making as much of a one-stop-shop as possible apart from collectibles, I've compiled all 28 achievements that aren't related to the two big sets of collectibles into one video.
For A Story About My Goat, head to the garage in Goat Bay City and lick the crystal on the left, near the writing on the wall that says 'Do Not Lick'. It is floating in what looks kinda like a teleporter pad from Star Trek.
This can be seen at the 23:14
mark in the video below.