From the starting point of Goat Ville, if you walk forward you should see a fairly short cylindrical building to your left, with stairs that travel all the way around.
Travel up the stairs past the goat in the alcove and see a second alcove at the top of the stairs, enter this and you'll be transported to Goat Castle and a double door should open. From here walk in between the goats to the large throne at the back and jump on the seat.
Well done on your achievement Goat Queen!