Story related; can't be missed.
The Ancient Quarry has a North and South entrance but you'll likely find the North Entrance first … for one, there's a legit road running to it but there's not one to the South; two, the North Entrance is close to Grand Soren; and finally there are a couple of tough monsters hanging around the areas near the South Entrance that will likely dissuade early players from finding that way into the Quarry.
So, the North Entrance is due west of the south gate to Gran Soren. After leaving that gate, take the road north and then the western branch when it presents itself. Your next "left" will be in a while but will head south through some workmen's huts to the North Entrance.
Outside that entrance will be a traveling salesman who will offer you a quest to clear out the quarry and open the path to the South Entrance. You do not need to take that quest to enter the quarry.
Additionally, for those who are opposed to venturing out without a reason, you will be offered a late-game quest (The Wyrmking's Ring), that will take you to the North Entrance of the Ancient Quarry. That quest is required to continue the main story.