To be able to access the Graveyard, you'll need to have gained all runes. (The Spider Rune is used to scale the wall to access this place)
After you beat Conjunctivius in the insufferable graveyard, instead of jumping down to Slumbering Sanctuary, there's actually a wall you can scale upwards. It's above the platform of the Slumbering Sanctuary Portal.
Prisoners Quarters -> Toxic Sewers -> Ancient Sewers -> Insufferable Crypt -> Graveyard
I've linked a video below as well as a full playlist showing you how to get all areas/runes. (If you need to get Spider Rune as they are all in order)