There are 3 skulls you have to defeat before the nightmare spawns.
You start in a random location. The first thing you need to do is find the master key. It will be located in an obvious relic box in one of the main rooms that is not locked. Once you have the key, you will need to find the 3 skulls in these locked rooms. When you kill all 3 skulls, a big skull (the nightmare) spawns in the main lobby. I used a card for unlimited grenade ammo and a maxed out outfit with a maxed out bow. It was very easy. The 3 normal skulls die with one direct grenade arrow and the boss skull takes 3.
The skulls can be located in these rooms (list to be updated):
1)In the library.
2) In the west wing in the room with the paintings past the bedroom.
3) In the wine cellar. The door is in the same hall as the library.
4)The Hidden room in the wine cellar
5)In the West wing bedroom
6)In the hall outside the study
Boss) Main lobby
The boss may throw up a shield after one or two shots. I just shot a grenade arrow at the spawning enemies on the bottom floor and the mission completed but you may need to shoot him again after the shield is gone.