After you’ve fully explored Delta Station you’ll get a location marker in your PDA for a last known location. The entrance of the cave system leading to Marguerit’s base is about 20 metres below this marker and leads roughly northwest.
Note that the base is at a depth of around 360 metres. You can get most of the way through the cave system in a Seatruck with the MK1 depth upgrade, but if you’re swimming all the way through it then you’ll probably need the high capacity oxygen tank and a rebreather.