To enter Mangar's Tower you must first complete Kylearan’s tower and obtain the key. Then you need to return to the Wine Cellar and work your way through the sewers to eventually find an exit to the outside of Mangar’s Tower.
The easiest way is to enter the Wine Cellar then immediately cast Apport Arcane and enter the coordinates 16N, 17E and -3 Up. You will be teleported to the exit stairs that lead to Mangar’s Tower.
Side note: If you leave Mangar’s Tower to return to town the gates will close behind you and you’ll have to re-enter via the Wine Cellar. There is a man on the 3rd level of Mangar’s Tower (at 12N, 19E) that will sell you a key for 50,000 gold that will then allow you to open all gates around town.