You do not need to raise your Bounty level so you get attacked by mercenary ships. You will ultimately get this when you kill the Cultist named Asterion, of the Gods of the Aegean Sea branch.
He is stationed on an epic ship in the water between Skyros and Andros islands. (at least he was for me) You might have other cultists from this branch available who may also be on ships.
You can defeat his ship and sink it right away, OR board it and defeat him in open combat, confirm his death, then loot the chests and the ship will sink.
This video from Caravan Kidd on YouTube shows the battle with Asterion