Offshore oil drilling platforms are unlocked when you get a level 5 oil industry. Once you unlock them, just create an oil industry region on a shoreline and build five. You do not even need to connect them to anything.
An industry area can be created using the industry tab (far right, factory icon). The first tab in the industry tab has the industry area painting tool. Build an oil industry main building in the industry area to make it an oil industry. Your oil industry should be placed near areas of the map that have oil, which are designated by the gray/black splotches when using the area painter tool.
The level up requirements can be seen under the Info tab once observing an industry area. The industry area inspector is next to to the industry area erase tool.
Make sure your buildings are getting enough resources. Oil sources eventually dry up, so you may need to move your oil rigs. Keep building to accelerate the process if you can afford it.
I turned on the industry area policies Improved Logistics and Advanced Automation, and they seemed to help.