We were at css back in 2002 and we are trying to remember what hotel was close by.I remember you could see it from the nude beach and their boat would come by very slowly with a bunch of people staring.
Sandals and Royal Plantation are nearby. Their boats would come by periodically. But as for the very colorful buildings easily viewable from SSB, I'm not sure what that was or if it's even a resort. I know that's not Sandals. Sandals OR is in the next cove and you can see it from the very end of SSB (to the right).
The last few years when we were at SSB the Sandals Resort boat would come by. They would do exactly what you described. Slow down and let the people stare. We all usually just wave, we know they realy want to be on the beach with us.
"Once you go...you know."
One year when we were at CSS the S boat kept coming by and slowing down. Finally one day my husband thought he would really give them a show, and got up on the stump, turned aroun and full mooned them. They didn't seem to stop as long after that.