Any algae that is growing ON a corals skeleton is generally bad because it can cause the coral to recede or take over. That looks like a type of turf so it would be best to remove it.
You should be able to pull it off with your fingers. If there's still some attached to the skeleton, get a small screw driver or something and chip away at the base. GL.