First, it is impossible for that sponge to harm a coral. Second, it is very normal for sponges to grow in every part of your tank that is in complete shade. Third, it is actually a sign of a healthy tank and the sponges filter you water, another good thing.
The sponges will not live in bright light. No, it won't hurt your coral to expose it to air for a few minutes.
Now that you moved that candy cane so that it is in the light that part of the sponge will die quickly.