I just found out what they really are after an hour of searching, it isn't even a hydroid, it is a snail! Here is what I found on another forum site. It must be what you have also because they mention that fish also in here and the description of these snails is exactly what I have. Mounds and all. I may have to get a hermit crab now.
Vermetid snails are relatively diverse; over a hundred species have been described, and some of them are commonly found in aquaria. Although several species are found occasionally in our systems, generally entering on live rock, one variety in particular may become very abundant, and be a serious nuisance in some systems. This species, probably the most common, is small, with a brown, reddish, or purple shell. Interestingly, the animal is difficult to identify, although that has not stopped numerous reef aquarists from doing so. It probably is Spiroglyphus annulatus, which is a small vermetid originally from the Caribbean. However, similar small species live elsewhere in the world, and they all look pretty much alike. It will probably take genetic testing to verify the identity of our aquarium friends. Whatever species it is, this particular one has small individuals. The tube seldom is over one or two millimeters wide. The shells are typically reddish or reddish-brown; sometimes they are even tinged with violet. The animal forms a small, calcareous shell mound and then sends up a short, three to five millimeter long, vertical stalk. The upper edge of this tube may be razor sharp, and may inflict rather nasty cuts. A few of these would be no real problem; however, this animal reproduces very well in marine aquaria. Left unchecked, it can reach populations of over several thousand in a few months. They prefer high current areas, and will infest and clog plumbing, significantly reducing water flow. In severe infestations they can clog and shut down pumps. The only solution in cases like these is physical removal of the animals using whatever method is easiest (a muriatic acid bath works well).
Fortunately, some fishes such as Copperband butterfly fishes, seem to eat them, and some hermit crabs will eat them as well. Eating these worm snails may well be the only truly beneficial effect of hermit crabs in aquaria.
