Yea, sugar fine is usually a bad idea IMO, because with the flow required for a healthy tank, it will usually be stirred up constantly.
If you do decide to use it, make sure you rinse it VERY well before you put it in your tank, or you will be constantly fighting a sand storm.