I use dessicants all the time. I always save the little pouches that come with electronics, purses, pills, etc and use them to dry my weed. Never noticed any weird smell or taste.
**I should point out that I only use these for quick dry. For example, if I'm getting desperate and don't want to wait for things to dry the conventional way, I'll put a few popcorn buds in a brown bag with some dessicant packets and lay it on top of the tv for a few hours. Works great.