Just stop feeding them. When they need water, give them plain water until you see a bit coming out the holes. It's not an aggressive flush, but it lets them use up the nutrients stored in their tissues, and washes excess out of the soil gently so they aren't eating more.