I'm using a 250cfm fan I bought online for about 40 bucks. It's positioned the right way, as I feel more air coming out the filter than I do from the fan itself. It feels like backpressure, as if the air isn't going through the filter fast enough, so it kicks back.
It might be the dryer...