You have to evacuate the heat from the closet itself somehow.
Is there any way you can drill holes for ventilation ducting? It'd be an easier repair job for the closet later.
With flexible ducting, and a couple computer fans, it probably wouldn't be noisy either.