You can start wehnever once its warm enough and no more frosts. Just if you start later you will most likely end up with a smaller plant. I have no idea about temps from where you from just i know in the US theres only some states ready for growing this early.
You can plant outdoors as late as June with very little negative effects. The later that you plant, the less time they have to veg, which will affect your yeild. Less vegetative time=smaller plants, smaller plants=smaller yeild.
I don't believe it's too late at all, if anything, I would think still a bit early for outdoor plants in NE.