In my opinion, dropping money into a 328 for performance is a waste of time and money. Buy the real thing. The power you will extract from the N52 wont be worth the dollars you'll spend doing it.
I personally feel like it's worth it as long as you get your M3 with a warranty. The maintenance of the older car will be more to keep up with. Also the car isn't as refined so be prepared to make some concessions in comfort.
In my opinion, if you were fine with owning a 328 in the first place, maybe you should drop a few thousand and get a 335 and get the best of both worlds. You could probably even get one under warranty. 335 will offer similar performance, better comfort, and probably similar reliability as the M3. Its not as raw but if you have a 328, maybe not being too raw is a good thing.