What about a used 5D?
Buy a new body if you feel that it will provide you with specific opportunities not afforded by your current body. The 50D provides better resolution than the 20D and maybe somewhat better high ISO noise (though some will argue that point). If you're not planning on making huge prints (or cropping extensively) I don't know that the 50 would let you do anything special that the 20 would not.
This is not the case with the 5D. Switching from the 20D to the 5D will open up the world of Full Frame with its potential for low DOF and sharply isolated subjects. If you aspire to this sort of shooting then you should certainly consider a FF body. Right now with all the gearheads jumping on 5D2's you should be able to find a used 5D for somewhere in the $1K - $1200 range. This seems like the most cost effective approach to getting into FF and determining whether the advantages/disadvantages of the larger format suit your shooting style. Unless you absolutely have to have the latest and greatest (or need ISO 6400 and/or 21MP and/or video) for your work I'd recommend that route rather than the 5D2.