RE: Track a bad idea?
Not at all - track days are some of the most fun you can have anywhere. I've been to most of the Southern California tracks a number of times and as has been said they are overall pretty safe with lots of run-off.
The one exception is actually the new-ish Horse Thief Mile at Willow Springs. It's neat with lots of elevation change but it also has one off-camber turn coming down a hill that is just above a straight such that if you lose it you'll drop directly into on-coming traffic - and likely upside down. I've never seen a car get hurt nor have I hurt my car even with my fair share of off-track excursions but if any SoCal tracks were to have a problem then Horse Thief would be it.
Go try it and be safe.