Bluegrassmom, I LOVE CBG so I hope you manage a trip there sometime. (and I've been disappointed by a couple of supposed grand gardens so please trust me) Be aware that it's so large and lovely you will want as much of the day there as you can get. You would probably be so sad if you got there and realized you could only spend 4 hours. Or 6. (unless you're just about the daylilies, and nothing else. But I have a feeling you might like gardens!)
If you do drive in, try to hit the south edge of the city around 9:30 am or a little later and you should be able to get up to Highland Park easily (although you never know when crazy traffic will happen). Then spend the rest of the day there and stay in a hotel near there. There's a wonderful music festival park called Ravinia that is just a mile or two away, you could even try to coordinate an evening concert with your visit (although you may be quite drained by the garden delights)
