The early August hunts generally produce more deer IMO. However, some days it feels like you are up in the dark, glass for an hour in the morning and then say, okay, its 8am, I just have 11 hours to kill before the deer are moving again... That's not always fun.
The rut hunts are more difficult for many of the reasons already stated. Unpredictable deer, never staying still, poor weather conditions, wet, cold, snow... etc.
I prefer hard-horned over velvet everyday. If I could, I would hunt the late seasons. Just a hard to time to find the time.
"Therefore, wo be unto him that is at ease in Zion!" 2 Ne. 28: 24