+1 swbuckmaster
My experience leades me to the same conclusion.
I have had deer that tasted pretty gamey, even sagey, but if I am honest with myself it was because of my ignorance at the time of how important proper field care is and how much it affects the taste of the animal.
Cool it down quickly, treat the meat like it's something you are going to eat (out of the dirt, etc.) and it won't much matter if it was a 2 year old or a 7 year old.
Just my expereince!