A little background: I'm 56, but in fairly good shape. I've been hunting in Idaho my whole life. When I got "too old" to hike into the back country, I bought llamas to carry my "stuff". I walk, but they carry everything. Sometimes I go a week at a time and often I hunt by myself. I've taken a lot of game and could possibly be described as fanatical.
In Idaho, it is often said if you meet an arrogant Texan, just threaten to iron one of our counties flat and Texas will become the third largest state. Much of Idaho mule-deer hunting is described in that adage; 3/4 of our state is vertical and far from roads.
If you want more easily accessible country (not as scenic), try unit 54; you actually described it in your post. Also, the desert bounded by Idaho Falls, Pocatello, Twin Falls, Carey and Arco is flat. It's difficult to hunt (as in finding game), but physically easy...it's flat. The game is not real plentiful, but a new state non-typical was taken there a couple of years ago and the pressure is fairly light. It's not quite as easy, but, the SW corner of the state is also less physically intimidating.
I like the high country and still recommend getting into it, but you can walk uphill (at a fairly good rate) for 3 to 5 hours, before you get to the hunting area. You either have to camp up there, or make the hike every day; there will be others making the same hike if it's within 1.5 miles of a camp site.
Unit 43 allows dirt bikes to go almost every where. That makes it physically easier (not a cake walk), but it also increases hunting pressure. My party saw no other people (in unit 48) this fall, because we were a long way in. It's tough, but it can be done. My 3.5 year old son actually made the hike (5 miles each way and the first 2 VERY steep) during a snow storm. We just had to go slow. If he can do it, your friends can do it. Physicality is important, but so is mindset and preparedness.
Good luck. Grant
If you want specifics, call me. (208) 233-4150 in the evening.