I've now driven this Sienna over 100 miles on dirt roads. It has no problem on the maintained stuff, getting slightly skittish if you quicken the pace enough. But that sort of driving is rare because the minivan has such limited ground clearance.
While in Montana, I spent most of my time in the dirt straddling erosion, lest an unseen road-rut ruin our day. Two or three such ruts snuck up on me along the road to Bowman Lake, which gets considerably more rough after passing the Polebridge Mercantile Store. The sharp impacts quickly reminded me what the Sienna was not. Still, the van proved plenty capable so long as I didn't get too eager. And the views at Bowman took my breath away, so all was forgiven once we got there.
The only time AWD kicked-in noticeably was a few days later, on a long, single-lane uphill section of loose gravel. I'd actually just missed the turnoff to Morrell Falls Trailhead, which is where we were headed. The van didn't miss a step. I had the IP display already switched over to watch the front-to-rear power distribution or I may not have even noticed it was happening. Atop the hill I found a turnout, zoomed-in on the native navigation screen and found the road I missed there on the map. I suppose the nav system deserves some credit, too, for its accuracy. It at least saved me from further, "Daddy took the wrong turn," in-van ridicule.