I think expectation and reality are far apart here.
I don't work for VDOT. I don't manage their permits, but I do have practical experience from running these exact permit programs for multiple entities over the last 20+ years.
For us, we sweep roughly 1/4 of our lane miles ONCE per year. The rest are never regularly swept. Why? Because they don't need to be, aren't required to be by a permit, and we don't have enough staff, funding or equipment to actually do so.
My agency manages 4000+ lane miles in my area - and I have - count them - 2 Sweepers permanently assigned across around 20 facilities we dispatch crews out of. (Each costs roughly $225,000 USD when new, and they are maintenance queens...) With a top speed of just over 20 MPH, the sweepers don't cover a lot of ground, and really don't when sweeping at 5 mph.
So yes, you get piles of "stuff" on the roads. We spot clean them if we get calls, or if we have the equipment in the area and its observed. And that typically means a bobcat with a kick broom... But we don't "sweep" miles of roadway that way. And that pile can show up literally minutes after we've swept...
So again, I'm not speaking for VDOT.