Look at the stands, Rob:
YouTube video
JYsWhf-FKOI&feature=related
JK Rowling said in an interview that there were thousands of students.
Another thing to think about....you said:
-Van Dammes get 42 minutes prep to sneak about, survey and position themselves outside of Hogwarts.
and then
-Wizards are unaware this is happening until the 42 mins are up.
That means that once team VD has done their little surveying thing, the wizards are aware of their presence.
Also, team VD will have no idea what's inside Hogwarts. There are many floors, passages, there are staircases that move around as they please. There are paintings and ghosts that will see every move team VD makes. Team VD will be spread thin, and they will have no idea what's around each corner. The wizards know the castle, they know where each passage and staircase leads.
Way I see it, Dumbledore dons Harrys cloak, sips the felix felicis, and solos here.