Eh, I would agree that DOFP used more theatrically acclaimed actors (Stewart, McAvoy, Fassbender and McKellan have all played Hamlet, etc), and in general those four top anyone that was in Avengers, but it didn't hurt that the script was written with more character moments than Avengers was.
I'd argue that RDJ is the equal of anyone in that cast though. He's arguably one of the best actors of this generation.