I'd say this is honestly a battle of first response.
The sheer scale of Elsa's power significantly outstrips that of Gandalf's, and probably anyone in the LotR movieverse that I can recall (Haven't seen Hobbit 2 yet), but she herself is very physically frail, relatively speaking.
Gandalf is, as Q mentioned, more physically powerful than one might think, far more-so than Elsa, and in close-quarters he'd easily crush her, but despite his durability, he hasn't shown the ability to withstand power of Elsa's magnitude, though shields could aid in this regard. Shields would however render Gandalf himself incapable of attacking.
In terms of reaction-time, both have reacted to fast projectiles, Elsa to a crossbow bolt, Gandalf to an arrow from Legolas. I don't know enough about how fast either travel to make an educated guess on which is faster.
Elsa does have a significant advantage in her ability to actually conjured frost constructs to battle Gandalf, though, despite their size, they are relatively frail. Hans is able to cut clean through Marshmallow's leg after all.
Another considerable advantage is Elsa's "crowd control" abilities, so to speak. Rendering the area a hostile, deadly blizzard, the floor slick due to ice, and covered in stalactites or whatever, would seriously impede Gandalf, but Elsa would have complete freedom of movement.