Magneto and Doom can give Thor decent fights but neither are taking him solo. Victor has received a push in the last few years regarding magic but as we saw in their recent encounter, Thor would wreck his shit in a straight up fight. At least if he was pissed.
Still, the team should take the majority on average unless Thor is unusually pissed off or something.
For the record, Magneto can control Thor's hammer to some limited extent. However, the moment the Odinson utilizes his connection to Mjolnir, that's the end of the advantage.