They are definitely in the same tier of both power and skill.
Physically Malgus holds the edge. However i would give the slight nod to Dooku in terms of sabers, due to his showings against Anakin and Obi wan and Yoda, and the accolades he received from the jedi order and Sidious and feats during his time as a jedi.
in the force it gets trickier. Honestly i can see this going either way. So for the sake of argument i would call this to be even.
all out- I would go Dooku. He has roughly 20 years more experience which really doesn't matter since Malgus has been out in the field for 40+ years. However I don't see Malgus doing a swell against Obi wan and Anakin as Dooku did or performing as well against Yoda as Dooku did, and in the force its arguably a tie, so Dooku's eve so slight saber advantage brings him to a slight victory.
Sabers only it's probably even. Malgus is up there with Dooku, given his ability to contend with HoT, Barsen'thor and 2 great republic warriors at once (or Nox, Wrath, BH that defeated Obi-Wan level guy and Agent that survived Darth jadus - even better).
Force only Dooku doesn't stand a chance. I'm not going to go through Malgus's feat of choking 3 of the protags at once again - simply put, Malgus can choke out Hero of Tython who can defend against Vitiate's power long enough to batter through with the saber... and two other guys, both of them completely effortlessly, as seen by the fact he could still fight the 4th combatant in that time. Dooku gets Force Crushed on the spot.
All out Dooku's skill doesn't make a difference, Malgus can one shot him as soon as Dooku draws his saber.
The fact that you use a random int generator in a flashpoint that does even require 2 let alone four people, of more than one of the same class I might add, as evidence is pretty kekworthy. Guess when two people can go he can only choke one, and when one goes, nobody. And choking out someone and still losing is pretty sad, imo, but maybe that's just me.
Dark Temple Vitiate was creating physical illusions that actually were like independent beings and they were actively fighting HoT. That feat is already above RotS Sidious, Kun, etc. And HoT fought through dozens of fully trained Sith Warriors stationed as Vitiate's temple guards, so they were obviously part of the elite forces. HoT fought above a hundred of them, all the time on the dark side nexus.
Is there any actual proof that the Imperial Guard is anything but a bunch of fodder in expensive armor? Hell, couldn't the HoT have just killed everything before T7 was even involved? Not to mention they had time to expend more energy, fight and army, and Vitiate still lost.
Also, didn't Boba Fett lose to a child anyway? Is that really someone you want to be compared to?