Though Queen's Peril(which is set right before TPM) does heavily imply that Palpatine and Maul are the only two Sith who were around at the time. So I'd guess he killed Plagueis a good bit before that.
__________________
"I am tired of Earth. These people.
I am tired of being caught in the tangle of their lives."