About a Scene in Reloaded

I have a question about something that happened right in the beginning of reloaded. After neo has the fight with three agents Smith comes out of nowhere and says something along the lines of it's going according to plan. Where did he come from? Why was he there? Why didn't he come out and fight neo if he was there? That has been bugging me since I saw the film and I can't figure it out. Thanks for the help!

Smith was there before when he gave neo his ear piece - "you set me free"

i think he wanted to see if neo could handle the upgraded agents so didnt fight then.

also as hes a rouge program and ex-agent he wouldnt fight with the agents.

I also think that Smith was waiting for his moment to fight Neo