So are the later Banite Sith "Force atheists?"

What I mean is Plagueis and Tenebrous, with their description of the Force being nothing but an energy source/biological component that can be used without any consequences seemed to imply that. They also didn't believe in Sith Spirits, well Plagueis didn't in Book of Sith. And it seemed as though they never believed in the will of the Force.

Those 2 might be, but I know sidious isnt a "force atheist"

A lot of Sith take a fairly atheistic view on the 'will of the force'. To them, the force is something you impose your will on, not listen to, after all.

Though, force spirits are definitely a thing.

Maul uses the line the Clone Wars episode Eminence: "It was the will of the Force." And he definitely meant it in a "Destiny" type way.

Plus Palpatine tells Vader in ROTS: "You are fulfilling your destiny Anakin." And Vader to Luke in ESB "Luke come with me, it is your destiny."