of course not, cause the full potential of the power gem is decribed as followed (by Thanos): "It drwas strength from the ends of infinity", "with it I need fear nothing" so it's simply impossible for another brick to defeat someone with that gem, now it's still arguable by your definition "knows how to use it" how much power he can pull from the gem.