It is the Power Gem that operates on the physical spectrum, allowing it's user to be invincible, not Time, Soul, Reality, Space, Mind.
Like I said you haven't gotten one shred of evidence where I actually have more than one.
1. The Force Block was employed well after the conflict started, and only when imprisoned within it was it stated that Thor began to amp off of it.
2. Thanos can not take hits capable of folding a universe.
3. Thor with the use of the Power Gem would not have only crushed Thanos, but he would have destroyed the facility and the rest of the the planet in the process. This did not happen. There is far more evidence that goes against Thor tapping into the Power Gem in a contiguous fashion, than there is evidence to the contrary.
4. Thor's state of mind would not allow him to use the Power Gem, as I mentioned before Titania had to be tutored in it's usage by the Champion, and even then it took time.
You read comics, but take only what you deem relevant, and the rest just goes in and out. Thor fought Thanos, and gave him hell under his own steam. This is exactly how Thor would fight Thanos without the Power Gem.
Warlock with the PG would steam roll Thanos.