The issue is that scripture is not the word of God. It is the writing of men that heard the word of God. The only portion I have heard of as being directly written by God, and not told and then written, would be Moses' telling of how the stone tablets came into being. Everything else is man's written account of what happened. Being inspired to write something and told each and every word to write are very different things.
To qualify this against those that claim the bible is God's word direct, they also claim Christ is the Word of God, there is no reason to believe the Bible is the Word that was claimed to be infallible.