They also knew much about God. It was required of the Jewish men to memorize large portions of the Torah (the first five books of the Bible). Their whole society revolved around God's moral standards and religious observances. But they did not know God.
There is a difference between knowing about someone and really knowing them. Millions of people are acquainted with knowledge about Jesus. They may even be moral and observe religious ceremonies, but if they don't personally experience knowing Jesus, they are not saved and will not make it to heaven. Even the devils believe and tremble but they aren't saved.
Our spiritual man became dead (separated from) God through sin. Just as we didn't accomplish our physical birth, we cannot produce this spiritual rebirth. We are to incapable of saving ourselves, we need a Savior. We simply believe on the Lord Jesus Christ and we are saved.
Salvation is not a reformation, but rather a regeneration, a new birth, a new creation, that can only be accomplished by a creative miracle of the Holy Spirit. We must be born again.