Dedicated to the Lord Jesus Christ Who loved us, gave Himself for us, and washed us from our sins in His own Blood; and to the Church, which is His body.

Our spirit is a part of our heart, but as in 1 Peter 3:4, it is only a part. Sin, iniquity, unbelief, etc., come from the heart but not the born-again spirit. Hebrews 4:12 suggests that spirit and soul are both part of the heart. The Bible admonishes us to believe with all our heart (Acts 8:37), have singleness of heart (Col. 3:22), and tell us our hearts can have two minds or ways of thinking (James 4:8).
Even Christians still struggle with things like pride and foolishness which Jesus said came out of our heart. It's certain that our born again spirit is not the source of these sins. The heart includes more than our spirit.
The English word "soul" comes from the Greek translated "heart, life, mind, and soul." The mind is the leading part of the soul, followed by the will and emotions. We could also describe the soul as being the hidden part of all existing beings or what most people would call the personality. The soul is the center of the feelings and emotions, appetites and desires, as well a perception and consciousness. The soul may also speak of the total person - his total being or self, a person, and the hidden man or inward man. 
The soul and the body do not get born again. It is the spirit of man that becomes totally new at salvation. So, although every believer receives the same miraculous spiritual rebirth, the visible results of that inward change will vary from person to person according to how much he renews his mind. The term "heart" can include all of the inner man - spirit, soul, or any portion of it. The peace of God will keep our heart at rest in Christ Jesus.