The high cost of removing our guilt
Jesus paid a high price—his life—in order to remove our guilt before a holy God. While Jesus paid the price only he could pay, freedom from guilt is costly for us in other ways. The high price for us is the difficult task of admitting to God (and often others) that we were wrong and that we need forgiveness. This is called confession. As costly as confession is, the rewards are even greater: peace with ourselves and others, restored relationships, the removal of sin, and a renewed relationship with God.
But pride is a powerful force in our lives. Pride tells us, "All is well! You're OK!" Pride denies the reality of sin; in contrast, guilt warns us that all is not well. Guilt acknowledges the presence of sin and urges us to confess it. So these two—guilt and pride—battle it out in our souls. When guilt wins, a sin that separated us from God is removed. When pride wins, we remain isolated from God's grace, love, and care. There may be a high cost to removing guilt, but there is a higher cost to keeping it.
(Tyndale House), p 638