(Question inspired by several of the questions in Bible Study: Synoptics #19: Agony in Gethsemane)
Why did Jesus have to die? Couldn't he have mediated for us to God without such a sacrifice? And if he did have to die, why did he pray to God asking to have the "cup" pass him by?
This is a very complicated question (partly because there are three questions in it). The summary of the teaching of the Church is that the blood sacrifice of Jesus was the last sacrifice to be made to God, and ushered in the New Covenant: we received forgiveness for sins and eternal salvation through Jesus's self-sacrifice on the cross. So... why did Jesus have to die? Let's first look at the nature of sacrifice in the Old Testament.
The Purpose of Sacrifice
As I mentioned in an earlier Q&A (Who were Adam and Eve?), the book of Genesis contains the first record of a sacrifice to God: it occurs after Adam and Eve are expelled from Eden. Cain made an offering of "the fruit of the soil" and Abel made an offering of "the best firstlings of his flock" (Genesis 4:3-5). God looked favorably upon Abel's offering, but not upon Cain's. Why might this be? The text says that Abel's offering was the "best" of his flock, but it does not describe the quality of Cain's offering. There are generally three elements in any sacrifice: the thing being sacrificed, the manner in which it is offered, and the state of the person making the sacrifice. Consider Luke 21:1-4, in which Jesus comments on a poor widow donating two coins, compared with the wealthy people making considerably larger donations. Jesus says that the widow, "from her poverty, has offered her whole livelihood." Truly, the widow's sacrifice was more favorable to God than that of the wealthy men.
So then, if God was pleased with Abel's offering, we can assume that it was an especially good lamb, that Abel presented it up to God in a reverent manner, and that Abel's intentions and conscience were clean. There are, then, three places were Cain could have strayed: object, deed, or conscience. We do not know the quality of the fruits he offered up, nor the manner in which they were presented, nor his spiritual state. Although Abel and Cain were sinful in nature, I think it would be erroneous to assume Cain was "more evil" than Abel before that sacrifice just because, afterwards, Cain becomes resentful. Still, it seems something in Cain's sacrifice was not right. Jesus tells us in Matthew 5:23-24 that we should present our offerings to God with a clean conscience; so it is likely that Cain bore some grudge or sin in secret that sullied his offering.
So then, it appears that "sacrifice" was a man-made institution designed to appease God by giving up something that we find favorable to our personal nature that, in the sacrificing, it may take on a favorable spiritual nature to God. Before we had forgiveness of sins through Jesus Christ, we made peace offerings to God through blood sacrifices: the finest lamb or bull or fowl, incense, etc. I offer that sacrifice was man-made, rather than God-made (like the Ten Commandements), because there is no evidence in Genesis of God asking Cain and Abel to give of the work of their hands. As man-made, it also represents a tendency in our free will to seek reparations with our God, that we would willingly give of ourselves to please God.
While sacrifice may have been man-made, it was further "regulated" by God through the laws recorded in Leviticus 1-7. Leviticus 1:1-4 explains the nature of animal sacrifice and what it aims to accomplish: "If his holocaust offering is from the herd, it must be a male without blemish. To find favor with the LORD, he shall bring it to the entrance of the meeting tent and there lay his hand on the head of the holocaust, so that it may be acceptable to make atonement for him." (Leviticus 1:3-4)
The Shift in Sacrifice
Christ's mission on earth was to bring redemption to mankind by offering them what animal sacrifice could not: repentance of sins through personal actions, not by proxy. Paul's letter to the Hebrews goes into detail about the nature of sacrifices and atonements under the old covenant (Hebrews 9-10). The old covenant was full of ritual practices to cleanse the flesh, whereas the new covenant is designed to purify our consciences and refresh our spirits (Hebrews 9:13-14).
Paul explains the manner of sacrifice in the temple. The priests and the high priest entered into the outer tabernacle frequently, but only once per year did the high priest go behind the veil into the inner tabernacle, the Holy of Holies, with "blood that he offers for himself and for the sins of the people". This physical separation between man and God, the veil through which only the high priest could enter once a year, represented that man, in his sinful nature, could not stand in the presence of God (for the Holy of Holies is considered God's dwelling place on earth). These offerings and sacrifices were merely symbols of the greater need for spiritual renewal.
Paul goes on to explain in Hebrews 10:1-10 how this sacrifice-by-proxy cannot fulfill our spiritual needs: "it is impossible that the blood of bulls and goats take away sins" (Hebrews 10:4). Paul invokes the words of Psalm 40:7-9 in the person of Jesus: it was not sacrifice and offerings for disobedience God wanted, but rather our free obedience to His will. This was accomplished in Jesus Christ. The sacrifices under the old law were superceded by the will to obey God. And it was the very will that Jesus obeyed that simultaneously fulfilled the old law and the new covenant!
Christ's Death for Us
When Jesus prayed in Gethsemane for the "cup" to pass him by, he ended his plea with "not my will but Yours be done". Why did Jesus ask for "relief" from his mission? Because he was human. Jesus is our role model, our example. We do not have the impossible expectations of God-like perfection as our example, we have the perfect human as our example. Christ was made perfect in his humanity by his constant devotion to God's will. Jesus was tempted on many occasions, but never gave into his tempter. The prayer in Gethsemane was Satan's final temptation.
Because Jesus is our human example, it is only "fair" that he experienced the same pains and problems we face. Jesus, in a sudden shattering moment of realization, feared death. And maybe not just his death, but what his death would mean for all he had done. Was his death necessary to complete his mission, or would it end all he had worked for? And was death the necessary end? Jesus got through this temptation by ending his supplication as he did: not my will but God's be done.
And so we finally close in on why Jesus had to die, and why nothing less than his death would satisfy his mission. As I said, his death fulfilled both the old law and the new covenant: a blood sacrifice for sins as well as the perfect obedience of God's will. Jesus became our blood sacrifice, our lamb; but he was different from every other lamb or bull or bird offered up, in that he gave himself willingly. He took his place as a human accepting the weight of the sin of all humanity on his shoulders and on his soul. He offered himself up to God and to all of us as the final blood sacrifice and ushered in a new covenant. We are forgiven of our sins through Christ and we are promised eternal salvation if only we obey the will of God.
Jesus died for us as a human, not as a God. Our God did not succomb to sin, for God is above and outside sin. And, just as sacrificing beasts could not truly restore us, neither could God stepping in where humanity could simply not tread have restored us. Jesus, in his human nature, took our sinful burden upon himself. Death is the wages of sin (Romans 6:23) and Jesus in his sinlessness suffered our death-to-sin. In this act, he bridged the gap between us and God: the veil in the tabernacle was torn in two, signifying an open communication between us and God. Christ did for us, as a human, what we could not, and we have his example to follow. Jesus does not ask the impossible, he asks that we obey God and repent for our sins at those times when we disobey.
The Son Also Rises
The story of the gift of salvation and the new covenant does not end with Jesus's death. Jesus was raised to new life, resurrected into a perfect body. This is not a "perfect body" as we think of bodies, although that is not to say it is not a physical presence. The perfect body of the Resurrection is one from which sin has been purged absolutely, and where sin was has been replaced by grace, by God's presence. Christ had no sin of his own, but bore ours to his death; it was in his resurrection that he was given the divine, perfect body which we all wish to attain and can achieve through the example of Jesus.
Turn away from sin and be faithful to the Gospel of Jesus Christ. Reconcile yourself to God and obey His will. This is the price of eternal salvation: love, not taken away, but given in abundance.