Peter is distinguished from the other Apostles a lot in the gospels. At the first meeting Christ announced the change of his name from Simon to Cephas = rock. The name itself indicates the office he was appointed by Jesus. In all the lists of the Apostles Peter is named in the first place. In St. Matthew he is explicitly called the first. Since he wasn't actually first from the point of view of his time of calling, Andrew was before him, the constant placing of Peter's name at the head of the list of the Apostles indicates the dignity of his office. Peter together with James and John, waas permitted to witness the awakening of the daughter of Jairus (Mark 5:37), the Transfiguration (Matthew 17:1) the Agony (Matthew 26:37). The Lord taught from Peter's boat and paid the temple tax for Himself and Peter jointly. He ordered him to strengthen his brethren after His own return.
In the passage you quote (plus a little bit more you missed) the words are addressed solely and immediately to Peter. In them Christ promises to confer on him a threefold supreme power in the Church which he is to found. Peter is to guarantee to the Church a unity and unshakable strength, similar to the rock foundation of a house. He is to be the holder of the keys, the steward of the Kingdom of God on earth. He is to bind and loose, to impose the ban or loose from the ban, and also interpreting the law, prounounce a thing to be forbidden (bound) or permitted (loose). The primacy was conferred when Christ gave Peter the three-fold mandate, after His Resurrection, to feed Christ's lambs.
After seeing the high office which Christ gave to St. Peter it seems like common sense that He didn't give it to him only to have it end when Peter died 30 years later. As the function of the primacy is to preserve the unity and solidarity of the Church, it is to continue substantially unchanged until the end of time. Catholics believe, with the witness of tradition, that the successor of Peter is the Bishop of Rome. Now the Church cannot exist without the temporal foundation which supports it, so to be in the Church it is necessary to be in communion and agreement with the successor of Peter. If you are not you are outside the Church. I should also note for clarity that the Pope is only the visible head of the Church, the representative of Christ on earth. Christ is the invisible head of the Church.
Hope this helps. If anyone sees any errors let me know.