Tradition and ancient writings find both Peter and Paul in Rome, where they both were martyred; Peter by crucifixion upside-down, and Paul by beheading, since he was a Roman citizen. That they both earned their crown of glory in Rome is providential: the great persecutor of the Jews and Christians would itself be conquered by Christian Jews. Both Saints were familiar faces in that City, and the many ancient artistic depictions of them, including recently discovered ones, are quite similar:
The two chief Apostles, on the other hand, are always easily recognized and are of marked individuality. St. Peter appears as a man of great energy, with a short, thick beard, and close cut, curly hair, which in the earlier frescoes is partly, in the later wholly, gray. St. Paul is represented as the Apostle of intellect, bald, and with long, pointed beard, dark brown in colour.The Roman co-Emperors Constantine and Licinius, in the Edict of Milan of A.D. 313, proclaimed religious toleration throughout the empire, including toleration of Christians. Soon afterward, Constantine started the construction of a new capital of the Empire, a city he named after himself. Constantinople was to be a new start, and the old ancient power blocks of the Senate, Tribunes, and other traditional offices were swept away, consolidating power in himself.
— Catholic Encyclopedia, Portraits of the Apostles
Many were dismayed that the leadership of Christ's Church did not follow the Emperor to Constantinople, but this is also providential. Just when Rome became a Christian city, it ceased to be a seat of worldly power. It is unthinkable that the Church ought to abandon the tombs of the great Apostles Peter and Paul, and the places of martyrdom of countless Saints and their cemeteries. By remaining in Rome, the Church distanced itself from Imperial meddling, and was better able to follow Christ.
Tyrants despise any power besides themselves, and the Church is the greatest threat to any despot. Indeed, religion is often seen as being merely a tool of politics, as was seen in the Sadducees of ancient Jerusalem, and in much contemporary religion in the United States today. A savvy Pontiff may have seen the move to Constantinople as being a good way to influence the Emperor, but rather what typically happens is the opposite, and religious leaders ending up being manipulated by power. This was seen in a major way in the Iconoclast controversy; and during the 20th century, the Orthodox Churches were severely persecuted and often controlled by Communists.
By remaining in Rome, with the tombs of the great Apostles, the Church was better able to follow the example of these holy men.