So by what standard is the determination made whether the pope is speaking infallibly?
Glad you asked.
While I can provide the standard (and do so below) how the standard is applied is debated. Some Theologians argue that aside from affirming those parts of ecumenical Council decrees that meet the standard (there are reasonable arguments to claim that NONE of the Vatican II documents had any parts that met these standards), only the definition section of the decrees on the Immaculate Conception and the Assumption have met the standard.
I have heard arguments advanced on Paul VI’s teaching on contraception and JPII’s teaching on the ordination of women.
Suffice it to say that the general consensus is that it is extremely rare. Moreover, if in doubt about a particular situation, one is not obliged to presume that infallibility has been exercised.
Given this rareness, the more interesting question is how to deal with likely non-infallible statements of the Pope-—but that is not what you asked about, and so I will proceed to actually answering your question
Vatican I (1869-1870), and to a lesser extent Vatican II, are the places to look for the most authoritative and definitive explanations (which isn’t to say that there aren’t places that aren’t clearer or more detailed—just that other things are often in no way official and Catholics are under either no or less obligation to affirm them.
Here is what Vatican I ended up saying:
This is literally the last paragraph and conclusion of the official work of Vatican I (Church Counsel from 1869-1870).
https://www.papalencyclicals.net/councils/ecum20.htm
Two relevant (probably the most relevant) paragraphs out of the Catechism of the Catholic Church:
891 “The Roman Pontiff, head of the college of bishops, enjoys this infallibility in virtue of his office, when, as supreme pastor and teacher of all the faithful - who confirms his brethren in the faith he proclaims by a definitive act a doctrine pertaining to faith or morals.... the infallibility promised to the Church is also present in the body of bishops when, together with Peter’s successor, they exercise the supreme Magisterium,” above all in an Ecumenical Council.418 When the Church through its supreme Magisterium proposes a doctrine “for belief as being divinely revealed,”419 and as the teaching of Christ, the definitions “must be adhered to with the obedience of faith.”420 This infallibility extends as far as the deposit of divine Revelation itself.421
This is a larger, also relevant section of the Catechisme:
I. Moral Life and the Magisterium of the Church
2032 The Church, the “pillar and bulwark of the truth,” “has received this solemn command of Christ from the apostles to announce the saving truth.”74 “To the Church belongs the right always and everywhere to announce moral principles, including those pertaining to the social order, and to make judgments on any human affairs to the extent that they are required by the fundamental rights of the human person or the salvation of souls.”75
2033 The Magisterium of the Pastors of the Church in moral matters is ordinarily exercised in catechesis and preaching, with the help of the works of theologians and spiritual authors. Thus from generation to generation, under the aegis and vigilance of the pastors, the “deposit” of Christian moral teaching has been handed on, a deposit composed of a characteristic body of rules, commandments, and virtues proceeding from faith in Christ and animated by charity. Alongside the Creed and the Our Father, the basis for this catechesis has traditionally been the Decalogue which sets out the principles of moral life valid for all men.
2034 The Roman Pontiff and the bishops are “authentic teachers, that is, teachers endowed with the authority of Christ, who preach the faith to the people entrusted to them, the faith to be believed and put into practice.”76 The ordinary and universal Magisterium of the Pope and the bishops in communion with him teach the faithful the truth to believe, the charity to practice, the beatitude to hope for.
2035 The supreme degree of participation in the authority of Christ is ensured by the charism of infallibility. This infallibility extends as far as does the deposit of divine Revelation; it also extends to all those elements of doctrine, including morals, without which the saving truths of the faith cannot be preserved, explained, or observed.77
2036 The authority of the Magisterium extends also to the specific precepts of the natural law, because their observance, demanded by the Creator, is necessary for salvation. In recalling the prescriptions of the natural law, the Magisterium of the Church exercises an essential part of its prophetic office of proclaiming to men what they truly are and reminding them of what they should be before God.78
2037 The law of God entrusted to the Church is taught to the faithful as the way of life and truth. the faithful therefore have the right to be instructed in the divine saving precepts that purify judgment and, with grace, heal wounded human reason.79 They have the duty of observing the constitutions and decrees conveyed by the legitimate authority of the Church. Even if they concern disciplinary matters, these determinations call for docility in charity.
2038 In the work of teaching and applying Christian morality, the Church needs the dedication of pastors, the knowledge of theologians, and the contribution of all Christians and men of good will. Faith and the practice of the Gospel provide each person with an experience of life “in Christ,” who enlightens him and makes him able to evaluate the divine and human realities according to the Spirit of God.80 Thus the Holy Spirit can use the humblest to enlighten the learned and those in the highest positions.
"If his mouth is moving..."