I think Trump can call them back.
“the President can call the Senate (or both houses) into a special session on “extraordinary occasions” under Article II, Section 3 of the U.S. Constitution. This authority allows the President to force lawmakers back to Washington D.C. to address urgent, specific matters like economic crises or nominations.”
https://constitutioncenter.org/the-constitution/articles/article-ii/clauses/348
Interpretation: Article II, Section 3 | Constitution Center
Article II, Section 3 grants and constrains presidential power. This Section invests the President with the discretion to convene Congress on “extraordinary occasions,” a power that has been used to summon the chambers to consider nominations, war declarations, and emergency legislation.