Chapter 6 describes the miraculous hiding of the sacred vessels of the temple (cf. 2 Machabees 2:1-9), including the veil, in anticipation of the fall of Jerusalem to Nebuchadnezzar. So the general context in Chapter 10 is lamentation over the loss of Jerusalem and sovereignty of Judah. Let us keep this general context in mind.
The chapter begins by woes to "all who live", and then it enumerates the husbandmen with their agricultural products, then bridegrooms and their brides, then wives, especially the mothers. This procession of grief ends with death of beauty itself: "From this time forward speak not of beauty, And discourse not of gracefulness" (v.17).
Next the lamentation focuses on the priests. The rhetorical advice to the priests is to toss the keys to the sanctuary to heavens, and somewhat defiantly say to the Lord: "Guard Your house Thyself, For lo! we are found false stewards".
It is at this point that the virgins are brought into the picture. They are advised to cast their handiwork into fire. What will the fire do? It will "send them to Him who created them". In other words, the linen and silk woven by these virgins are likewise returned to God.
At this point Chapter 10 ends and with it the lamentation ends. Chapter 11 contains an angry speech directed at Babylon.
So this is my case for the temple virgins:
So, yeah, I'd say these are the virgins working on the temple accouterments, whose work is valued alongside other temple possessions.
Now let us recall that the sacred vessels, the raiments of the priests, and the veil of the Tabernacle are to be hidden. Indeed, we know that Jeremiah, Baruch's boss, hid them in a cave. Here we have an allusion to that (Chapter 6 verse 10). So the suggestion, perhaps rhetorical, to destroy the linen, silk and gold is a continuation of the same idea, to not let the enemy possess sacred objects of any kind. Again, this corroborates the thesis that the virgins worked on the temple sacramentals.
The virgin weavers certainly weren't making cheap garments for the general populace so their listing with the priests is making a connection between the two.
However the original premise was a cult of temple virgins located within the temple, perhaps with special quarters.
The book of 2 Baruch fails reliability and accuracy in several ways, not the least in that Baruch didn't write it.
2 Baruch says God or angels on His behalf, hide the temple's sacred accouterments.
However Dan. 5:1-3 describes Belshazzar and his party drinking from the sacred vessels Nebuchadnezzar had taken from the temple.
2 Baruch 10:1-5 says Baruch stayed back while Jeremiah went to Egypt but Jeremiah 43:6, 7 says both Jeremiah and Baruch were taken off to Egypt together.
These are not small lapse's. And as one reads 2 Baruch it becomes apparent that it is in every way a fanciful myth.
Baruch is turned into the prophet while Jeremiah is rather like his servant in a role reversal. Baruch is carried around by angels and angels destroy Jerusalem's walls. I could go on but why do so?
You said we know Jeremiah hid sacred objects in a cave but “we know” no such thing. It would seem an important point yet the authentic book Of Jeremiah says nothing of the kind, nor most of what is falsely attributed to Baruch.
So I think any argument based on 2 Baruch would have to show it is anymore than a ginned up story by an unknown writer/s.