A forty percent tariff on a five dollar shirt would be around forty cents retail. Around the same amount as the massive sales tax most states charge.
If a shirt retails for $5. It costs around $1 to purchase. The rest is delivery, warehousing, merchandising, overhead and handling costs.
It also takes into account that a textile product is among the most labor intensive of produced consumer goods.