Not at all. And wild figs are stunningly resilient. Even I can't seem to kill mine.
During a visit to Monticello, I saw that Jefferson's figs survive, even on a mountaintop in Virginia, because they're right up against a south-facing wall.
So I planted figs along our south-facing wall, which is brick, and the next door house blocks almost all wind except east wind. No north wind at all.
They don't even die back, just go dormant, even when we have hard freeze.
This is in Northern Va.