Cooling in the past have been accompanied by famine as crops failed.
That's true. It's already happening, as we enter this new solar minimum. Wheat yields were off a full third in some places during 2017. This is going to increase, as the minimum gets deeper.
By some estimates, we won't begin to come out of it until 2050.