People who do not pay taxes don't deserve a "refund."
You still payroll taxes and it is intended as a partial refund of those taxes to avoid the otherwise regressivity of them.