I made a thread on here, still thinking about it. As long as you don’t have to move 150lb packages, I’d be alright. I’m a small guy and don’t weigh 150, so moving that would be very hard for me without at least a simple machine.
I've seen little guys make it in freight. You learn how to handle freight without too much lifting. You use tools, such as fork lifts and two wheelers. Work smart. If you can get your CDL and are old enough, you might be able to skip the dock work, but it's a good idea to do some before going to driving because you learn how to handle freight.
Driving P&D for an LTL company can be a great job if you have a good attitude, enjoy meeting people, and have patience in traffic (which is no problem when you're paid by the hour.)
Once you get in line haul, physical size is completely irrelevant. The stress is the intensity of long drives, and working at night.