That is a well founded concern. There is the possibility of liver damage from Niacin. The solution: have a periodic blood test that will show up any liver problems. That is what I do.
Also it is worth pointing out that there are other adverse effects from Niacin. Flushing is the primary one that people worry about. Using a time release (over-the-counter) version and taking a baby aspirin just before the Niacin pretty much takes care of that problem.
A more important issue, and one that seems to be less well understood is the effect of too much Niacin. Typical dosages are in 500 mg tablets (I'm talking over-the-counter here, not prescription versions). I take two a day at the same time and have no problem. At the urging of a former cardiologist I did at one time take 2000 mg (4 tablets). The result: 24/7 nagging stomach and extreme lethargy. This went on for several months until I did a self diagnosis, guessed that it was the Niacin, and quit abruptly. All symptoms were gone in less than 24 hours.
The bottoms line: take it easy and find out your tolerance for the Niacin. Make sure you get your liver tested periodically - probably every 6 months at first and then maybe once a year or so. Ask your doctor and do what he says.
Good advice & my thanks.