Celeron by itself isn't necessarily bad, and MS Windows by itself isn't necessarily bad. However, in my limited experience, the two combined together are definitely bad news.
....I should add that today's Celeron may be better. I remember when they first came out with the Celeron, all the tech people I knew were warning me away from buying anything with a Celeron processor because of issues of compatibility with Windows. I think this was around the time that Windows 98 was coming out as well.
However, that was before Intel started making the celeron. Now, they might be okay, but I am pretty leery of using anything other than a Pentium just because of those issues that cropped up several years ago.