Ever since I published my first book, this question seems to come up a lot. In all honesty, I asked that same question a few times to other author's I knew before I wrote that book. Now that my second book is coming out in a few short weeks, I figured I'd write a bit about these experiences.
First, let me point out that the differences between writing the first book and the second book were enormous. The basic things I was hearing when I was asking around about writing a book was that it was a major time sync. It was difficult, and time consuming. Yet minor things like weren't going to deter me!
For anyone who knows me, they probably know that I'm verbose. I tend to 'type' a lot, and many times they think it's just because I like to read the words I'm thinking on screen. For me, writing the first book wasn't difficult at all. I found the experience rewarding, and while not something I would consider 'easy', given my knowledge of the subject, and my knack for rambling on while writing, it just seemed to come naturally.
The second book on the other hand, was much more difficult. For one, my real job beckoned, and the little free time I had when writing the first book basically disappeared. It doesn't matter how good of an author you are, you suck if you can't write. Then, when you only have a short period of time any given day or week to write, you barely get anything written. Then with long stretches between writing sessions, you forget where you were in your train of thought, and have to go back and re-read the last sections you've already written to get back into the writing mode.
So, to answer the underlying question, I suppose the best answer is 'It depends!'. If you have a passion for writing and know you'll have the time to dedicate to writing (at least an hour or two a night, 3-5 days a week), then no, not really. On the other hand, if you lack those two things, it can be a taxing experience to say the least.
It's definitely not something I regret, let me tell you that much.