Well, we did it. I finally finished the book that I had been working on with my co-author for the last two years. I thought I’d write a short post on my experiences writing a technical book and getting it published. I know many people think about writing books, and I’d like to share my experiences so that others might learn from lessons that I learned the hard way. Overall, it was an absolutely amazing experience and I have a feeling that the adventure is only beginning….
Pick an Interesting Subject…
The first thing you’ll have to do is decide what you want to write about. Now, you could have an amazing idea, but the key thing you have to do is convince a publisher that your idea is interesting enough to publish AND that a book is the right medium for this. In today’s day, books aren’t always the best way to distribute technical knowledge, so you have to be reason why a book is the right answer.
In order to get a publisher’s attention, you’ll need to have:
- An interesting idea
- An outline of the proposed book
- Some evidence that you are the right person to write the book
- Some evidence that you can write coherently
- An understanding of the market (IE what competing books are there)
This was actually my third attempt at book writing. I had a contract to write a data science book, and attempted to get a contract for a machine learning in security book.
…that a publisher will want to print…
Getting a publisher is the next big step in the book writing process. If you buy books about how to write a book, they’ll tell you how hard it is to get a publisher. First of all, in the tech world, I don’t really think you need an agent to get a book contract, but it can be difficult to get a response from the publishing companies. The key thing that you have to understand as a potential author is that publishers exist to sell books and that your idea, while interesting, may not be the right idea for a book. A good publisher will help you take your idea and translate it into a marketable publication.
Should You Self Publish?
You might be thinking that it’s not worth all the hassle and just publish the book yourself. There are many publish-on-demand companies that you can use to publish the book yourself, and the quality of the physical books are quite good.
In my opinion there are a few reasons not to do this.
- Distribution: One of the key things that publishers do is distribute your book to bookstores, and it can be quite difficult and time consuming to do this yourself. I would imagine that most people (including myself) have little understanding of what it entails.
- Editorial Quality: One of the things I’ve noticed about self-published books is that the editorial quality never seems to be as good as professionally published books. There are exceptions, but in general, that seems to be the case. When we were going through the various review phases, I was very impressed with the attention to detail that the O’Reilly editorial staff paid to our manuscript.
- Credibility: I don’t like to think of myself as a snob, but having a reputable publisher publish your work is a recognition of quality work. Whereas, anyone with a few dollars can self-publish a book. I’ll leave this by telling a story. When my son saw my book cover, he told me that it was like one of the books in Weird Al’s White and Nerdy video. (And in case anyone is wondering, I do actually own THAT book!)
Ultimately, I’ve always felt that it is worth the effort to work with a major publisher because the end product will be better. Major publishers have really solid editorial teams, and your end result will be a lot better. There are of course, exceptions to this rule.
Now Start Writing…
Now that you have your publisher, the next step is to actually write the book. For me, when I started this project, I was one of several co-authors, so the burden did not entirely fall on me. However, one of the initial challenges we faced at the time was who is the book for and how much detail do we want in the book? In our book, we ultimately came to the conclusion that there are three potential audiences:
- Drill Users: This audience would have a more analytic skill set and would be comfortable writing SQL queries. This audience is going to want to know how Drill can improve their analytic work and so the book deals a lot with how to use Drill to access and analyze difficult data sources as well as merge data together. (Basically me)
- Drill Developers: Developers would be individuals who will be writing code to extend the functionality of Drill. They will need to understand the internals of Drill and the nuances of developing for Drill. Paul wrote these sections because he is a rockstar Drill developer.
- Drill Administrators: Since Drill can be used with large distributed systems, it goes without saying that systems administrators need to understand how to properly set up and configure Drill on a cluster. (As an aside, one of the things that I like about Drill is that unlike most big data tools, Drill can also be used quite effectively on small data) Sysadmins will care about things like optimizing CPU, Memory, Security etc. The problem here was that neither Paul nor I really had this expertise in this area going into the project, so we had to do a lot of research in these sections.
Pro tip: when you submit a book proposal, your publisher will like you if you have an outline down to the subsection level. This shows that you are serious and your project will go much more smoothly.
I’m not the fastest writer in the world, but it’s bloody difficult to write 300+ pages coherently. I don’t know where I heard this, but someone wiser than me said that the way to write a book is to set a small goal every day and write at least one page per day. When I committed to this project originally, I thought that my contribution would take about 3-4 months to complete at that rate. The problem I encountered was that a lot of what I was writing about was completely undocumented and I had to research it from scratch. Furthermore, what was documented I also had to research to find all the “gotchas” and little details that often aren’t in technical documentation. So one page a day, more accurately was one paragraph a day, if I was lucky. It actually wasn’t even so orderly… it was more like do a bunch of research, then write up what I figured out.
Once the chapters were complete, and in my case, at least 5 or 6 deadlines were missed, I’d finally get a chapter to the editor. Fortunately, my editor seemed to like our style and didn’t really make us rewrite very much. (They even left my sarcastic comments and nerd references in the book!) Once it’s all done, then the book goes through various reviews, proof reading etc. until months later, it finally arrives at your door via FedEx!
Is it worth it?
The first thing I would caution any perspective book author is that it is going to take you A LOT more time than you think. Take the amount of time you think it will take, and multiply that by two. Then take that number, and multiply THAT by two and that’s probably a more realistic number. As for financial compensation for your time, book royalties are not that much. You can expect between 5%-10% for hard copy and up to 25% for online sales. Bottom line here is that you should not expect to make significant amounts of money from the book.
So if there’s no money in it and it’s a huge time suck, why do it? I didn’t work on the Drill book to make money. I worked on it because I believe that Drill is a really powerful tool that not many people know about and I wanted to change that. That is why I worked on the book. If I make some money, or get some consulting work from it, great, but that’s not why I did it. I guess my conclusion is that technical book writing has to be a labor of love.