Why Spidering Hacks?

The term hacking has a bad reputation in the press. They use it to refer to someone who breaks into systems or wreaks havoc with computers as their weapon. Among people who write code, though, the term hack refers to a "quick-n-dirty" solution to a problem or a clever way to get something done. And the term hacker is taken very much as a compliment, referring to someone as being creative, having the technical chops to get things done. The Hacks series is an attempt the reclaim the word, document the ways people are hacking (in a good way), and pass the hacker ethic of creative participation on to the uninitiated. Seeing how others approach systems and problems is often the quickest way to learning about a new technology.

Spidering Hacks is about coming up with easier, more automated ways of performing everyday tasks and gathering information. Yes, there are legal issues to consider, and we address them in this book before delving into any code whatsoever. Likewise, there are many right and wrong ways to scrape and compile information; the best practices in this book create wholesome spiders, as opposed to harmful or out-of-control automatons.