I got vacations last Friday and decided to have fun with parallel computing & networking.
Introduction
I wrote a framework that I called "PegasusHPC is a High Performance Crawler", which is a web spider aimed at allowing high performance and scalability. I chose the image on the left as a tribute to pinkie pie, who found critical & amusing bugs in the chromium project.
I think this framework will be useful for me and potentially to other people, since this kind of tools happen to be useful for security testing purposes and other web related ones. It's been open source released and you can find both the first beta release and the code in its web site.
read more.........http://guifreruiz.blogspot.com/2013/03/PegasusHPC-is-a-high-performance-crawler.html?spref=tw