General & Miscellaneous Software, Network Programming, Internet & World Wide Web - General & Miscellaneous, Searching the Web, Scripting Languages, Web Programming, Intelligent Agents

Webbots, Spiders, and Screen Scrapers: A Guide to Developing Internet Agents with PHP/CURL

Name: Webbots, Spiders, and Screen Scrapers: A Guide to Developing Internet Agents with PHP/CURL
Author: Michael Schrenk
ISBN: 9781593273972

by Michael Schrenk

Available on Bookshop Write a review

Books.org participates in affiliate programs including Bookshop.org and the Amazon Services LLC Associates Program. We may earn a commission from qualifying purchases made through links on this page, at no additional cost to you.

Overview

The Internet is bigger and better than what a mere browser allows. Webbots, Spiders, and Screen Scrapers is for programmers and businesspeople who want to take full advantage of the vast resources available on the Web. There's no reason to let browsers limit your online experience-especially when you can easily automate online tasks to suit your individual needs.

Learn how to write webbots and spiders that do all this and more:

* Programmatically download entire websites
* Effectively parse data from web pages
* Manage cookies
* Decode encrypted files
* Automate form submissions
* Send and receive email
* Send SMS alerts to your cell phone
* Unlock password-protected websites
* Automatically bid in online auctions
* Exchange data with FTP and NNTP servers

Sample projects using standard code libraries reinforce these new skills. You'll learn how to create your own webbots and spiders that track online prices, aggregate different data sources into a single web page, and archive the online data you just can't live without. You'll learn inside information from an experienced webbot developer on how and when to write stealthy webbots that mimic human behavior, tips for developing fault-tolerant designs, and various methods for launching and scheduling webbots. You'll also get advice on how to write webbots and spiders that respect website owner property rights, plus techniques for shielding websites from unwanted robots.

As a bonus, visit the author's website to test your webbots on sample target pages, and to download the scripts and code libraries used in the book.

Sometasks are just too tedious-or too important!- to leave to humans. Once you've automated your online life, you'll never let a browser limit the way you use the Internet again.

This text first outlines the deficiencies of browsers, and then explains how these deficiencies can be exploited in the design and deployment of task-specific webbots. Readers will learn how to write stealthy webbots that read email, emulate online forms, auto-authenticate, manage cookies, and handle encryption.

About the Author, Michael Schrenk

Michael Schrenk develops webbots and spiders for clients across North America. He has written for Computerworld and Web Techniques magazines and has taught college courses on web usability and Internet marketing. He's also an occasional speaker at DEFCON.