Проверка Вашего сайта на битые ссылки как это делают поисковые роботы (для Linux). Check your site for broken links as Crawler or Spider (useful for Linux)



Вот плюшки (buns: ) ):
The "Spider" or "Crawler"

- HTTP/1.1 compliant with persistent connections and cookies support
- HTTP Basic authentication supported
- HTTP Proxy support (basic authentication included)
- Crawl customisable through many configuration attributes which let the user
limit the digging on URLs pattern matchings and distance ("hops") from the first URL.
- MySQL databases directly created by the spider
- MySQL connections through user or general option files as defined by the
database system (/etc/my.cnf or ~/.my.cnf)

А вот ложка дёгтя (oil tar: ):
No support for Javascript and other protocols like HTTPS, FTP, NNTP and local files.
Отправить комментарий

Популярные сообщения из этого блога