Webcrawler uses an interface
to search through all its databases for sites that contain all the words
in the query. It then searches for the individual words in individual pages.
The pages Webcrawler likes and considers apt are added to a Yahoo-type
hierarchical directory system. Your site can be accessed through a search
or through the directories. Webcrawler also has a random URL service, web
reviews, as well as a "find similar site" service.
Click on the Add URL button
to submit a URL to Webcrawler. The tool allows you to add 9 URLs at a time.
It may take up to a fortnight for submissions to be added to the Webcrawler
To remove your site from
the Webcrawler database, place it in the robots.txt file. The following
visit the robot makes to your site, your site will be removed. The process
can be accelerated through submitting your URL again, so the robot knows
it has to visit your site.
To have your site return
on more searches you should include meta tags and not use irrelevant words
in your home page. Meta tags are good for graphical pages and for customizing
your keywords to what is appropriate to your page.
The constant repetition of
keywords, or spamming, is not appreciated by Webcrawler. Offending pages
are being removed. Once the spamming has been removed from a page,
it can be resubmitted and added to the index again.