Unit 8: Search Engines and Web Sites

Post-Christmas Greetings, dear Reader!  Here's hoping your holiday season is going well.
  Today's blog post will concentrate on two misnomers: Search Engines and Web Sites.  One sounds like a mechanical thing, and the other sounds like it should have a spider on it.  Who names these things anyway?
  Seriously, today's blog post will concentrate on the features of various methods of searching for information.
  First up: The Search Tools.  They are used by most people for everyday-type searching, like when you want a recipe for fried okra.  Search tools have a huge list of websites that are found using a little program called a 'spider' or a 'bot' that 'crawls' around in the 'web' searching for new websites.  This little spider program doesn't look to see how useful the website is to anybody.  It's just looking for a website.  Once found, that website is examined to see what it is all about.  This is just a sort of once over.  Key words from the website that describe what that website is all about and many of it's contents are stored in the Search Tool's database and linked to the website.  That way, when you want to search for something, say, that recipe for fried okra, then the Search Tool can point you in the right direction.  "The thing you are looking for might be in that website.  At least, I think it's over there, because the words you are looking for are there.  Click on that link and see if it's what you want."
1.  One search tool that I had never used before but had heard about is Lycos.  I decided to give it a try.
  Interesting note: Lycos appears to derive it's name from a the genus name of the large wolf spider, Lycosa.  I see a little geek humor, there.
  Lycos can be found here.  It does support advanced searching.  However, to be able to do that you need to scroll down to the bottom of the page and click on the 'help' link almost in the middle of the bar at the bottom.  Once on the next page you will find the words, "advanced search" in tiny print above the search button on the upper right.  Click on that, and an advanced search window will open.
  Above the search window Lycos tells you that you can truncate or stem from your original term by using an asterisk strategically.  Also, "+include or -exclude words with +/-".   Lycos, by default, will put an 'and' between the words you input into the search box unless you stipulate otherwise.
  If that's not easy enough, in the list below the search box you can click buttons that will allow you to:

• 1. find all words (default: the word 'and' is implied between search terms)
• 2. find any words (the word 'or' is implied between each search term)
• 3. find exact phrase (as if you had placed quotation marks around your whole search string)
• 4. by ID (this looks for a website by it's numeric ID)

 Below that list is the option to specify how recently the website was modified, or not.  Under that you can search in a particular Lycos product, or you can search within a particular category of product or branch of Lycos.
2. The Metasearch engine I chose is again one I have heard about but never used: Dogpile.  It can be found here.  The difference between a search tool and a metasearch tool is, well, I'll let Dogpile explain it for you:
"What is Metasearch? Metasearch means instead of getting results from one search engine, you'll be getting the best combined results from a variety of engines, and not just any engines, but industry leading engines like Google, Yahoo!, and Yandex."
  According to Dogpile, the information gleaned by search engines only overlaps about 1%, which means that searching with only one search engine might not get you the results you are looking for.
  Dogpile has an advanced search option.  It does not strictly allow you to use boolean logic, but it will allow you to search for:

• All of these words (which is the same as putting an 'and' between your search terms)
• The exact phrase (like you put quotes around it)
• Any of these words (as if you put an 'or' between your search terms)
• None of these words (as if you put a 'not' or a minus sign before each word)
• Language (search in a specific language)
• Include or Exclude domain (allows you to specify a certain domain, like any .edu site)

  Dogpile might also take a little longer than a search engine, but if your computer and your internet speed are fast anyway you probably won't notice the difference. 
   However, neither a Search Tool nor a Metasearch tool can take you past the front door of websites that are password protected.  That is, you have to have a password that works as a key to allow you to search within the site.

  One such type of website that can also acts as a kind of search tool is a Subject Directory.  A Subject Directory is usually part of the deep web and contains things that cannot usually be found on the web that is available to the public at large, but there are exceptions to this rule.  One such is About.com which allows free access to its database.  For the most part, however, Subject directories are closed to the general public.  That is, they are not easy to access.  ‘Spiders’ or ‘bots’ from search engines can’t go into these to find out what they contain.  You have to have a password, and sometimes you have to input characters that are displayed in unique fashions in order to prove you are not a ‘bot’.  Subject Directories are generally run by actual people.  They choose what are in these repositories, and who gets access.

3.  One such that I chose is Digital Librarian.  It has books available for browsing, pictures, music, movies, maps ... in fact, most of the things you can access at a normal public library.  However, it does not contain items that are not in the public domain.  That is, it does not contain works that are copyrighted because to do so would violate copyright laws. However, it does contain links to other deep websites that require passwords and sometimes require membership fees.

  The Digital Librarian itself does not require the user to use a password.  Only when you link to pages like the de Young Fine Arts Museums of San Francisco are you required to become a member and use a password.

  So there you have it, the difference between Search Tools or Search Engines, Metasearch Tools, and Subject Directories.