An interesting exercise that most reasonably competent programmers can do now is to setup their own websearch engine with Common Crawl, Elastic MapReduce, JSoup or Gumbo or equivalent HTML parser, and ElasticSearch. It costs on the order of high-hundreds to low-thousands of $$$ to process the ~2B webpages in the Common Crawl corpus (about half the size of the first tier of Google's index) and stand up an ElasticSearch cluster with it for a few days.
You too can build your very own search engine.
Unfortunately, the result quality is roughly what AltaVista was like in 1995. That is why people keep going back to Google.
You too can build your very own search engine. Unfortunately, the result quality is roughly what AltaVista was like in 1995. That is why people keep going back to Google.