Entrepreneurs, the Surface Web, Deep Web and the Dark Web. Entrepreneurs in 2016 need to understand and learn more about the Deep Web and the Dark Web because of its implications for businesses of all sizes. So let’s get started: What we commonly call the Web is really just the surface. Beneath that is a vast, mostly uncharted ocean called the Deep Web, and as a subsection of the Deep Web we have the Dark Web (confusing right?). We all know the internet that we all use every day. The Surface Web is comprised of the usual sites such as Google, Facebook, Amazon, Twitter, Instagram and everything else a search typically shows. The top university researchers say the Web you know – YouTube, Baidu, Alibaba and all other sites from all over the world – makes up less than 1% of the entire World Wide Web. By its very nature, the size of the Deep Web is really difficult to calculate, some reports claim the Deep Web is 90% or 95% of the whole World Wide Web. But nobody really knows what’s the correct percentage.
The Deep Web
When you surf the Web, you really are just floating at the surface. Dive below and there are tens of trillions of pages – an unfathomable number – that most people have never seen. They include everything from boring statistics to drugs for sale online (illegally of course). The Deep Web is little understood however the concept is quite simple. Think about it in terms of search engines. To give you results, Google, Yahoo, and Microsoft’s Bing constantly index pages. They do that by following the links between sites, crawling the Web’s threads like a spider. But that only lets them gather static pages, like the one you’re on right now. What they don’t capture are dynamic pages, like the ones that get generated when you ask an online database a question (Databases are designed to offer an organized mechanism for storing, managing and retrieving information). When the web crawler arrives at a database, it typically cannot follow links into the deeper content behind the login or search box. Google and others also don’t capture pages behind private networks or standalone pages that connect to nothing at all. These are all part of the Deep Web.
The vast majority of the Deep Web holds pages with valuable information. One report estimates that over half of all websites are databases. Among the world’s top 10 largest are The World Data Centre for Climate, National Energy Research Scientific Computing Center, Central Intelligence Agency (CIA) and Library of Congress. Another large chunk of pages lies hidden because they’re only found on an Intranet. These internal networks – for example at corporations or universities – have access to message boards, personnel files or industrial control panels that can flip a light switch or shut down a power plant. Of the websites already mentioned earlier, Google, YouTube, and Amazon are also on the top 10 list with the largest databases.
The Dark Web – and Problems for Your Business
Entrepreneurs need to understand the dark web because of its implications for businesses. Many of the services offered via the darknet may pose a threat to your company. It is very easy for your information to be stolen, duplicated and quickly sold, all done in total anonymity and little risk of consequence. On the Dark Web, a Social Security number costs just US$ 1.00 and medical records go for around US$ 50. Just think about if your credit card or bank details were to be put up for sale. Or your website or online shopping site being hacked by fraudsters.
Most people, who have ever even heard about the Dark Web, probably got to know it in 2013 when the very infamous “Silk Road” web site was shut down by the FBI. The Silk Road was an online black market and the first modern darknet market, best known as a platform for selling illegal drugs. The Dark Web also hit the headlines in August 2015 after it has been reported that a large amount of data stolen from Ashley Madison, a site designed to enable bored spouses to cheat on their partners, was dumped on to the Deep and Dark Web.
It’s not all shady activity. The Dark Web isn’t totally full of criminal activity. The Dark Web (i.e. Tor – The Onion Router) was initially developed by US military researchers (in the mid-nineties) to allow intelligence operatives to exchange information completely anonymously and local protestors being muzzled by their governments abroad could communicate to fight for their freedom. In more recent years others, such as whistleblowers (like Bradley Manning & Edward Snowden) and activists (Aaron Swartz & Julian Assange for example) or Bitcoin users (an innovative payment network and a new kind of money) also take advantage of the anonymity. Regardless of everything, you should be aware that the internet is for many people still both mysterious, unexplored and sometimes very difficult to understand.
The Dark Web and Tor Anonymity Network. The Dark Web is a term that refers specifically to a collection of websites that are publicly visible, but hide the IP addresses of the servers that run them. Thus they can be visited by any web user, but it is very difficult to work out who is behind the sites. And you cannot find these sites using traditional search engines, like Google, Yahoo or Bing. Almost all sites on the so-called Dark Web hide their identity using the Tor Encryption Tool. You may know Tor for its end-user-hiding properties. You can use Tor to hide your identity and spoof your location. When a website is run through Tor it has much the same effect.
Deep Web and Dark Web Definitions. The dark web is often been confused with the deep web, the parts of the web not searched by the well-known search engines. The key thing to keep in mind is the Dark Web is a very small portion of the Deep Web. Some media is still confused about these two different terms and seeing them as one and the same and we want to do our best to clear up the confusion. This confusion actually dates back to at least 2009. Since then, especially in reporting on the Silk Road site shut-down, the two terms have often been mixed up despite many recommendations that they are distinguished.