Got a research paper or thesis to write for school or an online class? Want to research using the Internet? Good luck. There’s a lot of junk out there — outdated pages, broken links, and inaccurate information. Using Google or Wikipedia may lead you to some results, but you can’t always be sure of accuracy. And what’s more, you’ll only be searching a fraction of all of the resources available to you.
Google, the largest search database on the planet, currently has around 50 billion web pages indexed. That’s a lot of information. But it’s nothing compared to what else is out there. Google can only index the visible web, or searchable web. But the invisible web, or deep web, is estimated to be 500 times bigger than the searchable web. The invisible web comprises databases and results of specialty search engines that the popular search engines simply are not able to index.
Do you think your local or university librarian uses Google? Sure, but certainly not exclusively. In order to start researching like a librarian, you’ll need to explore more authoritative resources, many of which are invisible. Note: Although some of the following resources are visible and indexed, they have all been included here because of their authoritative nature.
Topics Covered in this Article
Deep Web Search Engines | Art | Books Online | Business | Consumer | Economic and Job Data | Finance and Investing | General Research | Government Data | International | Law and Politics | Library of Congress | Medical and Health | STEM | Transportation
To get started, try using a search engine that specializes in scouring the invisible web for results. None of these can search the entire invisible web, but they make some inroads that Google has not as of yet.
- Clusty — A metasearch engine that combines the results of several top search engines.
- INFOMINE — A virtual library of Internet resources relevant to university students and faculty. Built by librarians from the University of California, California State University, the University of Detroit-Mercy, and Wake Forest University.
- Internet Archive — A database of tens of thousands of movies, live music, audio, texts, and home of the Wayback Machine that allows you to find old versions of web pages, over 55 billion.
- DirectSearch — A list of hundreds of specialty databases and search engines. No longer maintained, but still perhaps the most complete list of the deep web.
- The WWW Virtual Library — Started by none other than Tim Berners-Lee, the VLib is not the biggest index of the Web, but it is one of the highest-quality.
Hundreds of other museums all over the world have their own databases. This is just a list of the databases for a few of the most popular. Within these databases are descriptions of the work, its location, and an image of the work.
- Musée du Louvre — One of the oldest and most famous art galleries in the world. Contains Leonardo da Vinci’s Mona Lisa and Alexandros de Antioch’s Venus de Milo.
- Guggenheim Museum — A collection of over 160 classical and new artists, searchable by artist name, title, date, movement, medium, concept, and museum.
- Smithsonian National Portrait Gallery — A searchable database of more than 80,000 portrait records.
- The National Gallery of Art — A searchable catalog of all of the museum’s 110,000 objects, with images of more than 6,000 available online.
There are tens of thousands of classic books with full text available online for free. Some authors whose books are available in full text online include Jane Austen, Charles Dickens, and Fyodor Dostoevsky.
- The Online Books Page — A searchable database of more than 25,000 English works with full text available for free online.
- Bibliomania — A searchable database of free online literature from more than 2,000 classic texts.
- Project Gutenberg — A searchable online catalog of more than 19,000 free books with full text available online.
- The National Academies Press — A searchable directory of 3,000 free books online and 900 for-sale PDFs.
- SearcheBooks –Crawl the Web for free downloadable books. Just type in the title and go.
The most accurate quantitative business knowledge often comes from specialty search engines and business directories/databases. If you’re enrolled in one of the best online business administration programs, these types of resources may prove beneficial.
- agencycompile — A searchable database of advertising and other marketing agencies.
- Kompass — A business-to-business search engine.
- GPO Access Economic Indicators — A portal to economic indicators from January 1998 to present.
- CBDNet — A searchable database of materials published by the U.S. Department of Commerce.
- Hoover’s — A searchable database of businesses.
- SBA Loan Data — A database of loan program approval activity from the Small Business Administration.
- GuideStar.org — A searchable database of non-profit organizations.
For consumers, authoritative product reviews and pricing information often times are hidden in the deep web or among subscription-based reports.
- U.S. Consumer Products Safety Commission Recalled Products — A database of recalled products, searchable by month and year, company, product type, and/or press release number.
- Melissa Data — A directory of complete demographic data, sortable by ZIP code.
- Kelley Blue Book — A guide to pricing new and used vehicles.
- Edmunds — Another guide to pricing new and used vehicles.
- Consumer Reports — A guide to consumer product reviews, including cars, appliances, electronics, computers, personal finance, travel, and more.
A wealth of information regarding microeconomic and macroeconomic data is available from government sources and other organizations.
- EUROPA Press Release Database — A searchable database of press releases distributed by the European Union.
- FreeLunch.com — A searchable directory of free economic data.
- Bureau of Labor Statistics — A directory of job-based and consumer economic data from the U.S. Department of Labor.
- Salary Wizard Calculator — A tool that shows national average salaries adjusted by location for different jobs.
- Economagic — A free economic data directory containing over 200,000 data files.
- Penn World Tables — A database of purchasing power parity and national income data for 179 countries for the years 1950-2007.
- America’s Job Bank — A searchable database of jobs and resumes.
- U.S. Office of Personnel Management — A searchable database of federal government jobs.
- Regional Economic Conditions (RECON) — A database of economic data available by state, county, and MSA.
A variety of web-based investment analysis tools and financial statistics can only be found in the deep web. Financially minded students, such as those enrolled in top online accounting programs, will find these resources useful.
- Bankrate.com — A directory of interest rates for different types of loans, mortgages, and savings accounts.
- InvestIQ — A database of market data from different world regions.
- BigCharts — A searchable database of quotes and performance charts for diffjournal with full text of current and past issues available online.
- Medical Matrix — A search engine for indexed and catalogued medical research results, peer-reviewed by physicians and medical librarians for quality.
- ClinicalTrials.gov — Search nearly 150,000 clinical studies from 182 countries around the world.
Science and engineering journals and databases are searchable through specialty search engines, while real-time geological data is available from the USGS.
- ScienceResearch.com — A portal allowing searchable access to numerous scientific journals and databases.
- Science.gov — A searchable gateway to authoritative science information provided by U.S. government agencies.
- The Complete Work of Charles Darwin — All of Charles Darwin’s published texts, fully searchable and available online.
- USGS Real-Time Water Data — A map of the United States showing real-time streamflow and water quality data of the nation’s rivers and reservoirs.
- USGS Earthquake Hazards Program — Maps of California, Nevada, the United States, and the World, showing real-time earthquake data.
- IRIS Seismic Monitor — The Incorporated Research Institutions for Seismology provide this map and other data on current and past quakes around the world.
- IEEE Publications — For engineers, this is a database of over 1.4 million documents from the Institute of Electronics and Electrical Engineers.
- TechXtra — Free access to reports, e-materials, research, industry news, and even job listings in the math, science, and engineering fields.
Real-time aviation data as well as historical and current transportation safety information is available from different U.S. federal government agencies.
- FAA Flight Delay Information — A map of the United States with flight delay information from the nation’s largest airports.
- NTSB Accident Database and Synopses — The National Transportation Safety Board’s database of aviation accidents, ranging from 1962 to present.
- National Highway Traffic Safety Administration — A database of car and car part defects, searchable by item number or car make and model.
- SaferCar.gov — Databases of vehicle, tire, and child restraint recalls searchable by manufacturer or time period.
- Flightwise — A real-time flight tracking service, with support for Google Earth.
- FlightAware — Another free flight tracking service, complete with history, graphs, and maps.
A special thanks to Gary Price at ResourceShelf for his thoughtful suggestions.