Tutorials

Step-by-step instructions of essential use cases // Last updated: 11.01.2024

Expected searches in systematic reviews (and meta-analyses)

Find the principal databases for 'systematic keyword search' (mandatory).
The goal of systematic reviews (and most meta-analyses) is to identify all relevant literature that meets the eligibility criteria in a transparent and reproducible fashion. To identify (closest to) all records, we need databases that allow us to search a large corpus of potentially relevant literature. As comprehensive keyword searches were found to be mandatory in systematic reviews (1,2), you must ensure the principal databases you rely on adequately support it. We recommend keyword searches at two or more large, multidisciplinary databases that cover most of your subject's records. We recommend searching at least one specialized database for each subject your review touches on. This means if your review topic touches both medicine and computer science, you should include a database specializing in medicine and one specializing in computer science.

These recommendations present minimum requirements for principal databases in systematic reviews. Thus, you may search more databases - even some that miss some critical functionality - in addition to the principal databases. We base these recommendations on the Cochrane Handbook's MECIR expectations (3), considered the "benchmark in systematic review methodology" (4) and a methodical review (5) that find that combinations of large, multidisciplinary, and specialized databases are good choices for good search coverage.

Follow these steps at Search Smart to identify databases with high absolute coverage (STEPS1-4) and relative coverage (STEPS5-6): 
  1. Select subject(s): At the Search Smart landing page, select the discipline(s) your database should cover. (in our example, we select: "Medicine") 
  2. Select 'Systematic keyword searching': This option ensures that the database provides keyword-searching functionality - detailed criteria can be found in the FAQ: "When to select 'systematic keyword searching'?"
  3. Click 'Start comparison': By default, absolute subject coverage (descending) is selected if you have chosen a subject filter. 
  4. Done. Review results sorted by absolute subject coverage: a list of databases mainly covering medicine and that support keyword searching. In the Find! view, you see a list of databases supporting your selected search functionalities sorted according to the number of records they cover in your selected disciplines. These databases will cover the most relevant documents. (in our example: Lens, Dimensions, and SciTech Premium Collection cover most and support systematic keyword searching)
  5. Sort by 'Relative subject coverage (desc)':
  6. Done. Review results sorted by relative subject coverage: a list of databases specialized in medicine and that support keyword searching. These databases will be the most specialized in your discipline, covering the least irrelevant records. (in our example: Epistemonikos, Cochrane Database of Systematic Reviews, CINAHL Plus, and Embase)
  7. IMPORTANT: Ask whether potentially relevant databases cover what you seek. Review the identified databases, particularly in terms of the record type coverage and retrospective coverage. (in our example: the most specialized database Epistemonikos covers journal articles and reviews, but only after 1990 in more significant numbers, so how relevant is it for what you seek?)
  8. OPTIONAL: Manipulate the filter settings if you have specific requirements for the database.
  9. OPTIONAL: Save your selection for future reference. With the URL link, you can update it at any time.


Tutorial


Consider: 
  • In keyword searching, make sure you search with a comprehensive search string. The best databases will not help if you cannot meaningfully access the documents based on keyword matches. In doubt, ask a librarian for assistance with search string formulation.
  • Based on a thorough keyword search at suitable databases, the guidance recommends backward (mandatory) and forward citation searching (highly recommended) for a comprehensive search strategy.
  • Report your search process transparently. The most popular multidisciplinary guidance is the PRISMA checklist and the PRISMA Flow Diagram.
  • Follow conduct guidance to ensure you search most comprehensively. We suggest the Cochrane Handbook (1) and its technical supplement to searching guidance (6), considered the methodical benchmark in systematic reviews. 
References:
  1. Higgins, J., Thomas, J., Chandler J, Cumpston, M., Li, T., Page, M., & Welch, V. (Eds.). (2022). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). www.training.cochrane.org/handbook
  2. Xiao, Y., & Watson, M. (2019). Guidance on Conducting a Systematic Literature Review. Journal of Planning Education and Research, 39(1), 93–112.
  3. Higgins, J. P. T., Lasserson, T., Chandler, J., Tovey, D., Thomas, J., Flemyng, E., & Churchill, R. (2022). Methodological Expectations of Cochrane Intervention Reviews (Version February 2022). Cochrane.
  4. Briner, R. B., Denyer, D., & Rousseau, D. M. (2009). Evidence-Based Management: Concept Cleanup Time? Academy of Management Perspectives, 23(4), 19–32.
  5. Bramer, W. M., Rethlefsen, M. L., Kleijnen, J., & Franco, O. H. (2017). Optimal database combinations for literature searches in systematic reviews: A prospective exploratory study. Systematic Reviews, 6(1), 245.
  6. Higgins, J., Thomas, J., Chandler J, Cumpston, M., Li, T., Page, M., & Welch, V. (Eds.). (2022). Cochrane Handbook for Systematic Reviews of Interventions: Technical Supplement to Chapter 4: Searching for and selecting studies. www.training.cochrane.org/handbook


Find databases for 'backward citation search' (mandatory).
In systematic reviews, backward citation searching is mandatory (1).
 
Backward citation searching via a comprehensive database offers two advantages: 1) it provides instant access to additional information (instead of title information only), 2) if backward citations of multiple records are screened at once, one might do so via a de-duplicated list, thus saving time of screening a share of duplicate references twice. Be careful with backward citation information in databases, some records might be missing. Before you start, verify whether the database's backward citations are accurate. From our experience, Scopus and Web of Science will provide the most accurate backward citation information. If you do not have access to a suitable database, you may just manually screen the reference lists of full texts.

Follow these steps at Search Smart: 
  1. Switch to the Find! view
  2. Open 'Citation search' in the filter bar
  3. Select 'Backward citation searching'
  4. Done. Review the sorted results, a list of databases supporting backward citation searching.
  5. Per default, databases are sorted via their total coverage (in descending order).
You may directly adopt this selection via this link.

References:
  1. Higgins, J., Thomas, J., Chandler J, Cumpston, M., Li, T., Page, M., & Welch, V. (Eds.). (2022). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). www.training.cochrane.org/handbook
Find databases for 'forward citation search' (highly recommended).
In systematic reviews, forward citation searching is highly recommended, particularly when concepts are difficult to capture via keyword searching.

Follow these steps at Search Smart: 
  1. Select 'Forward citation searching'
  2. Click 'Start comparison'
  3. Done. Review the sorted results, a list of databases supporting forward citation searching.
  4. Per default, databases are sorted via their total coverage (in descending order).
You may directly adopt this selection via this link.
Consider:
  • In general, larger databases that offer forward citation searching will report more complete citation information than smaller ones.
  • Currently, Google Scholar is the gold standard in the completeness of forward citation information. It captures - by far - the most forward citation information across quality levels. Its forward citation information includes citations in journals but also by gray literature. If one chooses to collect forward citation information only for peer-reviewed journal publications, one should consider a more restricted citation index, such as the one offered by Scopus or Web of Science.
  • Please be aware of Google Scholar's limited access and download options (see detailed information on Search Smart). To circumvent some of its downsides, you may use Publish or Perish software to systematically access and download its forward citation information. However, even with the help of this software, you may only access up to 1000 citations per query. 

Find databases via coverage

Find databases that cover most records in my discipline.
Follow these steps at Search Smart: 
  1. Select your subject(s): At the Search Smart landing page, select the discipline(s) your database(s) should cover. (in our example, we select: "Medicine, Dentistry, and Nursing").
  2. Click 'Start comparison'
  3. Done. Review the sorted results, a list of databases covering most record in Medicine, Dentistry, and Nursing. (in our example, Google Scholar covers most, with an estimated 102.76 million records of all different types)
  4. Per default, absolute subject coverage (descending) is selected if you have chosen a subject filter. 
Consider: 
  • Search Smart lists the most popular bibliographic databases in academia. However, there might be some databases missing, particularly those with a non-English focus. 


Tutorial

Find the most specialized database(s) in my discipline.
Follow these steps at Search Smart: 
  1. Select subject(s): On the landing page, select the subject(s) your database should cover. (in our example, we select: "Medicine" ) 
  2. Select 'Rank databases according to relative subject coverage'
  3. Click 'Start comparison'
  4. Done. Review the sorted results, a list of the most specialized databases in your discipline.
Consider: 
  • Search Smart lists the most popular bibliographic databases in academia. However, there might be some databases missing, particularly those with a non-English focus. 


Tutorial

Find databases with most 'clinical trial' coverage.
Follow these steps at Search Smart: 
  • Select 'Clinical trials'
  • Click 'Start comparison'
  • Done. Review the sorted results, a list of databases covering most 'clincial trials'.
You may directly adopt this selection via this link.

Consider: 
  • Search Smart lists the most popular clinical trials databases. However, there might be some databases missing. Please contact us if you feel we miss an important database. 
Find databases with most 'Covid-19' coverage.
Follow these steps at Search Smart: 
  • Select 'Keywords'
  • Click 'Edit keywords'
  • Select 'covid-19'
  • Click 'Select keywords'
  • Click 'Start comparison'
  • Done. Review the sorted results, a list of databases covering most 'Covid-19' records.
You may directly adopt this selection via this link.

Consider: 
  • The keyword search function on Search Smart will inform you about the number of records on the database that have this keyword in the title, not all records that are about that topic. With this standardized way of measuring coverage, we can estimate the importance of a topic on a database.
  • There are a number of dedicated Covid-19 repositories collecting research on the topic. We currently do not cover such focused databases, yet they might be helpful for searches aiming at high Covid-19 specialization. Some of these repositories are:
Find freely accessible databases with most open access coverage.
Follow these steps at Search Smart: 
  • Select 'Only show non-paywalled databases'
  • Click 'Start comparison'
  • Sort by 'Open access coverage (desc.)
  • Done. Review the sorted results, a list of freely accessible databases covering most open-access records.

You may directly adopt this selection via this link.
Find databases covering most records from before 1900.
Follow these steps at Search Smart: 
  • Switch to the Find! view
  • Set 'Retrospective coverage' to 1000-1899
  • Done. Review the sorted results, a list of databases covering most records from before 1900. (Currently, Bielefeld Academic Search Engine hosts the most ancient records, while the share of ancient records of all records covered is highest at JSTOR, which also specializes in providing old knowledge.)
You may directly adopt this selection via this link.

Tutorial

Access detailed information on a database's subject, record type, open access, or retrospective coverage.

You can find all detailed information about the coverage of a database in the 'detailed view'.

Follow these steps at Search Smart: 
  • Switch to the Find! view
  • Select the database of your interest (in our case we select Bielefeld Academic Search Engine)
  • Review the database's coverage. (review the information on: subject, record type, open access, and retrospective coverage)
Information on how much a database covers from specific types of records is essential for determining whether the database should be selected for search. A database might or might not cover the records from publication years or subjects you are interested in, or it provides or does not provide the types of records you need. To help you to assess the currentness of the data, we include the date the coverage information was last updated at Search Smart.


Tutorial

Find databases via functionality

Find databases for the discovery of “hot” papers in journals.
Follow these steps at Search Smart: 
  • Switch to the Find! view
  • Open 'Sorting options' in the filter bar
  • Select 'Attention/Alternative metrics (not citations)'
  • Open 'Results filters' in the filter bar
  • Select 'Journal title'
  • Done. Review the sorted results, a list of databases allowing you to discover “hot” papers in a journal.
You may directly adopt this selection via this link.

How to continue?
  • Continue by navigating to the #1 suggested database 'Dimensions' .
  • At Dimensions: To see what papers are "hot" in a journal, you can filter in 'Source Title' for Science and sort via 'Altmetric Attention Score' . You can further limit your results to specific publication years so you see what was "hot" last year, etc. To directly access this example query for"hot" papers in Science in 2022, follow this link here.


Tutorial


A variation to find databases allowing sorting via 'number of citations':
  • Open 'Sorting options'
  • Select 'Citation number'
  • Done. Review the sorted results, a list of databases allowing you to discover "most cited" papers in a journal.
Find databases that support citation classification filters.

Citation classifications give users additional information on how records were cited. For example, a citation may support the findings of the original records, yet it may also contradict them. Other classifications give insight into which document section the original article has been cited or which record type cited the original document. Further, some databases also offer to search within citation statements.

Follow these steps at Search Smart: 
  • Switch to the Find! view
  • Open 'Results filters' in the filter bar
  • Select 'Citation filters'
  • Done. Review the sorted results, a list of databases supporting citation classification filters (Scite_, for example, offers citation classifications so you may identify citations that support/contradict a specific publication.)
You may directly adopt this selection via this link.

Tutorial

Find databases with which I can search through full texts.
Follow these steps at Search Smart: 
  • Switch to the Find! view
  • Open 'Query' in the filter bar
  • Select 'Full-text search'
  • Done. Review the sorted results, a list of databases allowing full-text search.
You may directly adopt this selection via this link.

Consider:
  • Some databases that support full-text search only allow it for a subset of all records they cover. So, for some records full-text search will be available; for others, you can only access meta information such as on the record's title or abstract.
Access detailed information on a database's search operators, filters, field codes, sorting, and download options.
You can find all detailed information about the functionality of a database in the 'detailed view'.

Follow these steps at Search Smart: 
  • Switch to the Find! view
  • Select your database of interest (in our case, we select Bielefeld Academic Search Engine)
  • Review the database's search options. (review the information on search operators, filters, field codes, sorting, and download options)
Search Smart saves the verbatim information on the search options a database offers. These options are categorized so that they can be compared across databases. With the use of verbatim information, users know exactly which options they might use as input for queries and search strings. Additionally, you see the dates the information was last updated at Search Smart.

Tutorial

Detailed comparisons

Compare the functionality and coverage details of two to six databases.
To illustrate the insights that may be gained from detailed comparisons of databases, we want to compare the six most specialized databases in the Arts and Humanities.

Follow these steps at Search Smart: 
  • Pin two to six databases that interest you (in our case, we investigate the most specialized databases in the Arts and Humanities: Arts & Humanities Citation Index (via Web of Science), JSTOR, Conference Proceedings Citation Index - Social Science & Humanities (via Web of Science), Emerging Sources Citation Index (via Web of Science), Social Sciences Premium Collection (via ProQuest), and Sociological Abstracts (via ProQuest)
  • Click 'Compare'
  • Done. Review the detailed comparison of the databases' coverage and the systems' functionalities.


Tutorial


Interpretation of the exemplary comparison (you may directly adopt this selection via this link):
  • RETROSPECTIVE COVERAGE VIEW: The retrospective coverage comparison shows coverage differences between databases. Particularly, in specific year spans, individual databases cover more records than any other of the six. For example, JSTOR covers more from 1900 until 1986 than any other database and offers substantially more coverage of pre-1900 records. After that, the Social Sciences Premium Collection covers significantly more than any other of the compared databases. While these numbers reflect the total coverage of all records across subjects, we may assume that subject coverages likely change proportionally.
  • SUBJECT COVERAGE VIEW (RELATIVE/ABSOLUTE): The Arts & Humanities Citation Index (via Web of Science) is by far the most specialized in Arts & Humanities, also covering a substantial share of Social Sciences records. However, its total coverage in Arts & Humanities records is surpassed by JSTOR.
  • RECORD TYPE COVERAGE VIEW (RELATIVE/ABSOLUTE): All compared databases mainly cover journal publications. The Social Sciences Premium Collection (via ProQuest) also covers many non-scholarly records, such as magazines, news, and trade publications.
  • SEARCH FUNCTIONALITIES: All databases except JSTOR are paywalled. Each database's search system offers different functionalities - you need to review these functionalities closely to select the most suitable database for you. 
Compare the same database offered by different search systems.
Some databases are so popular that multiple search systems provide them. While the databases' coverages are (mostly) similar, the functionalities offered by the search systems differ significantly. E.g., a system may provide different search operators, field codes, or download options. Search Smart lists the same databases offered by different search systems to inform users about differences in coverage and functionality.
Follow these steps at Search Smart: 
  • Switch to the Find! view
  • Pin two to six similar databases (in our case, we investigate Medline provided by Web of Science, EBSCOhost, and Ovid. Additionally, we select PubMed, Europe PMC, and Virtual Health Library because they mainly consist of the Medline database.)
  • Click 'Compare'
  • Done. Review the detailed comparison: in terms of the databases' coverage and the systems' functionalities.
Interpretation of the exemplary Medline (+PubMed, VHL, and Europe PMC) comparison (you may directly adopt this selection via this link)
  • The retrospective coverage comparison shows all databases have similar retrospective coverages, yet some have distinct timespans where they offer more or less than the others (i.e., Europe PMC and PubMed). 
  • The subject foci of the databases are almost identical (relative subject coverage), yet some databases offer a higher absolute coverage within these subjects (i.e., Europe PMC). 
  • The comparison of record-type coverage is not available for all databases. We see that all we can compare (i.e., Europe PMC, Virtual Health Library, Medline (via EBSCOhost) report high coverages of journal articles. As the record types these database cover are reported at different levels of granularity, we recommend accessing official database information for further information.
  • The systems that provide the databases are very different in the functionalities they offer (review them at Search Smart).
Consider:
  • The different databases and search systems were each tested at different times. Thus, some of the differences between databases, particularly some of their coverage differences, will be attributable to the different testing times. However, there will also always be specific differences in the coverages of databases that are not explainable by the testing date. Some search systems are faster at updating databases than others.
  • Functionality differences will be based on actual differences rather than on testing time differences. You may review the last update times in the comparison charts in the 'compare view' and for each test in the 'detailed view'. 
Compare databases that offer the same record type.

Sometimes it is important to choose a database based on the type of records it offers. For example, if someone seeks to identify theses or dissertations, specialized databases cover this record type exclusively. 

Follow these steps at Search Smart: 
  • Switch to the Find! view
  • Select your record type of interest (in our case, 'Theses')
  • Sort by 'Record type coverage (desc.)'
  • Pin two to six databases that interest you (in our case, we pin database with high thesis coverage: WorldCat - Thesis/dissertation; Dissertations & Theses Global (via ProQuest); Open Access Theses and Dissertations)
  • Click 'Compare'
  • Review the detailed comparison, in terms of the databases' coverage and the systems' functionalities.
You may directly adopt this selection via this link.
Compare multiple versions of the same database.
Researchers might be interested in what specific advantages one version of a database brings over another. For researchers, this information is relevant because version differences may affect how they can search and how much they find. For librarians, this information is relevant because they need to assess whether pricing differences are warranted by the performance differences of database versions. While search providers typically emphasize differences in journal coverage, Search Smart provides a supplementary view of differences in database versions.

Follow these steps at Search Smart: 
  • Switch to the Find! view
  • Pin two to six databases that interest you (in our case, we pin two versions offered by EBSCOhost: Academic Search Elite and Academic Search Premier)
  • Click 'Compare'
  • Review the detailed comparison, in terms of the databases' coverage and the systems' functionalities.
  • In our example, it is apparent that both databases' retrospective coverage and subject coverage are almost identical. As we reviewed both databases on the same day, coverage assessments are highly comparable: As of the review date, the Premier version offered some 740k (+1.86%) more records than the Elite version. The functionalities provided by both versions were reviewed to be identical, except for the Premier version offering one additional "references available" filter and having a reduced max. keyword length it can compute (100 instead of 250 keywords for 'wide' field codes).
You may directly adopt this selection via this link.

Consider:
  • So far, Search Smart has reviewed only a few databases in different versions. This is because Search Smart focuses on offering the most diverse set of different databases first. With this use case, we want to illustrate the in-depth analyses made possible by Search Smart. 
Compare clinical trials databases.
Follow these steps at Search Smart: 
  • Select 'Clinical trials' as record types
  • Click 'Start comparison'
  • Pin two to six clinical trials databases that interest you (in our case, we pin Cochrane Library - CENTRAL, ClinicalTrials.gov, EU Clinical Trials Register, and WHO International Clinical Trials Registry Platform)
  • Click 'Compare'
  • Review the detailed comparison, in terms of the databases' coverage and the systems' functionalities.
You may directly adopt this selection via this link.

Search Smart functions

How to use presets?
Search Smart offers three pre-defined filter settings, so-called presets, to identify the best database(s) for you:
  1. 'systematic keyword searching'
  2. 'forward citation searching'
  3. 'grey literature searching'
Presets are a quick way to filter databases for specific search requirements. They give a good first selection of databases and are a good way to start thinking about the critical capabilities systems need to offer. Each of these presets filters databases based on each search type's specific requirements. In filtering with presets, we distinguish requirements that must be fulfilled by databases (minimum requirements) and those that would be nice to be met (desired requirements). If a database fulfills all minimum requirements, its capabilities are rated 'sufficient'. If it also meets all desired requirements, its capabilities are rated 'extensive'. If it fails to fulfill a minimum requirement, it is rated 'limited' and cannot be used for this type of searching or should only be used with great caution (depending on the specific limitation).

The "elevated functionality requirements" button allows you to raise the bar and filter for databases that meet not only the minimum requirements but also the desired ones (details see FAQs: presets).

It is important to note that your specific requirements might differ from what we think are requirements for certain search types. Thus, review the preferences and adjust the filters to your situation. We encourage you to play around with filter and preset settings so you build an understanding of what functions are available overall and which databases support what.

Tutorial

How to download a report of my Search Smart results?
After you have received a sorted list of suitable databases for you, you may download it as a PDF for future reference. In this PDF, you see all databases that met your criteria and which ones did not and why. Your selection is also saved there so that you can update your selection at any time in the future. As Search Smart updates information periodically, you may find different databases that meet your criteria in the future.

Follow these steps at Search Smart: 
  • After you selected all the criteria important to you...
  • Click 'Save selection to PDF'
  • Done. Review the sorted results with all search filters you selected.
Pinned databases 0/6
Trusted by
JKU LinzUniversität Innsbruck
Funded by
FWF