Siirry pääsisältöön

Systematic searching: 10. Evaluating and testing the search result

Evaluating the search result

The appropriate number of search results is not straightforward, especially when doing a systematic review. If the topic has been studied extensively, the set of search results will naturally be large. If, on the other hand, there has been very little research on the topic, you may need to broaden your search to find all the articles that at least touch on the topic. 

Your main focus should be on justifying your search results in your report based on your carefully formulated search plan. 

Evaluating your search terms and search strings

Once you have completed your search, it is worthwhile to look at the results and reflect further:

  • Do the search results match what you searched for?
  • Are there any relevant terms missing from the search?
  • Are all the terms used relevant and useful for the search?
  • Are there more terms in the search results that you could add to your own search?
  • Do any terms consistently produce unwanted search results?

You can evaluate your search with the PRESS checklist, a condensed version has been produced by the Karolinska Institutet. If you wish, you can consult the original checklist (see Table 1 in the linked publication). 

In our example topic, "Nursing students' hand hygiene skills", one could consider if it is necessary to include terms related to skills in the search, or whether it is sufficient to search using only the terms "nursing students" and "hand hygiene". If you search using search terms already related to nursing education or student nurses, do all search results already automatically relate to skills or competencies acquired during studies, without including terms related to skills or competencies? 

Testing your search string

It is advisable to test and review your search string before running your final search. 

  • Have you found relevant articles through manual or preliminary searches or from your tutor?  
    • Make a list of relevant articles already found. This is often called the gold set.
    • After this run your search strategy. Search for a gold set article by it's title. Check if the article is included in your search strategy results by combining your final search strategy results line with the gold set article using AND. If the gold set articles are not included, you should re-evaluate the comprehensiveness of your search string, since a systematic search should locate all relevant documents on a topic.  
    • The more articles you test, the more assurance you will have your search strategy is not missing relevant results.
  • You can compare your search strategy with other search strategies on the same topic. 
  • If you are choosing between two separate searches, more comprehensive and more narrow, there is an useful way to examine which search works better for your purposes:
    • Make a more comprehensive search first
    • Then do a narrower (more precise) search.
    • Go to the database search history and exclude narrower search results from the wider search. This is done using the NOT operator between sets of results. Are there any relevant results in the new search result list? If found, it is either necessary to further modify the search strategy or to use the wider search strategy.

                                     

 

Kuva: congerdesign