{"id":3028,"date":"2017-05-03T09:58:16","date_gmt":"2017-05-03T08:58:16","guid":{"rendered":"https:\/\/bestpractice.bmj.com\/?page_id=3028"},"modified":"2026-07-06T14:07:53","modified_gmt":"2026-07-06T13:07:53","slug":"clinical-search-big-data-and-the-hunt-for-meaning","status":"publish","type":"page","link":"https:\/\/bestpractice.bmj.com\/info\/us\/evidence\/discuss-ebm\/clinical-search-big-data-and-the-hunt-for-meaning\/","title":{"rendered":"Clinical search, big data, and the hunt for meaning"},"content":{"rendered":"

I\u2019ve been running the clinical search engine the Trip Database<\/a> for nearly 20 years and as it has evolved, opportunities have arisen for working with other sectors or individuals with different perspectives. Recently this has involved the academic info retrieval world; a chance conversation with academics at the University of Glasgow changed the way I looked at search. One really important notion I learnt about was clickstream data \u2013 the data websites collect of the user\u2019s interaction with the site. In the case of Trip, this data would equate to the search terms used and the articles users click on.<\/p>
\r\nOne thing\u2019s for sure, Trip has lots of it. With a million searches per month (the vast majority being health professionals), we\u2019ve amassed hundreds of millions of data points in the years since we started collecting it. This qualifies as big data. As with all big data projects the trick appears to be making sense of it all and that is a journey I feel we\u2019ve only just started.
\r\n
\r\nLet\u2019s take an example where a user searches for \u2018acne and minocyline\u2019; we could infer that the user was interested in the effectiveness of minocycline in the treatment of acne. The user may well then click on the Cochrane systematic review \u2018Minocycline for acne vulgaris: efficacy and safety\u2019 which might reinforce any inference. In isolation they may have very limited value, but aggregate the search behavior of hundreds of thousands of people and the results will reveal patterns of behavior that offer an insight into the uncertainties of clinicians. If we saw just a single search for \u2018minocycline and acne\u2019 we might conclude that it was of limited interest to health professionals, perhaps just a random search. But if it was searched a hundred times in a month, it would probably suggest an area of significant interest and uncertainty.
\r\n
\r\nTrip has been capturing the usage data (also known as clickstream data) since 2010 and has accumulated hundreds of millions of data points. But is this data useful? The first major analysis Trip undertook of our data was to map the articles users looked at in the same search session. The image below shows one example:
\r\n
\r\n\"\"
\r\n
\r\nIn the image we have selected searches for urinary tract infection and mapped the connections (connections being made when users click on the same article in the same session \u2013 the user links them based on their intention). As you can see, the articles form distinct topic clusters. In the bottom left there is a clear cluster of articles on UTI and cranberry. It seems reasonable to suggest that these 19 articles form the core articles on the topic; all selected by Trip users. Equally interesting would be the ones they never clicked on; so which articles don\u2019t appeal to our users? As \u2018impact\u2019 becomes an increasingly important concept, having articles that people don\u2019t look at should be equally as interesting as those that are clicked on.
\r\n
\r\nMore recently our journey\/analysis has been boosted by our work with the Technical University of Vienna (TUW) as part of the Horizon 2020-funded KConnect project<\/a>. The tools TUW have given us have allowed us to better understand the data. While a Twitter quiz might seem a superficial use of the data, it proved insightful. The first quiz asked which skin condition do users search for most, given the options of acne, eczema, and psoriasis. Over half thought the most popular search was acne, when in reality it was psoriasis. Similarly, when we asked which was most searched for out of influenza, measles, Zika, and malaria, the majority of users (80%) said Zika, when it was actually influenza. While the numbers were relatively small, it shows that it\u2019s not easy for individuals to guess where the uncertainties are. So, data-driven (\u2018evidence-based\u2019) analysis seems to be a useful tool in unearthing clinical uncertainties.
\r\n
\r\nThe data can be further analyzed to show that when people search for influenza, they most often also look for oseltamivir and the influenza vaccine. For Zika it was vaccine, embryopathy, and imaging!
\r\n
\r\nThe data even allows us to explore topic areas to look for patterns, and a recent sample of four weeks of data reveals that autism is the most popular topic looked for in the area of child health:
\r\n
\r\n\"\"
\r\n
\r\nThe data can be examined on a weekly basis, be it search terms or articles viewed. This allows us to see topical trends. Something might have been a source of great uncertainty but subsequent new evidence or guidance might give the certainty the health professionals need and therefore reduce the need to search for answers. We have shown, for instance, similar trends as shown by the Google Flu Trends service<\/a> \u2013 in that the number of queries we received closely matches the reported incidence of influenza.
\r\n
\r\nOur initial interest was one of curiosity: does the way our users interact with the site show anything useful? We think it\u2019s definitely interesting and potentially useful. For instance, we\u2019re already discussing using this data with a large research funder to help in the prioritization of new research. While neither side feels it gives a definitive answer, it\u2019s probably another piece of the jigsaw in understanding uncertainties. As such, it will complement more labor-intensive systems that are currently used.
\r\n
\r\nAuthor: Jon Brassey <\/strong>
\r\n
\r\n22\/09\/2016
\r\n
\r\nJon Brassey<\/strong> is the founder and director of the EBM search engine the Trip Database<\/a>. In addition to this he works as lead for knowledge mobilisation at Public Health Wales and is an honorary fellow at the Centre for Evidence-Based Medicine, Oxford. Clickstream analysis is a developing area for Trip and in the future this may have some commercial interest.<\/em>
\r\n
\r\nRead more<\/a>","protected":false},"excerpt":{"rendered":"

I\u2019ve been running the clinical search engine the Trip Database for nearly 20 years and as it has evolved, opportunities have arisen for working with other sectors or individuals with different perspectives. Recently this has involved the academic info retrieval world; a chance conversation with academics at the University of Glasgow changed the way I […]<\/p>\n","protected":false},"author":3,"featured_media":0,"parent":3027,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"marketing_new.php","meta":{"_acf_changed":false,"footnotes":""},"class_list":["post-3028","page","type-page","status-publish","hentry"],"acf":{"components":[{"choose_component":"generic","hide":"","logged_in":"","logged_out":"","grey_background":"","background_color":"","background_image":null,"title_text_color":"","title":"Clinical search, big data, and the hunt for meaning","text":"

I\u2019ve been running the clinical search engine the Trip Database<\/a> for nearly 20 years and as it has evolved, opportunities have arisen for working with other sectors or individuals with different perspectives. Recently this has involved the academic info retrieval world; a chance conversation with academics at the University of Glasgow changed the way I looked at search. One really important notion I learnt about was clickstream data \u2013 the data websites collect of the user\u2019s interaction with the site. In the case of Trip, this data would equate to the search terms used and the articles users click on.<\/h1>\r\nOne thing\u2019s for sure, Trip has lots of it. With a million searches per month (the vast majority being health professionals), we\u2019ve amassed hundreds of millions of data points in the years since we started collecting it. This qualifies as big data. As with all big data projects the trick appears to be making sense of it all and that is a journey I feel we\u2019ve only just started.\r\n\r\nLet\u2019s take an example where a user searches for \u2018acne and minocyline\u2019; we could infer that the user was interested in the effectiveness of minocycline in the treatment of acne. The user may well then click on the Cochrane systematic review \u2018Minocycline for acne vulgaris: efficacy and safety\u2019 which might reinforce any inference. In isolation they may have very limited value, but aggregate the search behavior of hundreds of thousands of people and the results will reveal patterns of behavior that offer an insight into the uncertainties of clinicians. If we saw just a single search for \u2018minocycline and acne\u2019 we might conclude that it was of limited interest to health professionals, perhaps just a random search. But if it was searched a hundred times in a month, it would probably suggest an area of significant interest and uncertainty.\r\n\r\nTrip has been capturing the usage data (also known as clickstream data) since 2010 and has accumulated hundreds of millions of data points. But is this data useful? The first major analysis Trip undertook of our data was to map the articles users looked at in the same search session. The image below shows one example:\r\n\r\n\"\"\r\n\r\nIn the image we have selected searches for urinary tract infection and mapped the connections (connections being made when users click on the same article in the same session \u2013 the user links them based on their intention). As you can see, the articles form distinct topic clusters. In the bottom left there is a clear cluster of articles on UTI and cranberry. It seems reasonable to suggest that these 19 articles form the core articles on the topic; all selected by Trip users. Equally interesting would be the ones they never clicked on; so which articles don\u2019t appeal to our users? As \u2018impact\u2019 becomes an increasingly important concept, having articles that people don\u2019t look at should be equally as interesting as those that are clicked on.\r\n\r\nMore recently our journey\/analysis has been boosted by our work with the Technical University of Vienna (TUW) as part of the Horizon 2020-funded KConnect project<\/a>. The tools TUW have given us have allowed us to better understand the data. While a Twitter quiz might seem a superficial use of the data, it proved insightful. The first quiz asked which skin condition do users search for most, given the options of acne, eczema, and psoriasis. Over half thought the most popular search was acne, when in reality it was psoriasis. Similarly, when we asked which was most searched for out of influenza, measles, Zika, and malaria, the majority of users (80%) said Zika, when it was actually influenza. While the numbers were relatively small, it shows that it\u2019s not easy for individuals to guess where the uncertainties are. So, data-driven (\u2018evidence-based\u2019) analysis seems to be a useful tool in unearthing clinical uncertainties.\r\n\r\nThe data can be further analyzed to show that when people search for influenza, they most often also look for oseltamivir and the influenza vaccine. For Zika it was vaccine, embryopathy, and imaging!\r\n\r\nThe data even allows us to explore topic areas to look for patterns, and a recent sample of four weeks of data reveals that autism is the most popular topic looked for in the area of child health:\r\n\r\n\"\"\r\n\r\nThe data can be examined on a weekly basis, be it search terms or articles viewed. This allows us to see topical trends. Something might have been a source of great uncertainty but subsequent new evidence or guidance might give the certainty the health professionals need and therefore reduce the need to search for answers. We have shown, for instance, similar trends as shown by the Google Flu Trends service<\/a> \u2013 in that the number of queries we received closely matches the reported incidence of influenza.\r\n\r\nOur initial interest was one of curiosity: does the way our users interact with the site show anything useful? We think it\u2019s definitely interesting and potentially useful. For instance, we\u2019re already discussing using this data with a large research funder to help in the prioritization of new research. While neither side feels it gives a definitive answer, it\u2019s probably another piece of the jigsaw in understanding uncertainties. As such, it will complement more labor-intensive systems that are currently used.\r\n\r\nAuthor: Jon Brassey <\/strong>\r\n\r\n22\/09\/2016\r\n\r\nJon Brassey<\/strong> is the founder and director of the EBM search engine the Trip Database<\/a>. In addition to this he works as lead for knowledge mobilization at Public Health Wales and is an honorary fellow at the Centre for Evidence-Based Medicine, Oxford. Clickstream analysis is a developing area for Trip and in the future this may have some commercial interest.<\/em>","image":null,"qr_image":null,"circular_image":null,"button_text":"Read more","button_url":"https:\/\/bestpractice.bmj.com\/info\/us\/evidence\/ebm-toolbox\/bibliography\/","button2_text":"","button2_url":"","video_embed":"","quote":"","quote_author":"","author_job_title":"","feature_iconsimages":null,"bullets":"none","bullet_text":null,"expiration_date":null,"review_type":"normal","reviews":null,"feature_image_cards":null,"feature_icon_cards":null,"podcast_date":null,"podcast_title":"","podcast_text":"","podcast_embed_script":"","campaign_advert_link":"","monthly":{"show_monthly_tab":false,"strikethrough_price":"","price_text":"","per_month_text":"","vat_text":"","subscribe_button_text":"","subscribe_button_link":""},"annual":{"show_annual_tab":false,"packages":null,"default_price":"","default_strikethrough_price":"","default_per_year_text":"","vat_text":"","subscribe_button_text":"","subscribe_button_link":""},"advisory_panel":null,"article_list_fields":null,"subscription_tab_main_heading":"","subscription_tabs":null,"accordion_for_page":null,"embed_code":"","table_structure":null,"column_two_text":""},{"choose_component":"bulletList","hide":"","logged_in":"","logged_out":"","grey_background":[],"background_color":"","background_image":null,"title_text_color":"","title":"Related links","text":"","image":null,"qr_image":null,"circular_image":null,"button_text":"","button_url":"","button2_text":"","button2_url":"","video_embed":"","quote":"","quote_author":"","author_job_title":"","feature_iconsimages":null,"bullets":"ticks","bullet_text":[{"text_data":"What does evidence-based actually mean?<\/a>","bullet_icon_url":""},{"text_data":"What is the best evidence and how to find it<\/a>","bullet_icon_url":""},{"text_data":"Simply making evidence simple<\/a>","bullet_icon_url":""},{"text_data":"Six proposals for EBMs future<\/a>","bullet_icon_url":""},{"text_data":"Promoting informed healthcare choices by helping people assess treatment claims<\/a>","bullet_icon_url":""},{"text_data":"The blind leading the blind in the land of risk communication<\/a>","bullet_icon_url":""},{"text_data":"Transforming the communication of evidence for better health<\/a>","bullet_icon_url":""},{"text_data":"Living systematic reviews: towards real-time evidence for health-care decision making<\/a>","bullet_icon_url":""},{"text_data":"The rise of rapid reviews<\/a>","bullet_icon_url":""},{"text_data":"Evidence for the Brave New World on multimorbidity<\/a>","bullet_icon_url":""},{"text_data":"Genetics and personalized medicine: where\u2019s the revolution?<\/a>","bullet_icon_url":""},{"text_data":"Policy, practice, and politics<\/a>","bullet_icon_url":""},{"text_data":"The straw men of integrative health and alternative medicine<\/a>","bullet_icon_url":""},{"text_data":"Where\u2019s the evidence for teaching evidence-based medicine?<\/a>","bullet_icon_url":""}],"expiration_date":null,"review_type":"normal","reviews":null,"feature_image_cards":null,"feature_icon_cards":null,"podcast_date":null,"podcast_title":"","podcast_text":"","podcast_embed_script":"","campaign_advert_link":"","monthly":{"show_monthly_tab":false,"strikethrough_price":"","price_text":"","per_month_text":"","vat_text":"","subscribe_button_text":"","subscribe_button_link":""},"annual":{"show_annual_tab":false,"packages":null,"default_price":"","default_strikethrough_price":"","default_per_year_text":"","vat_text":"","subscribe_button_text":"","subscribe_button_link":""},"advisory_panel":null,"article_list_fields":null,"subscription_tab_main_heading":"","subscription_tabs":null,"accordion_for_page":null,"embed_code":"","table_structure":null,"column_two_text":""},{"choose_component":"generic","hide":["yes"],"logged_in":"","logged_out":"","grey_background":"","background_color":"","background_image":null,"title_text_color":"","title":"Log in or subscribe to access all of BMJ Best Practice","text":"