Mark J. Parry1, Travis S. Dowdle2, Jesse N. Steadman3, Tiffany R. Guerra1, Kim L. Cox4
doi: http://dx.doi.org/10.5195/ijms.2020.653
Volume 8, Number 3: 257-262
Received 13 07 2020: Rev-request 08 09 2020: Rev-request 20 10 2020: Rev-recd 15 09 2020: Rev-recd 04 11 2020: Accepted 24 11 2020
ABSTRACT
Background:The Papanicolaou smear (Pap smear, Pap test) is one of the most routine screening tests performed in medicine. The development and widespread use of this test has brought a considerable decrease in the incidence of cervical cancer. Unfortunately, this disease process continues to convey significant morbidity and mortality. These persistent phenomena may be the result of inadequate compliance with routine Pap smear screening, in which limited education is thought to play a role, particularly among ethnic minority groups.
Methods:A Google search using the phrase “pap smear” was performed and the first fourteen web addresses were analyzed using four standardized readability indices: the Flesh-Kinkaid Grade Level, the Simple Measure of Gobbledygook, the Gunning Fog Index and the Automated Readability Index. The average grade level readability was then compared to the American Medical Association recommendation that health care information should be written at a 5th or 6th grade reading level (i.e., ages 10-12 years).
Results:The average grade-level readability values of the fourteen analyzed sites using the four aforementioned indices were 8.9, 8.8, 11.9, and 8.4, respectively. The mean readability of all four indices was 9.5.
Conclusion:The grade-level readability of commonly accessed internet information regarding Pap smears is above the recommendation of the American Medical Association. Health care providers and website authors should be cognizant of this, as it may impact compliance. This is particularly important given that this routine healthcare test is recommended for nearly fifty percent of the world's population at various points throughout their lifetime.
Keywords: Papanicolaou test; health literacy; early detection of cancer; search engine; comprehension; reading (Source: MeSH-NLM).
Introduced by George Papanicolaou in the first half of the 20th century, the Papanicolaou smear (Pap smear, Pap test) is an important screening method for cervical cancer.1 The goal of a Pap smear is to identify cervical cells suspicious for pre-cancer or cancer.2 To do this, a small number of cells are sampled from the patient's cervix by a health care provider. These cells are then prepared and evaluated microscopically for irregularities.2 If abnormalities are identified, a diagnostic colposcopy with cervical biopsy is performed to better categorize the cervical changes, following which, an individualized treatment plan is designed based on the patient's findings.2,3 Treatment may include destruction of the affected cells with extreme temperatures, removal of cervical tissue, or chemotherapy coupled with surgery or radiation.
The United States Preventive Services Task Force (USPSTF) recommends that women ages 21 to 29 receive a Pap test once every three years.3 When a woman reaches the age of 30, recommended screening can be performed in one of three ways: a Pap test every three years, high-risk human papillomavirus (hrHPV) testing every five years or co-testing with both a Pap test and hrHPV testing every five years. hrHPV testing evaluates for oncogenic types of HPV.2 Positive hrHPV testing, as with a positive Pap test, indicates an increased risk of developing cervical cancer.
Since its inception, the Pap smear has experienced widespread utilization and brought a substantial decline in the incidence of cervical cancer.2 Unfortunately, this screening method continues to be underutilized. The American Cancer Society cites the 2018 median compliance rate with cervical cancer screening recommendations at 85% and estimates that in 2020 there will be 13,800 new cases of invasive cervical cancer diagnosed and 4,290 subsequent deaths within the United States (US).2,4
Various studies in the US have recognized limited education as a potential barrier to cervical cancer screening.5-8 One study found that the number of women who correctly understood the term Pap smear was fewer than 10%.9 This is especially true among Hispanic women who scored the lowest among all ethnic groups on a questionnaire measuring knowledge of Pap testing; this demographic is also significantly more likely to have never had a Pap test.5,10,11 Data from the Centers for Disease Control and Prevention (CDC) indicates that Hispanics have the highest ratio of HPV-associated cervical cancer relative to other ethnicities, demonstrating a rate of 9 per 100,000 women.12 While cultural factors including the fear of finding cancer and language barriers appear to play a role in this discrepancy, it is the lack of knowledge regarding cervical cancer screening that will be further explored here.5
In addressing the limited knowledge and relatively low screening rates of Pap smears among certain demographics, easy access to comprehendible patient-education material becomes vitally important. In the current era, the internet is a common source for this health information. A study performed in 2013 found that the majority of US adults reported searching online for health information in the past year with over one third of respondents attempting to “self-diagnose” a particular medical condition.13 Among Internet search engines, Google is used most frequently, holding 86.86% of the global market share amongst all search engines.14
Google search data has shown how women react to important public health news, such as the Irish “CervicalCheck” scandal in 2018 when over 200 women were given incorrect Pap smear results.15 After this information broke to the public, Google searches for “cervical check” and “cervical cancer” rose substantially, and the conduction of Pap smear tests increased by 40% in the subsequent weeks. Given the unpredictable nature of such public health crises, it is important that online health materials are periodically evaluated.
The American Medical Association (AMA) recommends health information be written at a US 5th or 6th grade reading level.16 Therefore, the aim of this investigation was to evaluate the compliance of online reading material related to Pap smears with this recommendation. As few studies of this nature have been performed in the field of obstetrics and gynecology (OB/GYN), the effect of this work is expected to be particularly insightful to both providers and patients.17-19 Expecting consistency with prior readability studies, the hypothesis of this work is that the grade-level readability of online material on Pap smears is written at a grade level greater than what is recommended by the AMA.20-23
The readability of online health information has been evaluated in the past using standardized indices.19-23 These metrics have been discussed in papers by the National Cancer Institute and the Centers for Medicare and Medicaid Services.24,25 From these sources and further research on readability measures, four indices that each provide a grade-level readability were selected for this study.19-28 Considered together, these provide a reliable average readability for written materials; each measures readability in a unique way. The indices utilized are described below:
The history and cache on the Google search engine within the Google Chrome browser were completely cleared and the phrase “pap smear” was searched on June 13, 2020 in the US state of Texas. Various permutations of the search term “pap smear” could be employed by patients depending on their background and life situation. In order to account for this variance, the authors agreed that the best method in determining an overall grade-level readability of patient materials related to Pap smears would be to query on the topic itself, rather than related keywords and phrases. To validate this decision, a review of data available from Google Trends—an application that charts relative interest over time for selected search queries—was conducted.29 Four potential searches (“pap smear”, “cervical cancer”, “pap test” and “pelvic exam”) were compared within the three categories of “Texas”, “United States” and “Worldwide” to gain a sense of the relative popularity of the phrase “pap smear” within these regions.
With the chosen query of “pap smear,” the first 14 uniform resource locator (URL) results, excluding educational videos and advertisements, were selected as the aggregate to be evaluated. The authors determined the quantity of inclusions from an analysis of a large dataset measuring search engine user behavior by a metric entitled click-through rate (CTR).30 The website “Advanced Web Ranking” was used, which averages monthly Google CTR data from millions of keywords. Within the site, the categories “international” and “all devices” within the “year over year” grouping for the year 2019 were analyzed.30
CTR is a measure of the likelihood that a click will occur when an advertisement is placed at a given location in the query and has also been applied to URL results in Google searches.30,31 To explain this further, the CTR value for the first position in a Google search query from 2019 was 34.07.30 That is, the likelihood that a person clicks on the first link in a given Google search is just over one third. Looking at the first 14 URL results gives an aggregate CTR value of 98.90, making this an in-depth measure of the total material that a patient inquiring online for health information may view.
The grade-level readability from the four aforementioned readability indices (FKGL, SMOG, FOG, ARI) was calculated using an online software from WebFX.32 This is a verified online tool recommended for educators to guide their students.33 The educational text from each website was first copied and pasted into a Microsoft Word document. All advertisements were deleted, tables were excluded, titles that were not entire sentences were omitted, and lists or bullet points were converted into written sentence form. In instances where complete sentences could not be established, these words were excluded from the data entry. The objective was to ensure that the text inputted into the program was as close to the actual value as possible. In one instance (the 13th result in our query; a page entitled “Pap test” from Wikipedia), the set of text was too large to be evaluated using the WebFX tool. To accommodate this, the text was divided into 10 sections of about 300 words each. The sections were then individually entered, after which an average of all 10 readability values in each of the 4 specific indices was obtained.
Once the 4 indices had been obtained for each of the 14 included URLs (Table 1), the values were averaged together in Microsoft Excel to give an aggregate grade-level readability of the online material from the selected query of “pap smear”. The cutoff for grade-level readability was set at 6.3, in conjunction with the AMA's recommendation that healthcare information be written at a 5th or 6th grade reading level.16 6.3 was chosen in place of 6.0 because healthcare vernacular is inherently difficult to understand and a prior study found that the grade-level readability obtained by the FKGL would decrease by 0.3 if medical vocabulary were removed.23,34
Table 1.Comparison of Website URL Position in a Query Following a Google Search for “pap smear” and Average Grade-Level Readability as Determined Using Four Standardized Readability Indices.
Google Trends data from the week of June 7-June 13, 2020 using the search term “pap smear” gave a popularity value in Texas, the United States and worldwide of 63, 58, and 57 respectively, where a value of 100 represents peak popularity (Table 2). The phrase “cervical cancer” had values of 40, 43, and 63. The expression “pap test” showed values of 10, 7, and 23 and “pelvic exam” received values of 5, 7, and 6.
Table 2.“Interest Over Time” Values by Region for Four Different Search Terms Related to Cervical Cancer Screening using Google Trends Data Specific to the Week of June 7 – June 13, 2020, Peak Popularity for a Given Search is 100.
| Search Term | Texas | United States | Worldwide |
|---|---|---|---|
| “pap smear” | 63 | 58 | 57 |
| “cervical cancer” | 40 | 43 | 63 |
| “pap test” | 10 | 7 | 23 |
| “pelvic exam” | 5 | 7 | 6 |
The CTR data from 2019 showed that the click probability was much greater within the first seven results as compared to the second seven results (Figure 1). The subtotal CTR value for the first 7 URLs was 86.29 and the next 7 URLs accounted for 12.61, giving a total value of 98.90. Thus, the first 14 positions provide an aggregate of the vast majority of clicks that occur after a user inputs a query into a search engine.
Figure 1Probability of Clicks Occurring in the First Seven URLs vs. The Following Seven URLs of a 2019 Google Search, A Total of Fourteen URLs Are Shown with a Cumulative CTR Value of 98.90, Error Bars Represent Standard Deviation Based on Monthly Data from 2019.
The 14 sites were analyzed for readability using the FKGL, SMOG, FOG, and ARI with the average values being 8.9, 8.8, 11.9 and 8.4, respectively. Each individual URL together with its average grade-level readability is highlighted in (Table 1). These values ranged from 6.4 to 12.7. The URL that had the lowest average grade-level readability was from WebMD and the site with the highest average grade-level readability came from MedicineNet. Other websites analyzed that may be easily recognizable to the average health care consumer were: Mayo Clinic, Wikipedia, Cleveland Clinic and womenshealth.gov. The total average grade-level readability taking into account all 4 indices was 9.5 (Figure 2). Considering the first seven URLs and the next seven URLs separately, the values were found to be 9.1 and 9.9, respectively. The trendline of average grade-level readability was slightly upward, indicating that the mean increases as one moves to the URLs appearing later in the queue.
Figure 2Grade-Level Readability Values for First 14 URLs in a Google Search for “pap smear” Compared to Superimposed Static Lines Representing the Composite Average Grade-Level Readability of All URLs and the Recommended Grade-Level Readability by the AMA.
The grade-level readability of information from the popular search term “pap smear” obtained via a Google search is above what is recommended by the AMA. None of the URLs evaluated were below the chosen cutoff of 6.3 for grade-level readability. This is consistent with findings in other disciplines where the readability of healthcare information has been explored.20-23 There appears to be a paucity of research on similar topics in OB/GYN, however.17-19
When evaluating the first seven URLs, the average grade-level readability is more than two grade levels above what is recommended. This could imply that a number of patients are not finding readable information online about a topic that is of significant importance to women worldwide. If a woman receives Pap smears as part of routine screening throughout her life with no abnormalities, the minimum number of tests she will undergo is 11, making screening for cervical cancer one of the most frequent of all routine female cancer screenings.34,35 Thus, the potential relevance of this should not be ignored. Importantly, these findings may be particularly relevant for individuals with decreased health literacy or for those who speak English as a second language, given that Google Translate makes more errors in translating to another language when the original sentence is written at a higher grade level.36,37
It appears that the grade-level readability increases as one moves to the second page of results in the “pap smear” query performed. In this case, the limited data may suggest that the top-viewed websites possess more readable material for consumers. It is interesting to note, however, that the top “hit” in the query was from an article published by the Mayo Clinic with an overall grade-level readability of 10.3. This shows that while the readability of a particular domain certainly plays a role in determining which websites populate first in a given search engine, there are a host of other contributing factors. The particular set of strategies aimed at populating a link early in the search results is termed search engine optimization (SEO).38
Other factors that play a part in SEO include: the website's recognized expertise on a particular topic, relevancy of the site to the question asked, the overall quality of the website's content, the navigability of the site, and the location in which the search was conducted.38,39 Our analysis was not focused on the relationship between readability SEO, but rather analyzing the readability of websites that already had strong SEO ratings. Reassuringly, 3 of the first 8 URLs encountered are near the AMA's recommended grade-level readability level. This indicates that some of the information obtained via a Google search on Pap smears is being written at an appropriately readable level for patients.
Other websites may benefit from looking to such sites or involving patients and public partners as they prepare educational material for patients.
One significant limitation of this study is that only a single query was used in searching. A single query was chosen instead of multiple queries as “pap smear” was considered to be a broad enough search to give a representative sample of the Internet information that exists on the topic. Based on Google Trends data from the week in which the search was performed, this appears justified. The popularity of “pap smear” superseded the other three search terms demonstrably in both regional and US locales. Within the “worldwide” category, the phrase “cervical cancer” was slightly more popular, though the difference was marginal at 63 for “cervical cancer” and 57 for “pap smear”. Considering that Google uses location as one of the determining factors in which sites populate first following a search, the column for “Texas” (the location in which the search was performed) may be the most important.39 This column showed “pap smear” to be favored by more than 20 points.
The usage of a single query factored into the decision to evaluate 14 URLs instead of 7. It was believed that this could facilitate the discovery of some of the articles that would have been moved further up the list of results in similar search queries. No more than 14 websites were deemed necessary based on current behavior of online users as shown in the 2019 CTR data.
Another limitation of this study is that the FOG index gave higher average values than the other indices. This is, however, consistent with other published studies.18,20-23 This could be because the FOG is a unique metric for evaluating readability, which looks at the total number of words per sentence and how complex the words are. Medical terminology frequently employs the use of large, complex words as standard vernacular, which may help explain the higher value calculated by this index. Even if the FOG index was taken out of the analysis, the average grade-level readability would be 8.6, more than 2 grade levels above the AMA recommendation.
A final limitation is that this study did not consider additional reasons that vulnerable populations, such as Hispanic women, may have lower rates of cervical cancer screenings. This study was focused primarily on only one aspect of this complex issue: the grade-level readability of online materials on Pap smears. Other factors such as “fear of finding cancer”, male physicians, and language barriers have been noted as significant obstacles and could be further explored.5
The current era is one in which YouTube is the second most popular social media platform, garnering 1.9 billion users in 2020.40 In addition, the current COVID-19 pandemic has shifted much of school education to an online format and brought a huge uptick in the number of telehealth visits conducted.41,42 With this, it could be argued that the importance of audiovisual learning has never been greater. In our study, six of the first seven sites and ten out of the fourteen total sites had a video or image that was accessible to the viewer. Using readability indices alone, there is no way to account for the added educational value that these resources may confer. Thus, further studies could be performed to assess the significance of audiovisual learning in patient education. While it has been found that in certain scenarios audiovisual materials may be helpful for patients, this has not been widely examined.43,44 Factors within this domain that deserve further investigation include: the formulation of specific indices to measure the impact of audiovisual learning, the percentage of various cohorts that are audiovisual learners, and the potential impact of such findings on screening exam discrepancies amongst groups (such as ethnic minorities).
Another area deserving further investigation exists. There appears to be a tendency for patients to misjudge an abnormal Pap smear (one showing precursor lesions with malignant potential) as being consistent with a diagnosis of cervical cancer. One study found that nearly 1/3 of individuals who were asked the true-false question, “If you have an abnormal result on the Pap test: It means you have cancer” answered either incorrectly or “don't know”.11 This is worth exploring given the potential for physician confusion and patient mistreatment as a result of this inadequate understanding. This would seem especially pertinent in patients where some form of treatment for a precancerous lesion (such as loop electrosurgical excision procedure (LEEP) or cryotherapy) is required.
Finally, readability is only one aspect of patient education. An equally meaningful study could examine the accuracy of the content contained on the most frequently viewed websites for the search term “pap smear.” This could be accomplished by having a panel of experts blindly review each webpage and score them for correctness, thus providing a supplement to the important findings of the current study.
Medical information may be inherently difficult to understand. While the overall grade-level readability of articles discussing Pap smears via a Google search appears to be better than that of other healthcare readability papers, it still exceeds what is recommended by the AMA.18, 20-22 This discrepancy is significant given that the Pap smear is a routine test recommended for nearly fifty percent of the population at various points throughout their lifetime. The findings of this study should guide healthcare providers and website authors alike to be more cognizant of the information that is transmitted online to patients with the ultimate goal of decreasing the grade-level readability to what is suggested by the AMA.
None
The Authors have no funding, financial relationships or conflicts of interest to disclose.
Conceptualization: MJP, TSD, JNS, KLC. Data Curation: MJP, TSD, JNS. Investigation: MJP, TSD, JNS. Methodology: MJP, TSD, JNS. Validation: MJP. Visualization: MJP, TSD, JNS, TRG. Writing – Original Draft: MJP. Writing – Review & Editing: MJP, TSD, JNS, TRG, KLC.
1. Shaw PA. The History of Cervical Screening 1: The Pap. Test. J Soc Obstet Gynaecol Can. 2000 Feb 1;22(2):110-114.
2. American Cancer Society. Cancer Facts & Figures 2020. Available from: https://www.cancer.org/content/dam/cancer-org/research/cancer-facts-and-statistics/annual-cancer-facts-and-figures/2020/cancer-facts-and-figures-2020.pdf. Last updated 2020; cited Sep 14, 2020.
3. U.S. Preventive Services Task Force. Cervical Cancer: Screening. Available from: https://www.uspreventiveservicestaskforce.org/uspstf/recommendation/cervical-cancer-screening#fullrecommendationstart. Last updated Aug 21, 2018; cited Sep 14, 2020.
4. American Cancer Society. Pap/HPV test, women 21 to 65 years. Available from: https://cancerstatisticscenter.cancer.org/#!/data-analysis/Pap_Test. Last updated 2018; cited Sep 14, 2020.
5. Akinlotan M, Bolin JN, Helduser J, Ojinnaka C, Lichorad A, McClellan D. Cervical Cancer Screening Barriers and Risk Factor Knowledge Among Uninsured Women. J Community Health. 2017 Feb 2;42(4):770-778.
6. Musa J, Achenbach CJ, O'Dwyer LC, Evans CT, McHugh M, Hou L, et al. Effect of cervical cancer education and provider recommendation for screening on screening rates: A systematic review and meta-analysis. PLoS One. 2017 Sep 5;12(9):e0183924.
7. AL-Hammadi FA, Al-Tahri F, Al-Ali A, Nair SC, Abdulrahman M. Limited Understanding of Pap Smear Testing among Women, a Barrier to Cervical Cancer Screening in the United Arab Emirates. Asian Pac J Cancer Prev. 2017 Dec;18(12):3379-3387.
8. Ranabhat S, Tiwari M, Dhungana G, Shrestha R. Association of knowledge, attitude and demographic variables with cervical Pap smear practice in Nepal. Asian Pac J Cancer Prev. 2014 Dec;15(20):8905-8910.
9. Head SK, Crosby RA, Moore GR. Pap smear knowledge among young women following the introduction of the HPV vaccine. J Pediatr Adolesc Gynecol. 2009 Aug 1;22(4):251-256.
10. Chen HY, Kessler CL, Mori N, Chauhan SP. Cervical cancer screening in the United States, 1993-2010: characteristics of women who are never screened. J Womens Health (Larchmt). 2012 Nov 7;21(11):1132-1138.
11. Breitkopf CR, Pearson HC, Breitkopf DM. Poor knowledge regarding the Pap test among low-income women undergoing routine screening. Perspect Sex Reprod Health. 2005 Jun;37(2):78-84.
12. Center for Disease Control and Prevention. HPV-Associated Cervical Cancer Rates by Race and Ethnicity. Available from: https://www.cdc.gov/cancer/hpv/statistics/cervical.htm Last updated 2019; cited Jul 4, 2020.
13. Pew Research Center. Health Online 2013. Available from: https://www.pewinternet.org/wp-content/uploads/sites/9/media/Files/Reports/PIP_HealthOnline.pdf. Last updated January 15, 2013; cited Jul 4, 2020.
14. Statista. Global market share of search engines 2010-2020. Available from: https://www.statista.com/statistics/216573/worldwide-market-share-of-search-engines/. Last updated Sep 2, 2020; cited Sep 12, 2020.
15. Ryan, PM, Ryan CA. Mining Google Trends Data for Health Information: The Case of the Irish “CervicalCheck” Screening Programme Revelations. Cureus. 2019 Aug 29;11(8):e5513.
16. American Medical Association Foundation, American Medical Association. Health Literacy: A Manual for Clinicians. Available from: http://lib.ncfh.org/pdfs/6617.pdf. Last updated 2003; cited Ju 4, 2020.
17. Mac OA, Thayre A, Tan S, Dodd RH. Web-Based Health Information Following the Renewal of the Cervical Screening Program in Australia: An Evaluation of the Readability, Understandability and Credibility. J Med Internet Res. 2020 Jun 26;22(6):e16701.
18. Boztas N, Omur D, Ozbılgın S, Altuntas G, Piskin E, Ozkardesler S, et al. Readability of internet-sourced patient education material related to “labour analgesia”. Medicine (Baltimore). 2017 Nov 10;96(45):e8526.
19. Patel SK, Gordon EJ, Wong CA, Grobman WA, Goucher H, Toledo P. Readability, Content, and Quality Assessment of Web-Based Patient Education Materials Addressing Neuraxial Labor Analgesia. Anesth Analg. 2015 Nov;121(5):1295-1300.
20. Basch CH, Fera J, Ethan D, Garcia P, Perin D, Basch CE. Readability of online material related to skin cancer. Public Health. 2018 Oct;163:137-140.
21. Mehta MP, Swindell HW, Westermann RW, Rosneck JT, Lynch TS. Assessing the Readability of Online Information About Hip Arthroscopy. Arthroscopy: The Journal of Arthroscopic & Related Surgery. 2018 Jul 1;34(7):2142-2149.
22. Vargas CR, DePry J, Lee BT, Bordeaux JS. The Readability of Online Patient Information About Mohs Micrographic Surgery. Dermatologic Surgery. 2016 Oct;42(10):1135-1141.
23. Rothrock SG, Rothrock AN, Swetland SB, Pagane M, Isaak SA, Romney J, et al. Quality, Trustworthiness, Readability, and Accuracy of Medical Information Regarding Common Pediatric Emergency Medicine-Related Complaints on the Web. J Emerg Med. 2019 Oct 1;57(4):469-477.
24. U.S. Department of Health and Human Services, National Cancer Institute. Making Health Communication Programs Work. Available from: https://www.cancer.gov/publications/health-communication/pink-book.pdf. Last updated July 2020; cited Jul 4, 2020.
25. U.S. Department of Health and Human Services, Centers for Medicare and Medicaid Services. TOOLKIT for Making Written Material Clear and Effective: SECTION 4 Special topics for writing and design. Part 7: Using readability formulas: A cautionary note. Available from: https://www.cms.gov/Outreach-and-Education/Outreach/WrittenMaterialsToolkit/Downloads/ToolkitPart07.pdf. Last updated Sep 2010; cited Jul 4, 2020.
26. Ley P, Florio T. The use of readability formulas in health care. Psychology, Health & Medicine. 1996;1:7-28.
27. Wang LW, Miller MJ, Schmitt MR, Wen FK. Assessing readability formula differences with written health information materials: application, results, and recommendations. Res Social Adm Pharm. 2013 Sep-Oct;9(5):503-516.
28. Kincaid JP, Fishburne RPJ, Rogers RL, Chissom BS. Derivation Of New Readability Formulas (Automated Readability Index, Fog Count And Flesch Reading Ease Formula) For Navy Enlisted Personnel. Institute for Simulation and Training. 1975 Jan 1;56.
29. Google Trends. Available from: https://trends.google.com/trends/?geo=US. Cited Oct 26, 2020.
30. Advanced Web ranking. Organic CTR History. Available from: https://www.advancedwebranking.com/ctrstudy/. Last updated 2020; cited Sep 12, 2020.
31. Kolesnikov A, Logachev Y, Topinskiy V. Predicting CTR of new ads via click prediction. CIKM '12: Proceedings of the 21st ACM international conference on Information and knowledge management. 2012 Oct:2547-2550.
32. WebFX. Readability Test Tool. Available from: https://www.webfx.com/tools/read-able/. Last updated 2020; cited July 4, 2020.
33. DeVere Wolsey T, Lenski S, Grisham DL. Assessment Literacy: An Educator's Guide to Understanding Assessment, K-12. New York: The Guilford Press. 2020:125.
34. American Cancer Society. Cancer Screening Guidelines By Age. Available from: https://www.cancer.org/healthy/find-cancer-early/cancer-screening-guidelines/screening-recommendations-by-age.html. Last updated 2020; cited Jul 4, 2020.
35. Safaeian M, Solomon D, Castle PE. Cervical cancer prevention--cervical screening: science in evolution. Obstet Gynecol Clin North Am. 2007 Dec;34(4):739-760.
36. Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA. 2001 May 23-30;285(20):2612-2621.
37. Chen X, Acosta S, Barry AE. Evaluating the Accuracy of Google Translate for Diabetes Education Material. JMIR Diabetes. 2016 Jan-Jun;1(1):e3.
38. Search Engine Watch. SEO basics: 22 essentials you need for optimizing your site. Available from: https://www.searchenginewatch.com/2016/01/21/seo-basics-22-essentials-you-need-for-optimizing-your-site/. Last updated Jan 21, 2016; cited Sep 10, 2020.
39. Google Search. How Search algorithms work. Available from: https://www.google.com/search/howsearchworks/algorithms/. Cited Oct 31, 2020.
40. Brandwatch. 57 Fascinating and Incredible YouTube Statistics. Available from: https://www.brandwatch.com/blog/youtube-stats/#:~:text=YouTube%20is%20the%202nd%20most,Netflix%20and%20Facebook%20video%20combined. Last updated Feb 21, 2020; cited Sep 9, 2020.
41. Bosworth A, Ruhter J, Samson LW, Sheingold S, Taplin C, Tarazi W et al. Medicare Beneficiary Use of Telehealth Visits: Early Data from the Start of COVID-19 Pandemic. ASPE Issue Brief. 2020 July 28.
42. Education Week. COVID-19 Fuels Big Enrollment Increases in Virtual Schools. Available from: https://www.edweek.org/ew/articles/2020/09/03/covid-19-fuels-big-enrollment-increases-in-virtual.html. Last updated Sep 3, 2020; cited Sep 9, 2020.
43. Abu Abed M, Himmel W, Vormfelde S, Koschack J. Video-assisted patient education to modify behavior: a systematic review. Patient Educ Couns. 2014 Oct;97(1):16-22.
44. Lühnen J, Steckelberg A, Buhse S. Pictures in health information and their pitfalls: Focus group study and systematic review. Z Evid Fortbild Qual Gesundhwes. 2018 Nov 1;137-138:77-89.
Mark J. Parry, 1 Texas Tech University Health Sciences Center – Paul L. Foster School of Medicine, El Paso, TX, United States
Travis S. Dowdle, 2 Texas Tech University Health Sciences Center – School of Medicine, Lubbock, United States
Jesse N. Steadman, 3 University of Utah – School of Medicine, Salt Lake City, UT, United States
Tiffany R. Guerra, 1 Texas Tech University Health Sciences Center – Paul L. Foster School of Medicine, El Paso, TX, United States
Kim L. Cox, 4 Pocatello Women's Health Clinic – Pocatello, ID, United States
About the Author: Mark J. Parry is currently a 4th year medical student at the Paul L. Foster School of Medicine in El Paso, Texas (USA), which is a 4-year MD program.
Correspondence: Mark J. Parry. Address: 5001 El Paso Dr, El Paso, TX 79905, United States. Email: mark.parry@ttuhsc.edu
Editor: Paul MacDaragh Ryan & Paul Morgan Student Editors: Nikoleta Tellios, Madeleine Jemima Cox, Najdat Bazarbashi, Andrew Thomas & Ciara Egan Copyeditor: Ciara Egan Proofreader: Leah Komer Layout Editor: Sajjad Ali
Cite as: Parry MJ, Dowdle TS, Steadman JN, Guerra TR, Cox KL. Pap Smear Readability on Google: An Analysis of Online Articles Regarding One of the Most Routine Medical Screening Tests. Int J Med Students. 2020 Sep-Dec; 8(3):257-62.
Copyright © 2020 Mark J. Parry, Travis S. Dowdle, Jesse N. Steadman, Tiffany R. Guerra, Kim L. Cox
This work is licensed under a Creative Commons Attribution 4.0 International License.
International Journal of Medical Students, VOLUME 8, NUMBER 3, December 2020