A Comparison of Fake News Detecting and Fact-Checking AI Based Solutions
Scientific objective of this paper is to analyse how advanced are Artificial Intelligence (AI) tools to fight successfully information disorder. More specifically, this is an overview and ranking on existing tools based on AI in this specific area. Research method is comparative analytics. We compare the most developed and publicly available fake-news detecting and fact-checking AI based solutions (intelligent machines). The comparison is based on two key parameters: accuracy and comprehensiveness. Results and conclusions: Analyse show that a third of the examined AI systems are, in terms of comprehensiveness, in the top category, while the majority are in the medium category. As far as accuracy is concerned, very few AI machine developers are interested in providing further details about their products and functionalities for studies such as ours which raises suspicions about their actual performance. Surprisingly, one of the most discussed AI systems among EU leaders seems to actually belong to the least developed. Cognitive value: There is a need for a larger and more detailed study with involvement of AI specialists who would be able, and allowed, to test all available AI machines with their key features and functionalities.
Alaphilippe, A., Gizikis, A., Hanot, C., & Bontcheva, K. (2019). Automated Tackling of Disinformation. Retrieved from http://www.europarl.europa.eu/RegData/etudes/STUD/2019/624278/EPRS_STU(2019)624278_EN.pdf
Babakar, M., & Moy, W. (2016). The State of Automated Factchecking. How to Make Factchecking Dramatically More Effective with Technology We Have Now. Full Fact. Retrieved from https://fullfact.org/media/uploads/full_fact-the_state_of_automated_factchecking_aug_2016.pdf
Carriço, G. (2018). The EU and Artifi cial Intelligence: A Human-Centred Perspective. European View, 2018, 17(1): 29–36. DOI: 0.1177/1781685818764821
Cellan-Jones, R. (2014, December 2). Stephen Hawking Warns Artifi cial Intelligence Could End Mankind, BBC. Retrieved from http://www.bbc.com/news/technology-30290540
Chinnappa, M. (2017). We Are All in This Together, British Journalism Review, 28(3), 50-55. DOI: 10.1177/0956474817730769
Craft, S., Ashley, S. & Maksl, A. (2017). News Media Literacy and Conspiracy Theory Endorsement. Communication and the Public, 2(4): 388–401. DOI: 10.1177/2057047317725539
Faggella, D. (2018, May 14). How to Assess an Artifi cial Intelligence Product or Solution (Even if You’re Not an AI Expert). Retrieved from https://www.techemergence.com/how-to-assess-an-artificialintelligence-product-or-solution-for-non-experts/
Funke, D. (2018, February 23). Report: There are 149 Fact-Checking Projects in 53 Countries. That’s a New High. Retrieved from https://www.poynter.org/news/report-there-are-149-fact-checking-projects-53-countries-thats-new-high
Ghafourifar, A. (2017). How AI is Winning the War Against Fake News. Retrieved from https://venturebeat.com/2017/06/11/how-ai-is-winning-the-war-against-fake-news/
Graves, L. (2018, February). Understanding the Promise and Limits of Automated Fact-Checking. Factsheet, Reuters Institute for the Study of Journalism. Retrieved from https://reutersinstitute.politics.ox.ac.uk/sites/default/fi les/2018-02/graves_factsheet_180226%20FINAL.pdf
Hassan, N., Li, Ch., & Tremayne, M. (2015). Detecting Check-worthy Factual Claims in Presidential Debates. Proceedings of the 24th ACM International Conference on Information and Knowledge Management: 1835–1838.
Hassan, N., Zhang, G., Arslan, F., Caraballo, J., Jimenez, D., Gawsane, S.,… Shohedul, H. (2017). ClaimBuster: The First-ever End-to-end Fact-checking System. Proceedings of the VLDB Endowment, 10(12): 1945–1948.
Marr, B. (2017). The Biggest Challenges Facing Artificial Intelligence (AI) In Business And Society. Retrieved from https://www.forbes.com/sites/bernardmarr/2017/07/13/the-biggest-challenges-facingartificial-intelligence-ai-in-business-and-society/3/#37119b357b6
Özgöbek, Ö., & Gullain, J.A. (2017). Towards an Understanding of Fake News. Norwegian Big Data Symposium NOBIDS:35-42.
Orlowski, A. (2017, January 2). Artifi cial Intelligence’ Was 2016’s Fake News. Retrieved from http://www.theregister.co.uk/2017/01/02/ai_was_the_fake_news_of_2016/?page=1
Pavleska, T., Školkay, A., Zankova, B., Ribeiro, N., & Bechmann, A. (2018, February). Performance Analysis of Fact-Checking Organizations and Initiatives in Europe: A Critical Overview of Online Platforms Fighting Fake News. Retrieved from http://compact-media.eu/fake-news/
Popat, K., Mukherjee, S., Strötgen, J., & Weikum, G. (2016). Credibility Assessment of Textual Claims on the Web. Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, 2173–2178. ACM. DOI: 10.1145/2983323.2983661
Prateek, J. (2017). Artificial Intelligence with Python. Packt Publishing.
Rony, M.M.U., Hassan, N., & Yousuf, M. (2018). BaitBuster: A Clickbait Identifi cation Framework. Proceedings of the 32nd AAAI Conference on Artifi cial Intelligence. (AAAI–18).
Rony, M.M.U., Hassan, N., & Yousuf, M. (2017). BaitBuster: Destined to Save You Some Clicks. Proceedings of the 2017 Computation+Journalism Symposium.
Rony, M.M.U., Hassan, N., & Yousuf, M. (2017). Diving Deep into Clickbaits: Who Use Them to What Extents in Which Topics with What Effects?. Proceedings of 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.
Russell, S., Dewey, D., & Tegmark, M. (2015). Research Priorities for Robust and Beneficial Artificial Intelligence. Association for the Advancement of Artificial Intelligence: 105–114.
Shotter, J. (1997, May). Artifi cial Intelligence and the Dialogical, American Behavioral Scientists, 40(6): 813–827.
Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017, September 3). Fake News Detection on Social Media: A Data Mining Perspective. Retrieved from https://arxiv.org/pdf/1708.01967.pdf
Su, D., Zhang, H., Chen, H., Yi, J., Chen, Pin-Yu., & Gao, Y. (2018). Is Robustness the Cost of Accuracy? A Comprehensive Study on the Robustness of 18 Deep Image Classifi cation Models. Retrieved from http://openaccess.thecvf.com/content_ECCV_2018/html/Dong_Su_Is_Robustness_the_ECCV_2018_paper.html
Vlachos, A., & Riedel, S. (2014). Fact Checking: Task Defi nition and Dataset Construction. Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science:18–22.
Wang, W.Y. (2017, May 1). Liar, Liar Pants on Fire: A New Benchmark Dataset for Fake News Detection. Retrieved from https://arxiv.org/pdf/1705.00648.pdf
Wood, C. (2016, August 1). What Is Artifi cial Intelligence? Government Technology. Retrieved from http://www.govtech.com/computing/What-Is-Artificial-Intelligence.html
Zhao, Z., Resnick, P., & Mei, Q. (2015). Enquiring Minds: Early Detection of Rumors in Social Media from Enquiry Posts. Proceedings of the 24th International Conference on World Wide Web: 1395–1405.
Copyright (c) 2019 Andrej Školkay, Juraj Filin
Utwór dostępny jest na licencji Creative Commons Uznanie autorstwa – Użycie niekomercyjne 4.0 Międzynarodowe.
Publikacje na łamach „Studiów Medioznawczych” ukazują się na zasadach odpowiadających licencji Creative Commons Uznanie autorstwa 4.0 (znanej również jako CC-BY).