Text-mining and Analysis of the Doctors’ Meta-data and Text-reviews using Topic-modeling (LDA) Technique
Date
2019-10-30T15:29:06Z
Authors
khan, asad
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
The developments in the internet and web technologies along with smart devices have empowered consumers to rate, comment, review, recommend products and services for others using a plethora of platforms, such as RateMDs.com. Therefore, feedback is critical to improve the overall quality of a process, product or service. Hence, the healthcare industry is no exception. This thesis aims to mine and analyze physicians’ online reviews using web-scrapping and topic-modeling (LDA) technique. RateMDs.com was chosen as a case study for the period from September 2013 to January 2019. The thesis employed web scrapping, to collect physicians’ meta-data, and LDA technique, a generative probabilistic model of text-corpus to the text-corpus, for text-mining among Canadian provinces. The results revealed that physicians, in some of the specialities, such as plastic surgery, had a higher probability of being rated than others in specialities such as Radiation, Oncology and Osteopathy. The research also revealed that East coast provinces had a relatively higher rating than those in the West of Canada. Finally, this thesis validates the use of Python (BeautifulSoup, spaCy, Gensim, NLTK, re) for text-mining with LDA.
Description
Keywords
Artificial Intelligence, Convolutional Neural Network, Dalhousie Natural Language Processing, Hierarchical Anticipatory Learning, Human Computer Interaction, Python, Web Scrapping, Latent Dirichlet Method (LDA)