Web Scraping Techniques and Applications

Abstract

Data is one of the most valuable resources in today's business world, and economists often liken the impact of oil and data. According to a May 2018 Forbes article, every day 2.5 quintillion bytes of data is created. Businesses have been placing a greater importance on data analytics in search of quicker decision making, cost-reduction, and new products or services. This paper will talk about my experience creating a web scraping application for an insurance management company. I will explain the basics of web scraping, its use cases, and popular techniques. During the summers of 2018 and 2019, I interned at Chelsea Rhone LLC in Ann Arbor, Michigan. Chelsea Rhone is an innovative insurance management company that works with over 1,000 clients. Primarily, I was a software developer focusing on front-end web development, however my day to day responsibilities often varied. This paper will first talk about web scraping as a topic and then more specifically the project that I developed for Chelsea Rhone. iv