How to Extract Numerical Data from a Web Page for Dashboarding and Alerting
Over the years working as a software engineer and now a product manager, I’ve encountered multiple situations where I needed to extract numerical data from a page on a periodic basis and create visualizations, typically line charts to help me see trends over time. For example, I wanted to extract product prices and monitor them over time. Or, I wanted to query a search engine periodically and extract the number of matches or the position of a specific page for SEO purposes.