Data Analytics at Scale
Business Intelligence Engineers (BIE) must be experts with working on large data sets while also understanding the needs of the business. Combining both of these skills effectively allows BIEs to build products that deliver the most value. Solutions can include interactive dashboards, models, and ad hoc analyses.
A top 10 Fortune 500 company was struggling to track Travel and Entertainment (T&E) costs across the entire organization, encompassing more than 1.6 million employees. Multiple tech and business teams had tried to build solutions but end users did not have the insights they needed to see the big picture and then dive into the details. Many teams had users pulling data manually and compiling their own reports that often did not reconcile across groups.
Tony constructed a cloud datalake that ingested financial and spending data from multiple sources. He cleaned and organized the data in a unique way and then constructed an interactive Dashboard in Amazon Quicksight to ingest and display the information. This gave users an end-to-end view of T&E expenses form the high-level financial report all the way down to individual employees. The tool accumulated over 500 users in the first two months after launch. The reception was so positive that a fully staffed Tech team took over the datalake and dashboard to roll it out across the entire company serving thousands of users.
Tony joined an FP&A team that was using IBM Cognos. The system was very slow and primarily accessed through Excel. Teams would frequently need to reconcile different scenarios, comparing YoY, QoQ, or Actuals to Plan. This was time intensive because users would have to do multiple data pulls in order to identify where breaks occurred or uncover what was driving differences.
To solve this problem, Tony constructed a Python website application using the Dash framework that ran on AWS Fargate. The application gave users the ability to instantly pull any combination of scenario comparisons and then click through cost centers and accounts to immediately pinpoint the variance drivers. The application became the default way many users accessed firm financials.
An Exchange Traded Find (ETF) provider with over $50B in AUM did not have any internal tools or resources to view historical data on their products or competitor products. When internal stakeholders would ask for reports on standard metrics such as growth in AUM, fund performance, etc., users would have to manually pull data and compile the information into a PDF or PowerPoint document.
To streamline this process, Tony built a complex data architecture, and constructed a comprehensive interactive application using R Shiny that allowed users to view and analyze the entire $7T ETF market. Users have the ability to zoom all the way out and view the entire ETF market or drill down into different sectors, geographical regions, or individual ETFs. Users had millions of different views they could create with a few clicks. This greatly enhanced visibility across the entire market including detailed information on their own funds.