Fetch Integrates Plotly Dash With Snowflake to Power a Comprehensive Self-Service Analytics Platform

Introduction

Fetch is redefining the landscape of loyalty and rewards programs through its innovative approach to integrating technology and shopping. At the heart of Fetch's success is a platform that caters to millions of consumers, bridging the gap between them and their favorite brands.

Challenge

The interactions of Fetch users and brands on the Fetch application creates terabytes of data which the company’s partners and stakeholders need to operate and execute incentive programs every day. 

The Fetch Analytics Hub, a simple UI development framework using Dash and Snowflake, provides a seamless, intuitive data delivery tool for stakeholders that parses terabytes of data into hundreds of different slices in minutes. Additionally, the Fetch team developed a custom solution that allows any developer to build a data application quickly and easily, in a declarative way.

Solution

Fetch’s Client Analytics Hub provides a suite of tools that streamline the data analysis process. The platform combines Dash’s intuitive user interface development, Snowflake's powerful data warehouse, and an in-house custom Python package to facilitate strategic data exploration without the complexity of traditional SQL query construction or the need for exponentially growing pre-defined aggregations.

The platform offers analysts and engineers a declarative method of creating BI tools, defining SQL with complex logic, Dash Core Components (e.g., drop-downs, radial dials, and date slides), Plotly visuals, and other typical BI functions. It utilizes Dash’s routing to create a home for all reports, making navigation and discovery easy for end users.

There are dozens of independent applications that, in aggregate, get queried thousands of times per day by end users. Utilizing the features of Dash, development is simplified using the following pattern:

1. Simplified Creation of Filter Objects: A custom class transforms user inputs into dynamic Dash filters that generate the required predicates tailoring data retrieval to individual needs while remaining in scope of each application’s purpose. The developer simply chooses a Snowflake table and the fields from that table in a Python list and the filters are dynamically generated by button push, in order of which they are placed within the application’s core code. This allows end users to focus on only the fields they want while allowing access to any field they need. When instantiated, these class variables are then passed to the next class. On the front end, the team utilizes refresh buttons to avoid pre-downloading inputs for the drop downs, buttons etc., which re-query Snowflake based upon the current selections, allowing for fast but dynamic retrieval of inputs for selection in the UI.
2. Seamlessly Integration of Inputs into SQL Queries: An additional custom class takes the inputs from the prior class based on the user’s manipulation of the UI and integrates them into a combination of strings and logic to easily create the user’s desired outcome in SQL. This query gets executed against Snowflake with stored credentials and returns the desired output. The team utilizes Snowflake caching to re-query the results for multiple cuts without rescanning large tables by storing the query id in a Dash object.
3. Easy Visual Creation and Integration: A final custom class uses the outputs to declaratively re-query the cache with additional stored queries allowing users to aggregate the data quickly based on the analytical use case. The developer declaratively selects a visual type (line graph, bar chart etc.), and whether they want a download button or accompanying table. The tables, visuals, and assets are all pre-formatted to match Fetch colors and standards. They will automatically be ordered and fitted to the app based on the order declared and executed.

Finally, a simple connector class manages the app's pushdown to Snowflake, tagging, and any other necessary features that are available in Snowflake today. Below is an example of an app created in 500 lines or less:

A user can dynamically add drop downs as needed. After hitting refresh, Dash callbacks will retrieve the fields from Snowflake as needed to avoid pre-loading drop down parameters.

This specific report allows users a second set of options to exclude data based on the same criteria, in essence, creating anti-joins without the need for understanding SQL. Once the user is ready with their inputs, they can run the report which outputs the desired visuals as well as the query ID and text for reference and discussion with analysts.

Results

The applications developed by Fetch using Plotly Dash have enabled cross-functional teams within the organization to track offer performance, diagnose application errors, provide meaningful audiences to partners, quickly develop a sales pitch, deliver shopper insights, and summarize KPIs in a visually appealing way. The team has achieved the following results:

• Fetch developers can make a highly complex, self-service data application in less than 500 lines of code and have it deployed in 24 hours.
• The Fetch team has developed over 25 different applications tied to their Analytics Hub home page.
• The apps within the Analytics Hub platform are used for forecasting, and integrate some machine learning forecasts models to predict revenue in the future.

The team consists of Alex Kalish - Tech Lead, Tom Vanichachiva - Development Lead, Rebecca Reicherter - Product Lead, and major contributors Kajal Amarnani, Christian Carmellini, and Saketh Kumar Kappala.