Background Tasks and Queuing in Dash Enterprise
Meet the Speaker
Austin Kiesewetter
Austin Kiesewetter is a Plotly community member and software engineer at Martin Engineering.
This video covers how to handle background tasks and queuing in Dash Enterprise, a critical technique for improving performance and user experience, especially when working with large datasets or slow API calls. Two approaches are discussed: the Dash Enterprise Scheduler and a more advanced setup using Celery with Redis and PostgreSQL.
The Dash Enterprise Scheduler allows you to run background jobs on a fixed schedule. It's straightforward to configure by updating your Procfile and using the @schedule decorator in your app. This is useful for refreshing data on a timed basis and caching it locally for fast retrieval. It’s low complexity, fast, and well suited for smaller or regularly updated datasets, but not ideal for long-running tasks or tasks that need durability if the app restarts.
For more complex use cases, the video explores setting up Celery with Redis and PostgreSQL. This approach introduces task queues and a dedicated Celery worker, allowing your Dash app to offload long-running processes and remain responsive. Redis acts as a message broker and temporary storage, while PostgreSQL is used to persist results. This architecture supports real-time updates, persistent state, and task resilience, but comes with added setup steps and resource overhead.
The video also introduces the Dash Snapshot Engine, which allows PDF report generation and email delivery based on app views.
Watch the video to follow along and learn how to set up each option in your own Dash Enterprise environment.
