Komatsu Develops Advanced Analytics and Reporting Applications With Dash Enterprise and Snowflake

Introduction

Komatsu is a global leader in designing and manufacturing equipment for the mining, construction, forestry, and other industries. Its North America Surface Mining OE business unit focuses on electric mining shovels, electric wheel loaders, blasthole drills, and electric drive trucks, all manufactured in North America.

Operating in vast open-pit surface mines, these machines are equipped with control systems that produce data from hundreds of densely sampled sensor signals, between one and ten samples per second. The Surface Mining Analytics team oversees the analytics and digital products built on this machine data.

The team at Komatsu, comprising primarily data scientists and BI engineers, employs Python modeling to transform raw machine data into meaningful insights and metrics. Their work extends to creating comprehensive reports and visualizations combining raw sensor data with analytically derived metrics.

These reports cater to a variety of end user personas, such as product support and monitoring teams at Komatsu distributors, providing actionable insights to improve machine performance and health. The data also plays a pivotal role in engineering, enabling safety investigations and the evaluation of new automation technology for the development of new and next-generation machines.

Challenge

Komatsu's Analytics team faced challenges in developing complex data applications for their customers, when the requirements surpassed the capabilities of lower-code tools. The team is built around core competencies in Python and SQL, not traditional front-end frameworks. There were a number of use cases, each with unique challenges.

One of the more unusual datasets for visualization comes from vibration condition monitoring kits installed on electric mining shovels. Each data collection produces a waveform-and-spectrum pair of arrays that did not fit cleanly into any of Komatsu’s no-code visualization tools, like Grafana and PowerBI. The team sought a solution for advanced, customized data visualization, combined with a variety of highly customized analysis cursors and widgets. This was beyond the capabilities of existing tools.

Additionally, Komatsu required a more flexible tool for public-facing marketing use cases to demonstrate global fleet connectivity. Customization limitations in existing tools became apparent, hindering even basic elements like a corporate-standard font or custom mapping with unique Komatsu-developed icons.

In response, the team turned to Dash Enterprise, shortening the time required for application development while addressing data and design challenges.

Solution

Komatsu tackled their challenges across four major use cases, each addressing distinct value propositions.

The Customer Experience Center application aimed to illustrate a connected global fleet of equipment using marketing digital assets, including custom icons and fonts. The team used Dash's broader ecosystem of libraries, particularly the Leaflet Dash integration, to make almost anything possible in a Python framework. Direct Snowflake queries ensured live data updates for this marketing dashboard used to display Komatsu’s footprint across the globe.

This dashboard is on public display at Komatsu’s South Harbor campus lobby in Milwaukee, WI.

The second application, a Vibration Data Viewer, handled custom vibration waveform and spectrum data beyond the capabilities of traditional BI tools. Initially sourced from Cloudera's Impala data source, it later migrated to Snowflake while preserving front-end elements. This app, crucial for advanced diagnostics of rotating components on large mining shovels, is a core monitoring tool among Komatsu distributors’ condition monitoring experts in the US, Chile, and Australia.

For Electric Drive Haul Truck Payload Reporting, Komatsu addressed unique use cases for multiple personas in the organization — warranty adjudication, product support, engineering analysis, and generalized productivity. Dash Enterprise and the power of Python was well-suited for the team's needs to build interactive and customized elements, for functionalities such as warranty policy selection. The project involved integrating data sources beyond Snowflake, including CRM (Salesforce) and internal REST APIs. A two-month UI/UX design engagement emphasized the need for the capability in a framework like Dash for the Analytics team to preserve original design elements closely.

Lastly, the Advanced Factory Analytics Insights application showcased the full versatility of Plotly and other Python data visualization libraries, providing insights on large mining equipment for premium paying customers as well as internal stakeholders. Visualizations include scatter plots in polar coordinates for digging and loading/dumping locations, as well as a variety of other visualization types beyond the scope of no-code BI tools. Moving forward, the team plans to wrap up many of these visualizations into a more robust web app.

Results

• The applications developed by Komatsu's Surface Mining Analytics team generate insights, facilitate fault detection, and contextualize machine behavior from events and sensor data from several hundred signals generated by Komatsu mining machines, at a typical rate of 10 samples per second.
• The team was able to develop three highly customized, production-grade data applications for the public-facing Customer Experience Center in a span of just five weeks with Dash Enterprise. These screens display infographics on Komatsu’s active mining machine fleet at the company’s South Harbor campus in Milwaukee.
• The Vibration Data Viewer application is used on a daily basis to offer premium-level insights to Komatsu shovel customers, for a specialized technology add-on kit.
• Advanced insights are made possible in the Advanced Factory Analytics reporting by leveraging the full Python and Plotly ecosystem, combined with Dash’s web elements.