Sumitomo Heavy & NEC to automate site near-miss reports
Sumitomo Heavy Industries and NEC will jointly develop a system to identify near-miss incidents at construction sites and generate reports automatically, with work set to begin in April 2026.
The project focuses on video footage and sensor data collected from hydraulic excavators. Its goal is to detect potentially hazardous scenes on site and turn those findings into reports for safety management.
Construction work is shaped by weather, ground conditions and changing site layouts, all of which can create hazards. As a result, demand has grown for digital tools that help companies capture, review and document risks more systematically.
The companies say the construction sector still lacks a system that covers the full process in one workflow, from collecting machine video and extracting risky scenes to analysing near-miss events and producing reports. This effort is intended to fill that gap by combining machinery data analysis, video recognition and generative AI.
How it works
The system will use an AI model trained on hydraulic excavator data gathered through SHICuTe, a data platform used across the Sumitomo Heavy Industries group. It will first identify and extract what the companies describe as risk scenes from recorded footage.
Those scenes will then be analysed alongside excavator operating data using NEC technology that combines video recognition and generative AI. The resulting information will be stored as multimodal data with temporal and spatial details, allowing the system to compare site activity with records of hazardous behaviour, prohibited actions, equipment failures and operations requiring special attention.
The system will also take company-specific data into account. Based on those matches, it is expected to determine which incidents should be reported and generate a summary of the circumstances surrounding each near-miss.
Earlier test
Before moving to joint development, the companies carried out a technical proof of concept in September 2025. The test examined whether a system using footage from cameras mounted on hydraulic excavators could automatically extract near-miss incidents and generate reports.
According to the results, the system was able to report near-miss cases, including potential accident scenarios and the circumstances linked to them, based on risk scenes identified from video. The next stage is intended to expand the range of incidents it can detect and improve report generation for customer safety management needs.
Sumitomo Heavy Industries is contributing construction machinery expertise and data analysis experience. NEC is providing video recognition, generative AI and advanced technology consulting services.
Rollout plan
During fiscal 2026, the companies plan to carry out technical development and validation using on-site data and safety management expertise from Sumitomo Heavy Industries together with NEC's AI tools. They are targeting practical implementation in fiscal 2027.
Beyond worker-and-machine contact risks, they intend to extend the system to detect unsafe conditions that workers may not easily notice. They also plan to incorporate site-specific operating rules as they broaden the system's scope.
The development reflects a wider push in heavy industry and construction to use machine data, computer vision and large language models to reduce manual review and improve incident reporting. Here, the focus is on hydraulic excavators, which are widely used on construction sites and generate both visual and operational data that can be analysed together.
NEC has previously disclosed work using generative AI and video recognition AI to create explanatory text from video in other reporting contexts. Sumitomo Heavy Industries' SHICuTe platform is used to collect and store operational data from the group's connected products.
By combining those existing assets, the companies aim to automate a task that has often relied on manual review of footage and work logs. They are seeking to build a system that can flag risky scenes, classify incidents and produce near-miss documentation from machinery data and video captured during routine site operations.