Occupational safety inspection from the robot: helmet check and two-state equipment inspection
Feasibility of a specific job shown on real hardware; not taken into continuous operation.
- Industry
- Energy and industrial facility operations
- Customer
- Electricity distribution site, Turkey
- Period
- 2026-09
- Operating mode
- Human in the loop
Environment
Where the robot works
Indoor corridors and the yard; a 720p stream from the gimbal camera, variable lighting, 3-15 m to the target.
Problem
Operational problem
Safety inspection depended on the person: helmet checks and the state of lights, doors and fire doors were logged by hand during the shift. Off-the-shelf detection models do not contain these classes.
Task
What the robot does
- 1Detects people in the camera stream and assigns a track id.
- 2Runs the helmet model on the head region of the track and stabilises the result with hysteresis.
- 3Produces three values: helmet on, no helmet, undetermined. Undetermined is not counted as a violation.
- 4A rule layer turns a state that passes the duration and repetition gate into an event; every event is stored with its image.
- 5For equipment inspection the frame is compared against two taught reference images (open/closed).
Hardware
Hardware in use
SIYI A8
Gimbal camera with an RTSP H.265 stream, aimed at the target at the inspection point.
Robot üzeri işlem birimi
TensorRT inference runs onboard; events go to the dashboard.
Components used in the installation.
Software
Software and AI
YOLO11 + TensorRT (FP16)
Person detection, running single-threaded at a capped frame rate.
Baret modeli
A separate TensorRT model; the threshold is set fail-safe so unstable frames do not produce a violation.
Olay kuralı
Duration and repetition gate; a single frame alone does not create an event.
Denetim noktası karşılaştırması
ROI brightness and histogram signature; closeness to the two taught states is measured.
Workflow
Mission flow
- 01SensorsGimbal camera (RTSP)Radiometric thermal3D LiDARDepth camera
- 02Onboard inferencePerson detectionTrackingHelmet modelInspection-point comparison
- 03Rule layerDuration gateRepetition gateThree states: yes / no / undetermined
- 04OperatorFinding logImage recordApprove or reject
The decision stays with the operator: the system raises events, it does not close them. An unstable frame counts as undetermined, not as a violation.
- The operator teaches the inspection point: boxes the target and records both states.
- On patrol the robot returns to the same point and the gimbal turns to the same angle.
- The frame is compared against the two references; without a clear match the result stays 'undetermined'.
- If staff are present, helmet status is reported in the same frame.
- Events are written to the audit log with image and timestamp.
Human
The human role
Human in the loop
The operator teaches the inspection points and thresholds. The system does not decide, it produces findings: every event waits for operator approval or rejection.
Safety
Safety approach
- Fail-safe threshold: an unstable frame yields 'undetermined', not a violation.
- Three-state output: 'could not be checked' is never treated as 'compliant'.
- Personal data: identity verification is switched off; event records with images stay inside the facility.
Limits
What the system cannot do
- The threshold is scene-dependent: not a fixed number, it follows the measured noise of that scene. An operator adding a new point must teach both states.
- The helmet model could not be tested at close range; the false-negative rate in that condition is unknown.
- Identity verification does not work: on real staff photos the median similarity for the same person measured 0.31, so the feature was disabled.
- Results degrade at night and against backlight; these conditions need their own taught references.
- Scope is limited to three checks (helmet, light, door). Other safety rules need new points and new references.
Results
Measured results
- The three inspection types ran live on site and findings were stored with images.
- It was shown by measurement that an off-the-shelf model cannot do this job: the classes are absent from the COCO list, so the method moved to reference comparison.
- Identity verification was eliminated by measurement and switched off before it could create a false sense of security.
Detection accuracy is not reported, since no balanced, labelled test set was collected; the method and the limits are documented instead.
Platform
The robot used
Values are manufacturer data.
Site survey for a similar deployment
This deployment was configured around a single site's conditions. Floor, route, network coverage, charging point and safety rules differ from site to site; the scope is set by the survey report.