Remotely controlled forwarder crane with VR can be operated despite delays

Remote control of forestry machines can improve the work environment for the operator. A well-functioning teleoperation system is also a step towards increased autonomy. Full autonomy, that could handle all possible situations the machine might encounter, requires extensive development. Teleoperation allows a gradual transition to autonomous functions. However, system delay is a critical factor in tasks that involve high speeds, such as crane operation. This study has therefore examined how remote control affects operator performance in a crane operation task. Operators performed the task using both teleoperation via a VR headset and conventional operation from the machine cabin. In the teleoperation setup, the task was performed both with the system's minimal inherent delay and with two levels of additional, artificial delay. The crane operation was a standardized task, where two logs were moved between two designated points on the ground and two positions in the machine's load area. The task is complex enough to include many critical handling operations while being controlled enough to allow repeated measurements for statistical analysis.

The study was performed using the full-scale forwarder prototype Xt28, where remote control is implemented with ROS (Robot Operating System), and the video transmission system for the VR headset is provided by Voysys. To study the effects of greater delays, a function was implemented in ROS to introduce artificial delay. The teleoperation system’s minimal inherent delay is 100 ms when operating over the network. The impact of delay on the teleoperation system was also tested at 150 ms and 200 ms and compared with crane operation from the machine cabin. The machine’s internal delay in the hydraulics and control units also contributes to the overall delay.

The operators achieved surprisingly comparable performance regardless of delay or whether they were operating remotely or from the cabin. However, at the higher delays of 150 ms and 200 ms, operators reported that the crane sometimes felt as if it got "stuck" during rapid adjustments. At delays between 200 ms and 600 ms, the experience shifted to where the crane became impossible to control normally. For comparison, latency in a 4G connection is usually under 50 ms.

The VR experience was generally positive, with the lack of vibrations and noise being especially appreciated. However, the stereoscopic vision was not entirely perfect, and in some situations, the crane's position occasionally appeared to shift suddenly by 1–2 meters.

Conclusions

• The results suggest that a forwarder crane operated remotely with VR can achieve performance comparable to that from the machine cabin.
• Performance remains unaffected even with delays that exceed those expected on a 4G connection, up to levels involving an additional 100 ms delay in the connection.
• Although the chosen camera positions and VR technology provide stereoscopic vision that facilitates crane operation, depth perception is not as accurate as with traditional in-cabin control.

Further research is needed to address all aspects of forwarding work under remote control, as well as to assess machine safety and working environment conditions.