The future of industrial manufacturing and production requires highly flexible work cells that are able to combine automation with tasks traditionally done manually. But such cooperation between humans and robots in close proximity requires the safeguarding of the human at all times. As traditional safety systems like fences, laser scanners, or pressure sensitive mats are too cumbersome or not applicable for these scenarios, we have noticed a lot of development efforts by industry and academia over the past years towards sensor systems that allow a safe cooperation between humans and robots in the same work space. Many of these new systems depend on a three dimensional representation of the work space to detect intrusions of virtually established, three dimensional safety spaces. The researchers of the Fraunhofer IFF have recently developed and patented a novel sensor system that is based on projector and camera technologies and offers many advantages over three-dimensional safety systems.
This projection-based sensor system establishes safety spaces of arbitrary shapes such as rectangles, circles, lines, filled areas, or any further combinations by projecting light from a projector directly onto the environment. Violations of these safety spaces caused by disruptions to the emitted light are robustly detected by surrounding cameras. The operational principle is based on a comparison of a two-dimensional representation of the current state of the safety space and a two-dimensional, computer-generated expected (fault-free) state of the safety space. As there is no need for a complex computation of three-dimensional data of the environment, the implemented algorithms for image processing and collision detection lead to minimal reaction times of the system. Furthermore, the robustness and availability of the system is enhanced through synchronisation of projectors and cameras. Here, the cameras are adapted to the frequency of the light emitted from the projector, reducing the influence of environmental light conditions on the collision detection process.
The system was initially capable of establishing several user-defined safety spaces statically in the workspace, and it was possible to switch between them manually or automatically. The next development step included the adaption of the sensor system to a human-robot application and the dynamic generation of the safety space generation based on the robot’s movement. This research was done within the ECHORD [1] experiment “Experimental Evaluation of Advanced Sensor-Based Supervision and Work Cell Integration Strategies” (EXECELL). Here, a prototypical collaborative workplace with a KUKA LWR 4+ was built up to test the projection-based sensor system as a safety-rated speed and separation monitoring system. One projector and four cameras were used to monitor the collaboration area from multiple perspectives and a connection to the robot’s controller was established. On this basis, the system incorporated the robot’s joint angles and velocities in the safety space generation process. The resulting safety space minimally enclosed the robot at all times and left a maximum area of the workspace to the user. Figure 1 shows the collaborative workplace with the KUKA LWR that is enclosed by the dynamically generated safety space (white line).
Figure 1: Projection-based system monitoring robot’s motion with dynamic safety space (white line)
In the project “FourByThree” we will advance these predevelopments and deploy this technology for applications in real industrial environments. For this, the projection-based monitoring system has to be adapted according availability, safety, and modularity to meet the requirements of various industrial conditions. Beside technical improvements that include the enhancement of response time, detection capabilities, and robustness, the projection system should be composed of single modular projection unit that provides higher flexibility and customizability. Each unit will comprise one projector and at least two cameras that can be adjusted individually. By configuring several units to work together, it will even be possible to operate in difficult environmental conditions with low ceilings or large monitoring areas. The design and development of these projector-camera units is one of our main tasks in the project. The second task concerns the evaluation of the sensor system according a future safety certification. Thus, in cooperation with our “FourByThree” project partners we will analyse the technology using the relevant safety standards.
Beyond the topic of safety, this technology is also well suited for visualization to give the operator information. This means that this system can be also used to visualize process-related, robot-related, or even general information to the worker. Here it may be useful to depict some schemes for worker assistance, symbols for representing robot’s state, or text messages for describing the current task. The system can also be used to support the user in configuration and teaching during initial operation. In “FourByThree” we will enhance and apply this technology to the individual applications supporting the user at certain tasks. In Figure 2 are some examples of assisting the user by visualizing different types of information directly on the table top.
 |
 |
Figures 2 and 3: Additional visualized information for worker assistance
Another exciting feature of the projection system concerns the possibilities for human – machine interaction, whereby the operator can offer input and information back to the robotic system. This means that the projection system is capable of providing buttons or simple menus which can be used to control the robot, task, or process. The shape of these interaction areas and the reaction upon triggering can be configured individually. So for instance the interaction system allows the confirmation of a finished task, the execution of a certain process or the choice of a special task configuration. Figure 3 shows a two-hand interaction area that has to be triggered to continue the robot’s motion after a safety violation.
Figure 4: Two-hand confirmation of a safety violation for system restart
These developments are all made with an eye to future industrial deployment, and we look forward to testing the possibilities and showcasing the potential of novel, projection-based workspace monitoring system within the FourByThree project.
[1] “European Clearing House for Open Robotics Development”, Grant Agreement Number 231143