Principal Investigators : Prof. Dr.- Ing. Rüdiger Daub (TUM); Dr. Petr Dokladal (MINES Paris PSL); Dr. Flavien LUCAS (IMT); Other contact person: Benedikt Schmucker (TUM); Johannes Bauer (TUM); Sebastian Kurscheid (TUM); Dr. Guillaume LOZENGUEZ (IMT); Dr. Beatriz Marcotegui (MINES Paris PSL); Dr. ABDOUS Mohammed-Amine (IMT)

New publication from the project in 2023:

Bauer, J.; Abdous, M.-A.; Kurscheid, S.; Lucas, F.; Lozengues, G.; Daub. R.: Towards Enabling Human-Robot Collaboration in Industry: Identification of Current Implementation Barriers. In: Herberger, D.; Hübner, M. (Eds.): Proceedings of the Conference on Production Systems and Logistics: CPSL 2023 – 2. Hannover : publish-Ing., 2023, S. 67-447. Link for download: https://www.repo.uni-hannover.de/handle/123456789/15374

Short summary and central question 

Human-Robot collaboration aims at combining the strengths of robots, like high speed and good repeatability, with the flexibility and adaptability of human workers. These systems can support humans when performing physically challenging tasks and at the same time allow automation in scenarios where this was previously considered unfeasible without the capabilities of collaborative robots (short: cobots). As a result, complex manufacturing processes can be carried out in high-wage countries such as France and Germany, even as the shortage of skilled labor continues to rise. The use of collaborative robots could consequently contribute to social and economic sustainability within these European countries.

However, despite these benefits and despite being discussed in the scientific literature for many years, the systems are hardly used in industry today, especially when the robot is mobile in the industrial environment. Therefore, we propose to investigate the reasons for this in order to identify future research and development needs. The approach of this French-German project can be summarized as follows: After elaborating the state-of-theart, an extensive industry survey will be conducted to determine obstacles for and during the implementation of human-robot collaboration applications in high- and low-volume 4 production and service environments. Based on the generated insights, a future research agenda for funding opportunities such as Horizon Europe can be defined.

Overview of the state-of-the-art 
Introduction 

Recently, the European Commission started a complementary approach to Industry 4.0, called Industry 5.0. It is a transformative vision of the European industry, moving toward more sustainable, human-centric, and resilient systems [1]. 

In this new paradigm, human-robot collaboration refers to environments, where humans and robots work in close proximity, sharing their workspaces, resources, or even their tasks [2]. For the scope of this project we defined the term in a rather broad sense, including different systems that involve direct interaction between humans and robots. In the following, we will briefly address the state of the art regarding robot applications, the use of industrial robots for robot applications, mobile collaborative robotics, mobile robot manipulators as well as the role of machine vision in this context. 

Human-robot collaboration with robots 

Early works on collaborative robot applications date back to the year 1996 and 2001 [3, 4] and aimed at increasing ergonomics for human workers. Today, the term robot is mostly associated with lightweight robots, specifically designed for collaborative applications, like the UR5 or KUKA iiwa [5]. Their low weight in combination with limited speed results in lower forces during motion and, therefore, decreases the risk of injuries for workers, in case of contact. Furthermore, they are equipped with various sensors, such as gear torque sensors in the robots’ joint angles. Robot applications can be distinguished according to their level of collaboration. The Fraunhofer IAO defines with increasing levels of collaboration: Cells, Coexistence, Synchronized, Cooperation, and Collaboration [6]. Safety aspects are of utmost importance for the acceptance of this technology [7] and for the commissioning of such systems, various normative requirements must be met [8, 9]. In practice, the level of actual interaction between robots and humans is usually still quite limited [7]. 

Collaborative applications with industrial robots 

One main drawback of robots, compared with industrial robots, is their limited payload capacity, execution speed, and precision [7]. Therefore, using conventional industrial robots in collaborative applications yields further potential [10]. In this case, safety aspects can be met using external sensors. For example, vision systems can be used to monitor the workspace for human workers [11] or additional robot “skins” (sensitive covers) can detect collisions before critical forces occur [12]. 

Mobile Collaborative Robotics 

Mobile robot systems must continuously plan their path to the tasks to be performed. An emblematic scenario is order preparation in a warehouse environment, like the Ocado or Exotec solutions [13, 14], which reach a high level of efficiency. Here, restricted areas are defined for robots to operate without humans in the loop. Operators interact with the autonomous system only within specific workstations. But whatever the industrial scenario looks like, introducing mobile robots in a shared human-robot environment requires the 5 ability to integrate humans (co-workers, visitors, drivers of non-autonomous systems), anticipate their behaviors, and to adapt the robot motions consequently. However, Multipath Planning is already a challenging problem to solve [15]. In order to exploit the full potential of autonomous mobile robots in industrial environments, the uncertainty of human actions must be modeled and the robot paths have to be dynamically adapted accordingly. 

Mobile Robot Manipulators 

The integration of the human behavior in the robot decision process is of prior importance in collaborative task scenarios where the mobile robot and the human work with a close and continuous interaction [17]. Mobile robots and Mobile Robot Manipulators could replace the traditional conveyer to transport parts from one workstation to another and could assist operators in their workload. The use of such mobile solutions prevents the blocking and starving of workstations and leads to more flexible manufacturing systems. When designing and implementing Mobile Robot Manipulator applications, it is recommended to also consider the ergonomics of the workers to prevent musculoskeletal disorders (MSDs) and to maximize the performance of such a collaborative work environment [18, 19]. 

Machine vision in human-robot collaboration 

Machine learning and artificial intelligence have allowed considerable advances in computer science. However, its industrial adoption in the domain of robotics is still in its early stages. A review of recent achievements is available in [20]. A number of elementary tasks still remain open: distinguishing backgrounds and objects, identifying moving objects, identifying partially covered objects, recognizing changing shapes or articulation or understanding the position and orientation of objects. All these tasks are required to enable the following paradigms for human-robot collaboration:

  • safety collision avoidance (with humans and obstacles)
  • coexistence the robot capability of sharing the workspace with other workers 
  • collaboration capability of performing robot tasks with direct human interaction and coordination 

The presence of humans in the workspace or the random presence of objects to be manipulated introduces uncertainty that requires sensor inputs for robot control. Four sensory modalities have become dominant: vision, touch, audition, and distance. The project partner CMM (MINES Paris) brings expertise within the embedded vision and distance sensing modality [23] into the consortium. Both can be mounted on moving parts of the robot or fixed in the workspace (referred to as “eye-in-hand” and “eye-to-hand” visual servoing, respectively). The sensing can be used for actuation, servoing, trajectory planning, and security [21]. The sensor inputs are translated to servoing commands via the classic inverse kinematic problem [22] or recently, using reinforcement learning. 

Lidar scanners provide 3D perception to the robot. Their active components make them more robust than image-based 3D reconstruction counterparts. Recent solid-state lidar technology has been used to produce low cost scanners with appropriate specifications for mobile robotics applications.

Objectives of the project 

This project has the following three main objectives:

1. Elaboration of the state-of-the-art regarding human-robot collaboration:

    • Similarities and common design patterns in approaches presented in scientific literature should be identified. 
    • Interesting and competitive best practice applications from academia and industry will be defined.

    2. Industry study and identification of obstacles: 

      • Barriers for human-robot collaboration in industry should be identified from an industry perspective.
      • The results should be aggregated and compared, e. g. regarding companies from different industries, of different sizes, and from different countries (e. g. France, Germany, further European countries). 
      • Gained insights should be shared with the scientific community and industry by publication of a joint research report and conference papers.
      • If necessary, proposals for the adjustment of regulations that unnecessarily restrict the use of human-robot collaboration will be developed. Furthermore, the differences in regulations between France and Germany will be identified in order to support the path toward a unified European economy. 

      3. Definition of future research agenda:

      • Opportunities for further developments of human-robot collaborative systems, both in terms of the technical capabilities of the systems as well as their embedding in surrounding manufacturing systems, should be identified.
      • Shared and complementary know-how of project partners will be identified, and their research activities synchronized. 
      • Based on the above, a future research agenda will be developed, and further funding opportunities will be investigated.
      Expected impact on academia, industry and society
      Industrial Application and Society 

      We expect to give new impulses for academic research on systems involving human-robot collaboration and to identify research questions with high relevance for industry. Therefore, industrial companies might benefit from solutions tailored toward their specific needs. This contributes to the economic and social sustainability of the manufacturing industry. Furthermore, entry barriers into human-robot collaboration can be lowered for companies from different countries by pushing for uniform regulations.

       Academia 

      Identifying potential industrial application scenarios for collaborative human-robot systems can open up new research questions regarding human perception, integration of human partners in the robot’s decision-making process, safety, etc. The proposed project will help in the definition of specific scenarios and challenges to tackle. This helps researchers to adapt their research activities according to industry requirements in order to improve the state-of-the-art for autonomous robotic systems.

      Goals:

      To overcome the barriers of human-robot collaboration this project aims to:

      • Analyze the state-of-the-art: Similarities and common design patterns in human-robot collaborative applications will be identified and best practices will be defined.
      • Conduct an industry study and identify obstacles: Barriers for human-robot collaboration in industry will be identified from an industry perspective. Afterward, the results will be aggregated and analyzed, e. g. regarding companies from different industries, of different sizes, and from different countries (e. g. France, Germany, and further European countries)
      • Develop a future research agenda: Opportunities for further developments of human-robot collaborative systems, both in terms of the technical capabilities of the systems as well as their embedding in surrounding manufacturing systems, will be identified and future research questions will be formulated.