Emma M. van Zoelen1, 2 *, Karel van den Bosch2, David A. Abbink1, Mark A. Neerincx1,2
1 Delft University of Technology, Delft, The Netherlands
2 TNO, Soesterberg, The Netherlands
* Presenting author
A growing body of research on human-agent teaming ,  and human-robot collaboration ,  shows that machines are increasingly acting as team members to humans. The agents and robots in these studies often employ different forms of Machine Learning to enable them to adapt to their environment (e.g. , ). Given that humans are naturally adaptive, it is certain that partners will mutually adapt their actions. Over longer periods of time, this adaptation transforms into co-evolution .
In our research, we investigate how to support human-machine team partners in identifying co-adaptive behavior and in sharing successful Collaboration Patterns (reflective communication ). Such communication enables the human-machine team to achieve shared awareness of these Patterns, and helps them to successfully co-learn.
We have previously run experiments in a virtual Urban-Search-and-Rescue environment . Human participants collaborated with a Reinforcement Learning agent in saving an earthquake victim from underneath a pile of rocks. Both the human and the agent had to learn to use their unique capabilities to jointly complete the task (interdependence ). Teams developed a great diversity of Collaboration Patterns, that were often continued for several rounds of the task. However, due to spontaneous deviations by both the human and the machine partner, mistakes sometimes occurred.
To enable communication and the development of shared awareness of successful Collaboration Patterns, the human-machine team needs a common language. We have developed an ontology that provides a knowledge structure for Collaboration Patterns: a conceptual framework that functions as a basis for communication. An accompanying Graphical User Interface (GUI) enables team partners to formalize and refine Collaboration Patterns through communication. The ontology and GUI were evaluated using video recordings of human-machine teams at work. Results showed that they supported humans in recognizing and defining Collaboration Patterns in the videos successfully. We are currently preparing an evaluation of the use of the ontology and GUI by a human-machine team during task execution.
Collaborative learning is essential for human-machine teams to be successful. Our research contributes to the formalization, implementation and validation of a framework that supports identification, reflection and agreement of successful human-machine team behaviors.
 J. van Diggelen, J. Barnhoorn, R. Post, J. Sijs, N. van der Stap, and J. van der Waa, “Delegation in Human-Machine Teaming: Progress, Challenges and Prospects,” in Intelligent Human Systems Integration 2021, Cham, 2021, pp. 10–16. doi: 10.1007/978-3-030-68017-6_2.
 M. F. Schneider, M. E. Miller, T. C. Ford, G. Peterson, and D. Jacques, “Intent integration for human-agent teaming,” Syst. Eng., vol. n/a, no. n/a, pp. 1–13, 2022, doi: 10.1002/sys.21616.
 A. Ajoudani, A. Zanchettin, S. Ivaldi, A. Albu-Schäffer, K. Kosuge, and O. Khatib, “Progress and Prospects of the Human-Robot Collaboration,” Auton. Robots, vol. 42, no. 5, pp. 957–975, 2018, doi: 10.1007/s10514-017-9677-2.
 Y. Tirupachuri, G. Nava, L. Rapetti, C. Latella, K. Darvish, and D. Pucci, “Recent Advances in Human-Robot Collaboration Towards Joint Action,” ArXiv200100411 Cs, Jan. 2020, Accessed: Mar. 03, 2020. [Online]. Available: http://arxiv.org/abs/2001.00411
 S. Nikolaidis, D. Hsu, and S. S. Srinivasa, “Human-robot mutual adaptation in collaborative tasks: Models and experiments,” J Robot. Res, vol. 36, pp. 618–634, 2017, doi: 10.1177/0278364917690593.
 A. Shafti, J. Tjomsland, W. Dudley, and A. A. Faisal, “Real-World Human-Robot Collaborative Reinforcement Learning,” ArXiv200301156 Cs, Jul. 2020, Accessed: Mar. 08, 2021. [Online]. Available: http://arxiv.org/abs/2003.01156
 D. A. Döppner, P. Derckx, and D. Schoder, “Symbiotic Co-Evolution in Collaborative Human-Machine Decision Making: Exploration of a Multi-Year Design Science Research Project in the Air Cargo Industry,” in Proceedings of the 52nd Hawaii International Conference on Systems Science, 2019, pp. 264–273. doi: 10.24251/HICSS.2019.033.
 Y. Engeström, Interactive Expertise: Studies in Distributed Working Intelligence. Research Bulletin 83. Department of Education, University of Helsinki, Bulevardi 18, SF-00120 Helsinki, Finland., 1992. Accessed: Jun. 01, 2022. [Online]. Available: https://eric.ed.gov/?id=ED349956
 E. M. van Zoelen, K. van den Bosch, and M. Neerincx, “Becoming Team Members: Identifying Interaction Patterns of Mutual Adaptation for Human-Robot Co-Learning,” Front. Robot. AI, vol. 8, p. 200, 2021, doi: 10.3389/frobt.2021.692811.
 M. Johnson, J. M. Bradshaw, P. J. Feltovich, C. M. Jonker, M. B. Van Riemsdijk, and M. Sierhuis, “Coactive Design: Designing Support for Interdependence in Joint Activity,” J. Hum.-Robot Interact., vol. 3, no. 1, p. 43, Mar. 2014, doi: 10.5898/JHRI.3.1.Johnson.