Chinese scholars developed a nursing robot simulation environment and conducted real-person experiments, winning one of the best papers in IROS 2022

From October 23rd to 27th, the top robot conference IROS 2022 was held in Kyoto, Japan. The conference received a total of 3,579 paper submissions from 57 countries and regions around the world, and finally accepted 1,716 papers, with an acceptance rate of 47.9%.

## Among them, Lu Cewu’s team from Shanghai Jiao Tong University and researchers from Cornell University and Columbia University jointly launched Nursing robot simulation environment RCareWorld. This work received one of the six best paper awards at the IROS 2022 conference for Best RoboCup Paper.

Two Best RoboCup papers. Image source: Twitter@ctwy

• Paper address: https://arxiv.org/pdf/2210.10821.pdf
• Github address: https://github.com/empriselab/RCareWorld
• Paper homepage: https://emprise.cs.cornell.edu/rcareworld/

According to WHO data, there are 190 million people in the world with varying degrees of mobility restrictions and need the help of caregivers to achieve a higher quality of life. At present, major countries in the world are gradually entering aging societies of varying degrees, and the demand for nursing staff is rising sharply, and the training of relevant talents requires long-term investment. Therefore, designing nursing robots is a possible solution.

However, the development of the field of care robots faces many difficulties, such as:

1. Cutting-edge researchers lack caregivers and caregivers Research accumulation of first-person real needs (tasks, algorithms, data).

2. The development, deployment, operation and maintenance of real robots are also very expensive.

3. Experiments in this field need to be modified in a targeted manner after understanding the daily activity needs of the care recipients, which greatly increases research costs.

Therefore, a simulation platform that can highly simulate nursing scenarios can greatly lower the threshold for entering this research field, making it easier to conduct research in this field and compare it with academic peers. ​

#Different from previous simulation environments used for general-purpose robots, the RCareWorld team also absorbed suggestions from people involved in caregiving scenarios and robotics researchers to develop a more comprehensive approach to robots. Full support is provided for required learning skills, virtual human modeling, activity scene design, functional interfaces, etc.

Robotic Caregiving Skills

The author benchmarked common caregiving tasks in a simulation environment: feeding, dressing, wiping the body, repositioning limbs, and assisting with movement , help going to the bathroom, etc.

In addition, the authors conducted two real-world experiments:

1. The strategies learned in the body wiping task were directly transferred to the real machine experiment.

​

2. Social caregiving tasks in the real world: The author used behavior tree programming The NAO robot serves as a coach, guiding the physical rehabilitation of the person being cared for through a VR interface.

Complete virtual human modeling: musculoskeletal, soft tissue, joint range of motion

Musculoskeletal system

The musculoskeletal system of the human body will receive activation signals from the nervous system. This determines the contraction or relaxation of muscles, thereby driving the movement of bones and joints. In this process, the author uses Hill-type muscle model modeling and refers to relevant data from the OpenSim database to complete the setting of muscles in the human body model.

Soft tissue

On the other hand, when human body joints move, it will cause surface soft tissue deformation. The author used XPBD-based simulation technology to model this part. These soft tissues exist not only on the surface, but also in the oral cavity. In the oral cavity, the authors also modeled the tongue. As shown in the picture, when a strawberry is placed in a person's mouth, his tongue deforms.

Range of joint movement

When a person suffers some injuries, such as spinal injury (C1-C3, C4-C5, C6- C7), cerebral palsy, hemiplegia, stroke, etc., the mobility of the body's joints will be greatly affected, and the movement pattern will also change. Based on clinical data, the authors modeled the corresponding joint activities of the human body after such injuries.

Activity scene: three levels of accessibility

According to the accessibility of the activity scene, the scene is divided into three levels Level:

• Normal (L1): No modification is required at this time.
• Partial accessibility (L2): At this time, some objects need to be removed, and tools or parts that are inconvenient to use need to be replaced. For example, rotating door handles need to be replaced with push-type ones. , or replace the entire door with a sliding door.
• Completely barrier-free (L3): At this time, all activity areas should be passable and accessible, which may involve widening stairs, aisles, pull-down top-hanging cabinets, and under the countertops Hollow out and other renovations.

The comprehensive modification plan is referenced from the "Universal Design Manual". Following guidance from the Universal Design Guidelines, the authors modified 16 houses. The house model was selected from the Matterport3D dataset and includes a total of 17 kitchens, 17 living rooms, 59 bedrooms, 16 dining rooms, 70 bathrooms, 18 lounges and 41 other rooms. Suitable areas of the house are equipped with hospital beds, patient slings, wheelchairs and other medical assistive equipment.

Functional interface: model, sensing, interactive interface

According to the suggestions of robot researchers, the simulation environment should:

1. Supports control of common care robot models: including HSR, Stretch, Nao, Fetch, Kinova, Franka, UR, etc.

2. Provide multi-modal sensing: RGB, depth, LiDAR, joint and end force sensing, full Arm tactile perception.

​

3. Has a variety of interactive control interfaces and interfaces, supports planning, control and learning algorithms, making it easy for developers to use : Python, ROS, VR.

Shanghai Jiao Tong University launches the embodied intelligence platform RobotFlow/RFUniverse

The RCareWorld project is developed based on the RFUniverse simulation platform. RFUniverse is a multi-physics robot simulation platform under the embodied intelligence platform RobotFlow initiated by the team of Professor Lu Cewu of Shanghai Jiao Tong University. The simulation platform supports advanced robot operation tasks including food cutting, clothing folding and other tasks. Provide support for rigid body, joint body, flexible body, fluid and other object types. The MVIG Lu Cewu team of Shanghai Jiao Tong University has long-term research on embodied intelligence and computer vision. The team has published more than 100 papers in "Nature", "Nature. Machine Intelligence", TPAMI, ICRA, IROS, forming GraspNet (Anygrasp), Aphapose, Well-known robot learning and computer vision systems such as HAKE.

Now open source: https://github.com/mvig-robotflow/pyrfuniverse

RCareWorld Paper Common The first author, Dr. Xu Wenqiang, is the core staff of this system.

Introduction to the co-first author

Ye Ruolin is a bachelor’s degree student in the Department of Electronics at Shanghai Jiao Tong University and a first-year doctoral candidate in the Department of Computer Science at Cornell University, under the supervision of Professor Tapomayukh Bhattacharjee. The research direction is human-robot interaction. The main work of RCareWorld was completed during her internship at the MVIG Laboratory of Shanghai Jiao Tong University (mentor Lu Cewu).

Xu Wenqiang is a fourth-year doctoral student in the MVIG Laboratory of Shanghai Jiao Tong University, studying under Professor Lu Cewu. The research direction is embodied intelligence. Lead the RobotFlow project within the lab, which includes the multi-physics robot simulation platform RFUniverse, which is the foundation of RCareWorld.

• RobotFlow project: https://robotflow.ai
• RFUniverse platform: https ://sites.google.com/view/rfuniverse

The above is the detailed content of Chinese scholars developed a nursing robot simulation environment and conducted real-person experiments, winning one of the best papers in IROS 2022. For more information, please follow other related articles on the PHP Chinese website!