Scooping Grasping with Robotic Hand (with Diffusion Policy Rollouts)

Seunghwan Um, Tae Hyun Bae, Hyouk Ryeol Choi
Sungkyunkwan University

Abstract

Scooping lets a robotic hand lift and grasp objects through contact with the supporting surface. This page presents diffusion policy rollouts and the data collection demonstrations used to study this motion with an AIDIN Robotics hand.

Contact is challenging for robotic hands with rigid fingertips. Fingertip force sensing supports compliant interaction during scooping, as illustrated in the principle and comparison videos below.

As an additional technique, we compensate for the hand's low mechanical stiffness by transforming fingertip sensor forces into the wrist frame and adding them to wrist force feedback for the manipulator's compliance controller.

Main Demo

Scooping grasping with a robotic hand using a diffusion policy.

Policy rollout — Grasping objects of various sizes.

Data Collection

Grasping demonstrations collected for policy learning.

Compliance control absorbs contact impacts, reducing operator workload and making data collection easier.
Principle

Fingertip force sensing enables compliant contact with the environment during scooping and grasping.

Scooping Grasping — Adaptive Contact and Lift-off from the Floor
Scooping Grasping (Book) — Pulling a Book While Maintaining Contact
Comparison

Comparison with and without compliance control during scooping.

Effect of Compliance Control During Scooping
Additional Technique

Compensating for Low Mechanical Stiffness

To compensate for the robotic hand's low mechanical stiffness, we augment wrist force feedback with forces measured by the fingertip sensors. Each fingertip wrench is transformed into the wrist frame and added to the wrist F/T reading, allowing the manipulator to respond compliantly to fingertip contact during scooping.

Fingertip Contact Force — external forces are measured at each fingertip.

Fingertip Contact Force — external forces are measured at each fingertip.

Wrench Aggregation to Wrist Frame

Each fingertip wrench \(\mathbf{w}_i = (\mathbf{f}_i,\, \boldsymbol{\tau}_i)\) measured in the fingertip frame is transformed to the wrist frame \(\{W\}\) via the kinematic transform \({}^W T_i\). The rotation matrix \(\mathbf{R}_i\) and position offset \(\mathbf{r}_i\) are obtained from TF.

\[ \mathbf{f}_i^W = \mathbf{R}_i \, \mathbf{f}_i \]
\[ \boldsymbol{\tau}_i^W = \mathbf{R}_i \, \boldsymbol{\tau}_i + \mathbf{r}_i \times \mathbf{f}_i^W \]

The aggregated wrench at the wrist combines all \(N\) fingertip contributions and is merged with the physical wrist F/T reading \((\mathbf{f}_\text{wrist},\,\boldsymbol{\tau}_\text{wrist})\):

\[ \begin{aligned} \mathbf{f}_\text{total} &= \mathbf{f}_\text{wrist} + \sum_{i=1}^{N} \mathbf{f}_i^W, \\ \boldsymbol{\tau}_\text{total} &= \boldsymbol{\tau}_\text{wrist} + \sum_{i=1}^{N} \boldsymbol{\tau}_i^W \end{aligned} \]

This combined wrench \((\mathbf{f}_\text{total},\, \boldsymbol{\tau}_\text{total})\) is fed as the external force input to the impedance controller, enabling the manipulator to react compliantly to both wrist-level and fingertip-level contacts during the scooping grasping motion.

Fingertip Force Transformation — the fingertip F/T values are transformed and aggregated to the wrist frame for compliance control.

Comparison: w/ vs. w/o Wrist Wrench Aggregator

The hand's low mechanical stiffness can limit the response to fingertip contact when using wrist feedback alone. Adding the transformed fingertip forces to the wrist feedback helps the manipulator respond more sensitively to floor contact.

Without Aggregator — fingertip forces are not reflected to the wrist; the hand cannot react to contact sensitively.

With Aggregator — aggregated fingertip wrench at the wrist enables sensitive, compliant scooping from the floor.