Scooping Grasping with Robotic Hand(with Reinforcement learning & System Identification)

Sungkyunkwan University

Abstract

We study contact-rich scooping and grasping with reinforcement learning, using scene registration and closed-loop system identification to reproduce physical robot dynamics in simulation. Interactive policy recordings, physical contact observations and measured-response evaluations describe the modeling approach and its current validation.

Main Demo

Physical execution: grasp formation and supported lift. One 11-second recording at normal speed.

Simulation rollouts across object geometries

Select an object, replay the motion, and rotate the 3D view.

Earlier joint-PD policy recordings from the original RL page. These illustrate learned behavior; they are separate from the later identified-model experiments. Recording details ↗

Contact-constrained motion on the physical robot

A short unsuccessful attempt isolates the approach, object tipping and subsequent hand motion.

Seven-second excerpt: approach/contact, box tipping, then tabletop interaction. The yellow wireframe is the object goal; the green wireframe is a pose estimate. The attempt does not complete a lift, and the video does not measure impact force.
Read the measured / simulated response comparison →
Experimental platform and recording provenance

Platform: 6-DoF arm and 15-command dexterous hand (Doosan M0609 / AIDIN left hand). The object-centric policy uses 131 observations and 21 actions and is trained/configured at 60 Hz; identified-model training uses 960 Hz physics. RGB-D perception supplies object pose estimates.

Contact: B1 box, run 20261026_011804_983134_execute, policy 117k. Lift: O1 octagonal prism, run 20261030_122430_812426_execute, policy 129k. The recordings use different objects and runs.

Contact clip provenance · Lift recording provenance

Joint constraints, mechanism and contact geometry

Inspect the coupled workspace and the modeling choices behind the hand.

Recorded model studies retain their original conditions and source data. Open this study ↗

Engineering details

Control-model comparison, identification equations and measured-response results.

Joint PD and identified dynamicsControl structure, drive parameters and delay

The policy outputs joint targets in both configurations. The baseline uses the simulator's joint PD drives. The identified model converts the arm targets to a TCP pose and reproduces the deployed Cartesian impedance controller, sensor estimates and drive dynamics.

Arm actuation model; policy interface held constant
ComponentJoint PD baselineIdentified model
Control inputJoint target qdTCP target from FK(qd)
Control lawτ = Kp(qd − q) − Kdq̇Cartesian impedance replica + inverse dynamics
Plant and driveNominal rigid-body model and configured joint gainsIdentified payload, reflected inertia, directional friction and breakaway
Feedback and delayJoint position and velocity; no identified low-level delay modelAPI signal model, filters, friction observer and measured delays
Contact responseDeflection determined by joint-space drive gainsResponse determined by the reproduced physical controller and identified plant

Both models have finite stiffness. Identification targets the magnitude and timing of the physical closed-loop response.

Baseline gains and implementation

Arm gains: Kp = [326, 326, 160, 67, 33, 33] N·m/rad; Kd = [54.142, 60.94, 11.8525, 3.2775, 2.1714, 1.65] N·m·s/rad. The identified runtime disables these generic arm drives and supplies controller/drive efforts instead.

Baseline configuration · Identified runtime

Real-to-sim modeling and system identificationScene registration, equations and fitting procedure

2.1 Scene registration and geometry

Transform native RGB-D measurements into the policy world frame, fit the table plane, and apply the same surface and mounting transforms to training and deployment. The measured table height/orientation and 20 mm flange-to-hand offset determine object reset height and collision geometry.

Native D455 depth→Policy world frame→RANSAC + plane refinement→Aligned training scene
Plane estimation objective and calibration scope +

Native depth points are deprojected and transformed through the camera's factory depth-to-color calibration and the retained camera-to-policy-world transform. A RANSAC fit rejects outliers; orthogonal least squares refines the table plane using the inliers:

(n^,d^)=argmin‖n‖=1,d∑i∈𝓘(nTpiW+d)2

piW is a depth point in the policy world frame and 𝓘 is the RANSAC inlier set. Fixed spatial image tiles are held out from fitting. The 30-frame capture gives a 1.064° plane tilt and 2.343 mm point-to-plane fit RMS.

The plane determines table height, roll and pitch. The 20 mm flange-to-hand spacer is represented in the robot geometry and deployment kinematics; object reset height follows the table surface. The table footprint and placement can be varied during training. Plane residual is not absolute robot-to-scene alignment accuracy, and this step reuses rather than independently recalibrates the camera extrinsic transform.

Depth processing and plane fit ↗ · Training geometry integration ↗

2.2 Controller and plant identification

Record desired TCP poses, joint motion, commanded motor torque and joint torque sensor signals. Identify link/payload and drive effects separately, then evaluate the complete controller-in-loop response on a recording excluded from fitting.

CONTROLLER MODEL

Cartesian impedance reference acceleration

aimp=Md−1(Kde−Bdv^+F^ext)

Md, Bd and Kd are the desired Cartesian inertia, damping and stiffness. Pose error e contains translation and rotation-vector error; velocity and external wrench are filtered estimates.

Structural form. The replica also preserves the deployed controller's native unit conversions, Jacobian mapping, torque compensation, friction observer and limits.

PLANT MODEL

Rigid-body and drive dynamics with external load

[Mlink(q)+diag(B)]q¨=τcmd(t−δτ)−h(q,q˙)−fdrive(q˙,|τJTS|)−flink(q˙)+τext

B captures reflected drive inertia. h includes Coriolis and gravity terms; drive friction depends on velocity and joint-sensor load. External torque enters the plant, allowing motion to respond to contact.

01

Link and payload regression

Fit payload, link friction and sensor bias to joint torque sensor (JTS) data using linear least squares. Regularize wrist center-of-mass corrections and project payload inertia to physically valid values.

02

Drive identification

Use delayed commanded motor torque minus JTS to identify reflected inertia and directional, load-dependent friction. Estimate breakaway from stick-to-slip transitions; identify 2 ms state and 6 ms actuation delays, reproduced at the simulation step resolution.

03

Held-out trajectory evaluation

Feed desired TCP poses into the replica and generate its subsequent motion. Evaluate a held-out recording excluded from fitting, then run the identified model in the learning environment.

Identification objective and implementation details +

For each joint, bounded least squares separates reflected inertia, load-dependent friction and a velocity lookup table:

minBj,μj,wj∑t[dj−Bjq¨j−sgn(q˙j)μj|τJTS,j|−Hj(q˙j)wj]2

dj(t) = τtarget_motor,j(t − 6 ms) − τJTS,j(t), using the commanded motor torque before the actuation delay. The basis Hj uses separate positive/negative piecewise-linear velocity tables with 18 speed knots. The fit bounds reflected inertia to [0, 10] and the load coefficient to [0, 0.5]. Detected external-load intervals are excluded from link/payload fitting and free-motion evaluation.

The controller uses e = [pd − p; Log(RdRT)] and a low-pass-filtered API wrench after offset subtraction. Identification is performed component by component; full closed-loop trajectories are a subsequent validation.

The learning integration disables the generic PhysX arm position servo and applies the replicated controller and identified drive model. This is the concrete change from the baseline arm actuation used in this adaptation.

Drive identification source ↗ · Controller and simulation integration ↗
Measured-response validationFree motion, constrained contact and model ablation

3.1 Free-motion trajectory replay

The replay initializes from measured state at the start of each window. Subsequent motion and controller feedback are generated by the model from the recorded desired poses.

EvaluationTCP position RMSOrientation RMS
Independent recording; aggregate of 20 s generative windows1.45 mm1.14°
Later hardware command replays; approach segment6.0–8.3 mmNot summarized here

These are different captures and evaluation windows. The later approach mismatch remains unresolved. Identification report · Later replay audit

3.2 Constrained-contact and release response

Replay recorded arm/hand targets with the simulated box constrained, then release it at the recorded time. This tests motion under a prescribed constraint; it does not independently predict contact timing or force.

TrialQuantityMeasured robotIdentified simulation
01Held-back displacement (mm)6567
01Release speed (mm/s)384391
02Held-back displacement (mm)2316
02Release speed (mm/s)189166

Held-back displacement is relative to a contact-free reference, not direct target tracking error. Release speed is the peak in the following 0.5 s. Trials 01/02 use release times 3.660/2.778 s. Protocol and measurements

3.3 Actuation-model ablation at a fixed policy

Same reference policy, seed 42, 320 environments, one episode per environment
Arm modelPhysics rateLift ≥10 cmGoal success
Joint PD120 Hz92%76%
Joint PD960 Hz68%29%
Identified model960 Hz70%28%

The largest change is associated with the physics rate. At 960 Hz, the two arm models give similar task outcomes. This experiment tests policy sensitivity to the simulation model; it does not establish that identification improves hardware success.

Reference policy: 110k checkpoint from the identification report. Evaluation conditions

Matched PD / identified simulation / real-robot comparison

A direct comparison should replay the same commanded targets from the same initial state through both simulated arm models at 960 Hz, with matched geometry, object placement, friction and release conditions. Use synchronized footage and compare TCP trajectory error, constrained displacement and release velocity. This isolates model fidelity.

A separate policy-transfer comparison should train with each arm model under matched physics rate, observation/action definitions, rewards, training budget and object distribution, then evaluate on matched hardware tasks. This isolates the effect of the training model on transfer.

The available reports contain identified-model versus real response measurements, but no verified three-way video set. The latest simulation recordings are not present in this local review package. Unmatched policy demonstrations are not used as a controlled comparison.

Multi-object training distributionFive shape families and 480 geometric variants

The later training configuration contains five shape families and 480 object variants: the original 320 scaled geometries plus 160 larger solids. Variation includes object dimensions, initial orientation/position, friction and arm-model parameters.

DimensionTraining variation
ShapeOctagonal prism, flat-bottom apple, box, frustum, cylinder
Larger solidsAxis scaling 1.2–1.5; grasp width ≤120 mm; mass ≤0.5 kg
Initial poseStanding placements, continuous yaw, mixed 3D orientations and flipped frustums
WorkspaceExpanded table and a later ±0.20 m object-spawn window
Contact and dynamicsFriction variation; identified arm with payload/friction/sensor-offset randomization
Selected training-derived geometries: 20 designs across five families
Twenty selected printable object geometries from five training shape families
Twenty selected training-derived geometries used for the print/pose catalog. Geometry overview; four selected designs per family.

The 320-environment ablation above is a separate evaluation. It does not establish performance over all 480 later variants. Training distribution (D147) · Object geometry provenance

Supplementary model studies and sources

The earlier interactive studies remain available separately: coupled joint workspace, slider mechanism, and contact-pad support experiment. Their original conditions and data are preserved.

Code and reports · Archived policy recordings · Learning framework: SimToolReal