Use StretchSense Hand Engine data in Shogun Post

You can import hand skeletons and animation captured in Hand Engine into Post and merge it with your Vicon actor skeleton. You can then use the merged data as a source for retargeting in Post or elsewhere.

Sync capture of StretchSense Hand Engine data in Shogun Live

You can sync capture from Hand Engine with a Vicon system by streaming timecode and triggering capture from Live. Hand Engine then produces:

• An FBX file per take, per actor, per hand, containing a skeleton of the hand and animation. The FBX contains the timecode.
• A JSON file with various metadata

You can merge the hand skeleton with the Vicon body solve skeleton in Post. Merging these skeletons in Post enables you to produce full body and finger animation for your actor's solve skeleton.

StretchSense hand skeleton and mesh in base pose:

Import and merge StretchSense Hand Engine data in Shogun Post

A script that enables you to import and merge your Hand Engine FBX (see Sync capture of StretchSense Hand Engine data in Shogun Live) is installed with Shogun 1.8 and later in this location:

C:\Program Files\Vicon\ShogunPost#.#\Scripts\FileIO\ImportStretchSenseFBXFiles.hsl

The way you use the script depends on the version of Hand Engine you use:

Hand Engine 1.5.1

In Hand Engine 1.5.1, automatic import of all FBX files for all subjects in the open take is not supported.

To import the FBX and merge the data into the skeleton:

1. Load a take that contains the solved actor Vicon body skeleton.
2. If multiple subjects are present, set one as the current subject to import hand data for it.
3. Run ImportStretchSenseFBXFiles.hsl.

4. In the file chooser dialog box, select the two FBX files that are associated with the current subject.

   The script merges the skeletons and their animation in the FBX into the body skeleton, and transfers the animation for the hand bones.

5. For each actor, repeat steps 2–5.

Hand Engine 1.5.2

In Hand Engine 1.5.2, automatic import of all FBX files for all subjects in the open take is supported as long as you use the following naming and location conventions:

• The hand FBX files must be a sub folder of the opened take whose name is the same as the take name. For example:

  If you set the capture path in both Shogun and Hand Engine to:

  G:\MOCAP\StretchSense\ViconCapture

  The following are produced in that folder:

  • An MCP file named MyTake_001.mcp
  • A subfolder named MyTake_001 from Hand Engine, which contains the hand MCP files

• The hand FBX files must be named in the form P#*_PerformerName.fbx

To import the FBX and merge the data into the skeleton:

1. Ensure the files are correctly located and named (see above).
2. Load a take that contains the solved actor Vicon body skeleton(s).

3. Run ImportStretchSenseFBXFiles.hsl.

   All FBX files with the correct naming scheme are imported and merged to their respective skeletons for all actors in the scene.