Answered

Face cap fbx not recognized

Woodyy 3 years ago updated by Peter - Soxware Developer 3 years ago 9

I have rigged a blender character using the FACEIT' app to use the ARKIT shapekey system found in apples Face Cap app. I finished a normal rigging with auto rig pro and out put a humanoid FBX of character. when I bring the character into umotion pro it recognizes and lists the shapekeys in the character but keyed action does not show in the character. I have imported a face cap action fbx and  just get an error that it appears as generic not humanoid.

     Umotion would be a perfect tool to tune face cap dialogue and make it  the goto tool for unity mocap animation...if I could get this to work, any suggestions?

UMotion Version:
umotion pro
Unity Version:
2019.3.15

Answer

Answer
Answered

Hi Woodyy,

thank you very much for your support request.

Ok so the face rig is using blend shapes for the facial expression or is it using bones? Are you sure this isn't a problem with your character / your import pipeline? You can verify if the blend shapes on your character are working by dragging your character to the Unity scene. Then look for the GameObject that has the SkinnedMeshRenderer component and open the "Blend Shapes" foldout in the inspector. Here you can play with the blend shape weights (100 means full influence of the blend shape):

If the blend shapes also don't work in that case, you might want to double check your setup in blender and your import/export settings. If they do work using the method mentioned above, it might be a problem with UMotion not previewing the blend shapes correctly for some reason. If that is the case, I would love to take a look at it. Please send me your character (without textures if you want) and your UMotion project file via email support form (or request a link to my dropbox if file size is too big for email transmission).

Please let me know if you need any further assistance. I'm happy to help.

Best regards,
Peter

GOOD, I'M SATISFIED

Hi Peter,  I emailed my answer and links to the wrong address yesterday, Sorry! I resent this morning to the support.assets@ this morning. Thanks Again!

Satisfaction mark by Woodyy 3 years ago
Answer
Answered

Hi Woodyy,

thank you very much for your support request.

Ok so the face rig is using blend shapes for the facial expression or is it using bones? Are you sure this isn't a problem with your character / your import pipeline? You can verify if the blend shapes on your character are working by dragging your character to the Unity scene. Then look for the GameObject that has the SkinnedMeshRenderer component and open the "Blend Shapes" foldout in the inspector. Here you can play with the blend shape weights (100 means full influence of the blend shape):

If the blend shapes also don't work in that case, you might want to double check your setup in blender and your import/export settings. If they do work using the method mentioned above, it might be a problem with UMotion not previewing the blend shapes correctly for some reason. If that is the case, I would love to take a look at it. Please send me your character (without textures if you want) and your UMotion project file via email support form (or request a link to my dropbox if file size is too big for email transmission).

Please let me know if you need any further assistance. I'm happy to help.

Best regards,
Peter

Thanks for sending me the related files.


The reason why the blend shapes of the character "Gran14c" doesn't seem to work is because your character has each blend shape 3 times. For example the "cheekPuff" blend shape exists in the "EyeB_left", "EyeB_right" and (the correct one) in the "GranHead" skinned mesh renderer. In UMotion, you've probably only tried the one named "cheekPuff" which is the first one thus the one from "EyeB_left" which does nothing. The correct one would be "cheekPuff (2)" which is the one of the head. The cleanest way to fix this would be to remove all unnecessary duplicated blend shapes from your model (in your 3D modeling application).

In either project when I try to add the FaceCap FBX as clip, I receive the error-"Umotion incompatible Clip-the transforms referenced by this clip ‘generic’ etc”.

Ok so this is the error message you are referring to:

The problem is that the animation you are trying to import is referring to transforms that are named differently or are at a wrong position in the rig hierarchy compared to the character you are currently animating in UMotion. Thus the "addressing" of the animation curves can't be resolved (i.e. the animation curves in the clip you want to import point to transforms not existing in your character). You can fix this by making sure that the hierarchy of the animation you want to import is the same as the hierarchy of your character. You can make these adjustments in your 3D modeling application of choice.


Please let me know in case you have any follow-up questions.


Best regards,
Peter

Thanks Peter! On the first problem, I will eliminate any FaceCap blendshapes from eyes. Facecap makes these on eyes probably because it is used for animated avatars alot, where eye action wont always be symetrical. But in character animation we will always want the eyes to move together so I will make my own Lt-Rt ,Up-Dn blendshapes for these. The second problem of clip addressing transforms as named in my character may still be a problem because the FaceIt blender app that I used to create the blendshapes names these to be compatible with FaceCap. It seems like I should try some sort of file conversion between FaceCap and Unity. I will try running the mocap data thru the FaceIt app on blender and see if that helps. I wonder if it is just a syntax problem between the two files though. I'll try these things and update you on the progress. Thanks as Always!!! 

Yours,

Woody

I wonder if it is just a syntax problem between the two files though.

Transforms in an animation are addressed by "transform paths" (e.g. "Armature/Spine0/Spine1/Neck/Head" would address a transform named "Head" that is a child a transform named "Neck" and so on). The transform paths in the animation you want to import don't match with the transforms that exist in your character. You can see this if you try to assign the animation you want to import to your character (in regular Unity). Then open Unity's Animation Window and you will see that the animated transforms are all yellow because it can't find them in your character.


It is possible to edit the transform paths (by selecting the *.anim file that lies within the imported FBX and pressing CTRL + D to extract a duplicated version as a separate file, then open the duplicated file with a text editor; if you scroll a bit through the file, you are going to notice that it contains theses transform paths at some places; you can edit them manually here).


If you know how to code, you can also edit the transform paths of anim clips via Unity API.


Best regards,
Peter

Thanks Peter! This is just what I needed. I understand what to do now. Thanks again for your patience, and explanation, I really appreciate it!

Yours,

Woody

Hi Peter,

  I haven't succeeded yet in getting the renaming of transforms to work yet. I also tried Facemotion 3D as that has a Blender plugin but to no avail. I think I will try Rokoko next. From what I can find on the web, a lot of people have similar problems. I'll update after I try Rokoko. 

Yours,

Woody

Here is another thing you can try that came to my mind recently when answering another support request:


You could copy & paste the keys from your face only animation into your characters animation using Unity's animation window. First open an animation of your character, then add each blend shape property to it (by pressing on Add Property, blend shapes can be found under the skinned mesh renderer components). Then you can open the animation that contains the actual face motions, copy the blend shape keys. Then go back to your characters animation, select the exact same blend shape properties (in the properties list). Then press CTRL + V to paste the keys. As long as the blend shapes have the same names and you've selected the same number of blend shapes (and probably also in the same order), this should work. Maybe start by copying just one blend shape's keys for simplicity and see how that goes.

Let me know if you need further/more detailed instructions.

Best regards,

Peter

thanks Peter, that does work, but I got Rokoko yesterday and it works great. You just drop your character onto their stagw window in Unity press a button and it naps all your bones and blendshapes. You can do face capture or mocap with your character right there and the export seems to be be problem free. The minimum subscription is $58 /month…as much as Adobe but it works. The suit and gloves would be another $3500, but my wife would kill me if I got those. As it is all I need is my iPhone. I saw a lot of rigging problems in my character testing it yesterday, so I’ve got to improve that but now I can continue onward with the new film. I will keep you updated on all. Thanks as always Peter!

Yours,

Woody

Great to hear that you've found a workflow that works for you.

I wish you all the best for your new film.

Best regards,
Peter