Problem detecting legs with Orbbec sdk with Unity

[aLeXvI65]

Hello wveryone, I have a problem using the unity SDK for Orbbec, when I run the scene, the avatar puts the legs in a wear position, I already run the application with the orbbec device in many locations and different lighting conditions, I also made tests with the unity Chan avatar from unity asset store and I had the same results.

I hope someone could help me solve this problem.

[jmp909]

what happens if you wear white pants?

[younger]

Unity avateering joint orientation issue

I make avateering script with orbbec unity SDK but it has problem. In my video, it works pretty good but when I raise the knee, its joint doesn't get rotation value. How to fix it? If you need the avatar script, I send email or reply. Thank you. Younger.

It is already known problem and orbbec knows it.
If you want to solve it before next release, calculate rotation by position with Quaternion.LookRotation function.
Use calculated rotation value only for lower body.

[aLeXvI65]

Thanks for your response younger, I understand the Quaternion.LookRotation needs a vector3 to look at, usually a target object, like in this example:

https://docs.unity3d.com/ScriptReference/Quaternion.LookRotation.html

My question now is which target should each joint look at to fix this problem, for example, should knee look at hip?.

Thank you.

[younger]

Replace UpdateBone function in PoseUser script.
Honestly, I don't understand why it is because I don't know mathematics.
But it maybe works.
Additionally, change value of RotationDamping to 10.
Sensor must be able to see lower body.
Vector3.down is direction to desired joint joint from torso when T-pose state .
For example, if you want to use this to right arm, change Vector3.down to Vector3.right.

Vector3 GetPosition(Astra.Joint jo)
{
    return new Vector3(jo.WorldPosition.X, jo.WorldPosition.Y, jo.WorldPosition.Z);
}

private void UpdateBone(Body body)
{
    for (int i = 0; i < body.Joints.Length; ++i)
    {
        if (body.Joints[i].Type == JointType.LeftHip || body.Joints[i].Type == JointType.RightHip ||
            body.Joints[i].Type == JointType.LeftKnee || body.Joints[i].Type == JointType.RightKnee)
        {
            Vector3 mainJointPos = GetPosition(body.Joints[i]);
            Vector3 nextJointPos = GetPosition(body.Joints[i + 1]);
            
            if (mainJointPos != Vector3.zero && nextJointPos != Vector3.zero)
                _boneRotation[i] = Quaternion.LookRotation(nextJointPos - mainJointPos, nextJointPos - mainJointPos) * Quaternion.LookRotation(Vector3.zero - Vector3.down);
        }
        else
        {
            _boneRotation[i] = GetRotation(body.Joints[i]);
        }
        Joints[i].transform.rotation = _currentRotations[i] =
            Quaternion.Slerp(_currentRotations[i], _boneRotation[i] * _initialRotations[i], Time.deltaTime * RotationDamping);
    }
    return;
}