Face Anim showcases the use of Apple's facial tracking technology, included in TrueDepth API.
It detects when you speak and it animates the active character with your voice by applying an sound effect.
This project shows different 3D characters who will mimic your facial movements on your phone.
All characters are manually sculpted by the developer.
With Microphone permission, when the app is active (it has no background activity and it does not record anything in the background), the recording will automatically start or stop by itself in case the noise level is above or below a specific threshold that can be changed by the user.
Camera permission is required in order to capture facial expressions.
The app only works on devices that have the TrueDepth camera through ARKIT SDK.
*** Changing the 3D Character
By pressing the lower left button of the screen, a list of characters is shown. By pressing any of the previewed characters, it will replace the active mesh.
The last icon representing the wire frame face shows the mesh used by the ARKit SDK to track your face.
Tip: This debug mesh is highly performant and has the lowest impact on device thermals.
Each character has its own sound effect and facial effects.
*** Settings
The Settings window, accessed by touching the lower left button of the screen and then the gear icon when using the app, provides access to various modes and settings used by the app.
These include:
- Microphone noise threshold
- Loading the default settings of the app
- Resetting the microphone recording
- Show Debug Values
- Viewing the application and game engine log
- Help and About (this screen)
*** Microphone noise threshold
In order to calibrate the microphone recording threshold with the environment, it is possible to increase or decrease its value. It basically represents the noise level at which the recording should start.
In a noisy environment, it should be increased. In a silent environment, it should be decreased.
The default value is set to a medium ambient environment. Please feel free to change it. Its value will be remembered upon app restart.
It is possible to manipulate the recording functionality by pressing the state Icon on the lower right side of the screen, meaning:
yellow circle: Will start recording the voice including the facial movement
red circle: Will stop the recording and start the playback
playback: Will stop the playback and rerun the recording cycle
*** Show Debug Values
Debug Values is a special UI overlay that shows the data being sent in from ARKit SDK. This can be used to verify calibration or to read specific values for error tracking.
The data shown represents the current frame value of each of the 51 blend shapes being sent in from ARKit.
It detects when you speak and it animates the active character with your voice by applying an sound effect.
This project shows different 3D characters who will mimic your facial movements on your phone.
All characters are manually sculpted by the developer.
With Microphone permission, when the app is active (it has no background activity and it does not record anything in the background), the recording will automatically start or stop by itself in case the noise level is above or below a specific threshold that can be changed by the user.
Camera permission is required in order to capture facial expressions.
The app only works on devices that have the TrueDepth camera through ARKIT SDK.
*** Changing the 3D Character
By pressing the lower left button of the screen, a list of characters is shown. By pressing any of the previewed characters, it will replace the active mesh.
The last icon representing the wire frame face shows the mesh used by the ARKit SDK to track your face.
Tip: This debug mesh is highly performant and has the lowest impact on device thermals.
Each character has its own sound effect and facial effects.
*** Settings
The Settings window, accessed by touching the lower left button of the screen and then the gear icon when using the app, provides access to various modes and settings used by the app.
These include:
- Microphone noise threshold
- Loading the default settings of the app
- Resetting the microphone recording
- Show Debug Values
- Viewing the application and game engine log
- Help and About (this screen)
*** Microphone noise threshold
In order to calibrate the microphone recording threshold with the environment, it is possible to increase or decrease its value. It basically represents the noise level at which the recording should start.
In a noisy environment, it should be increased. In a silent environment, it should be decreased.
The default value is set to a medium ambient environment. Please feel free to change it. Its value will be remembered upon app restart.
It is possible to manipulate the recording functionality by pressing the state Icon on the lower right side of the screen, meaning:
yellow circle: Will start recording the voice including the facial movement
red circle: Will stop the recording and start the playback
playback: Will stop the playback and rerun the recording cycle
*** Show Debug Values
Debug Values is a special UI overlay that shows the data being sent in from ARKit SDK. This can be used to verify calibration or to read specific values for error tracking.
The data shown represents the current frame value of each of the 51 blend shapes being sent in from ARKit.
Show More
