This sprint was all about adding features to the RPM avatar in the game and then developing a usable pipeline for making custom RPM animations in Blender and remapping existing animations on other skeletons onto the RPM avatar and bringing them into the game.
The first thing that was needed to be done was to fix the fact that facial animations that synched the RPM Avatar mouth to the voice on the voice channel no longer worked for the newer version of RPM avatars created by the RPM development team. Working with the RPM team it came out that they had removed the blend shapes from the downloaded avatars to make the download smaller and faster. That means by default they would not work because the blend shapes were no longer available to be mapped to the phonemes that we had done in the myMO code. But, they said that we should be able to accomplish this by calling their API to ask that the blend shapes be included by adding some code in myMO. After a couple of days, we got this all working so that the lip sync for voice works again on all RPM avatars (new or old).
We then modified the screen flow for the RPM Avatar hub now embedded in the game to firstly add an Exit button, so that you can cancel out of the RPM avatar customization and return to the game. This was not in the original GUI in their plugins. As long as we were in these UMG Screens, we also added another screen at the end of the RPM avatar customization process to scale the Avatar and name it as it is saved into the games native Avatar Inventory screens.
The biggest coding task of the sprint was adding an initial Attachment system for the Avatar so that you could add a mesh attachment to the avatar. Attachments like headphones, guitars, microphones, etc, make sense to have for a music game like myMO. To start with 4 sockets were added to the skeleton to attach these meshes too. There is a chest socket, head socket, left hand socket and right hand socket. More may be added as we improve this attachment system in the future. You click the attachment to load it on the avatar and then you can edit it’s position and scale on the avatar. For example, take a guitar and place it where it should be for whatever animation you are planning on using. The positions and rotations are done using the same basic screens with slider switches that are used in the placing of props in the games, but the resulting positions are all relative to the socket, so that when your avatar moves the attachment moves with the avatar. This required quite a bit of programming as the basic inventory structure that is used through out the game had to be changed to add the socket mesh selected and the positions. A given avatar could add 4 attachments to their avatar (one in each socket).
Adjusting attachments:
The attachments are saved into the current Avatar Appearance as a SAV file, so that the next time a person get’s in the game they will be wearing the same attachments as they had on the last time they played the game. Currently you can’t save it to the Avatar Inventory, but that is planned as a future enhancement. The attachment system was the major change to the game for this sprint.
The rest of the sprint was dedicated to developing an avatar animation pipeline. Currently in the game we had converted a number of Mixamo Animations so that they worked within myMO. These included a bunch of standard animations for walk cycles, dancing and common gestures. We realized that this needed to be added to in order to have additional animations that made sense in a Music Game. Things like playing instruments, even more dances, etc would be needed as myMO is being developed. There is no good RPM Avatar based animation tool or workflow available as a plugin in UE or add on in Blender. So, we explored the options we had for developing one and ended up with the following workflows.
The first thing we need to support was making animations from scratch in Blender. In order to do this, we needed to have a control rig for the RPM Avatar in Blender and then have a way to export the resulting custom animation done with the control rig so that it would work with the Unreal Engine import process. This turned out to be not as straight forward as we hoped, but, we ended up with the following workflow which took advantage of the free Mixamo Control Rig that is available on their website.
The second workflow we needed to develop was one for retargeting animations that were done for the Poser Skeleton into something that would work on the RPM Avatar. We wanted this workflow because we have a library of over 5000 Poser animations that we had created based on the Carnegie Mellon Human Motion Studies Library done a few years ago, as well as hundreds of Poser animations and Poses from both a Game Development version of Poser we owned and the purchase of the Truebones Poser Libraries. A great number of these libraries will be used as at least a starting point for new animations in myMO as we try to make this as rich as experience as we can by adding these animations. Here is a video of the workflow for Poser To RPM Animation retargeting using Blender and an addon named Auto Rig Pro:
This concluded the sprint for improving Avatars in myMO!
