Overview

This document describes how to use the Creature runtimes in the Unreal 4 Engine. The language of the runtimes is in C++. We will go through an example of how to load an exported JSON file and play it back in the scene.

Using Creature in Unreal Engine, the Basics

Video Tutorials

Unreal

Click here to watch an overview of Creature Unreal Engine Integration. This tutorial uses the new Creature UE4 Plugins showcasing the CreatureMeshComponent framework as well as a couple of Blueprint functions used to swtich animation states. Watch this video tutorial if you are new to the Creature UE4 plugins.

Unreal

Click here to watch a more in depth look at using the Creature Runtime plugin and its Blueprints functions to switch animation sets + set collider objects that follow bones on the character

Unreal

Click here to watch an in depth tutorial on how a complete Dragon Character with multiple animations is brought into UE4 and driven using the Blueprints Visual Scripting System and Creature UE4 Plugin.

Unreal

Click here to watch a tutorial on how to setup the Creature plugin on a Mac.

Unreal

Click here to watch a tutorial on how to do runtime Image/Region Swapping as well as the usage of Anchor Points in your game.

Unreal

Click here to watch a tutorial on how to use the Live 2 Bone IK Posing functionality in your game.

Unreal

Click here to watch a tutorial on how playback a character with Animated Mesh Region Ordering exported from Creature.

Unreal

Click here to watch a tutorial on how to author a path using the Spline Component in UE4 and then import it into Creature for use as a character motion path.

Unreal

Click here to watch a tutorial on how to use the Alembic Export option in Creature to export your character as an Alembic file and bring it to life in UE4.

Unreal

Click here to watch a tutorial on how to use the Frame Callback Triggers function in the Creature Plugin to activate events useful for footsteps and spawning of particles at specific animation frames of the character.

Unreal

Click here to watch a tutorial on how to Author Frame Callback Triggers using the new Event Authoring Editor in the Creature Tool itself. The exported event triggers can be loaded in from UE4. This tutorial complements the previous tutorial showing you how to do event authoring outside of UE4 but from within the Creatue Editor.

Grabbing the runtimes

You can either download the runtimes directly from the Creature Game Runtimes window or grab them from the repository here. For your custom engine, you will need to grab the UE4 runtimes which contain the core Creature runtime library files.

Different Ways to play Creature Animations in Unreal Engine

Method 1

  1. Creature a Plugins folder within your UE4 Project directory

  2. From the github repository, go into Creature_UE4/CreatureEditorAndPlugin

  3. Copy the CreaturePlugin directory over into your game project's Plugins directory.

  4. Right-Click on your .uproject file and Generate Unreal Project Files to update your Visual Studio project.

  5. Compile and Run the project in Visual Studio.

Method 2

If you are only dealing with Bone Motor animation, simply export your characters into FBX or Alembic format and import them as regular FBX or Alembic animations in Unreal Engine.

How to use the UE4 Creature Runtimes

You can use the Creature Runtimes in UE4 in 2 ways:

  1. As a CreatureMeshComponent. You create an empty actor object and from there add in a CreatureMeshComponent for playback.

  2. You create a CreatureActor object and use it for playback.

Using the UE4 Creature Runtimes as a CreatureMeshComponent

This is the new way of using the UE4 Creature Runtimes. The concepts involved are very similar to the usage of CreatureActor. Please read the usage of CreatureActor for more information.

This section covers how to setup your empty actor for Creature animation playback:

  1. Create an empty Actor

  2. Convert it into a Blueprints Object

  3. Open the Blueprints Editor, and then Add Component

  4. Add a CreatureMeshComponent object in Blueprints

  5. From here on, you are good to go. The controls on the CreatureMeshComponent are the same as that of the CreatureActor so please read through the section on CreatureActor for more information.

CreatureMeshComponent and CollectionClips

The concept of CollectionClips essentially allows you to play back multiple animations in different exported JSON files as one giant clip.

This allows you to combine animations with different rigs and textures making it a very powerful tool for situations where you require a character representation with different rigs. More information about CollectionClips and the concept behind it is described in the section Combining Multiple Character Animations.

Here are the steps needed to use CollectionClips:

  1. In your asset panel: Right-Click -> Others -> CreatureAnimationAsset

  2. A window will pop up asking you to select your json file.

  3. With your created CreatureAniamationAsset, drag in your character's material into the Collection Material slot.

You should create as many CreatureAnimationAssets as the number of json files you want to combine.

Next:

  1. Again Right-Click -> Creature -> Creature Animation Clips Store

  2. Double-clicking on your Creature Animation Clip Store brings up a new panel.

  3. Now start filling up the Clip List array.

To Fill up the Clip List array:

  1. First click on the + sign to add a new element.

  2. Give your combined clip a new Clip Name e.g MyCombinedClip

  3. In the Shot Clip List array, start adding the actual combination of clips you want to compose.

To Fill up the Shot Clip List:

  1. Add a new entry by clicking on the + sign.

  2. First select the Source Asset. These are the CreatureAnimatioAssets you created previously.

  3. Fill up the actual Animation Name for that entry. This is the actual name of the animation clip in the json file.

Add as many entries into the Shot Clip List as you need to compose.

Finally, in the Blueprints Editor, check Enable Collection Clip to enable the collection clip functionality. You can tell which collection clip to play by calling the SetBluePrintActiveCollectionClip function.

Using the new CreatureAsset with CreatureMeshComponent

CreatureMeshComponent now works in conjunction with CreatureAsset, a new way to load in animations into the runtimes.

To create a CreatureAsset:

  1. Right click in the Content Browser

  2. Go to Misc -> CreatureAnimationAsset to create the asset

  3. Select your exported Creature JSON file

That is all that is required and a new CreatureAsset should be created in UE4. Now go into your CreatureMeshComponent and point the CreatureAnimationAsset slot into your newly created asset and you are done.

Using the UE4 Creature Runtimes as a CreatureActor

The main concept behind the UE4 Creature Runtimes revolve around the ACreatureActor class. This is the class that you add to your UE4 scene to playback your creature animations.

Take a look at the ACreatureActor header file and pay attention to the following properties/functions:

Base Properties

  • creature_filename This is the string that you can set in the editor pointing to the JSON file of your creature character

  • animation_speed This determines how fast the animation is played back when the game runs. The default value is 1.0 which is normal speed. To make the animation run 2x as fast, set this value to 2.0

  • start animation name The name of the animation to start with when it begins playing.

  • smooth_transitions Setting this to true will enable smooth transitions using the Blueprints function SetBluePrintBlendActiveAnimation. If you set this to True, you MUST USE SetBluePrintBlendActiveAnimation() to set the animations.

  • bone data length factor This is to be used in conjunction with the GetBluePrintBoneXform function. It scales the output transform for your attached bone colliders, altering their length. Play around with this value to fit your bone colliders onto the skeleton of your character.

  • bone data size This is to be used in conjunction with the GetBluePrintBoneXform function. It scales the output transform for your attached bone colliders, altering their height. Play around with this value to fit your bone colliders onto the skeleton of your character.

  • region overlap z delta This variable adds a z offset value to every region/layer in your character. This is required because you are authoring a 2D asset in a 3D engine. Every new region in your rig gets added this z delta value. You should set small positive or negative values for this variable (0.01 or -0.01). Pick between positive or negative value depending on where the camera is or where the character is facing. For example, if you character switches from facing left to right (a typical 180 degree rotation), you are going to have to flip or negate this value to make sure the layers/regions are rendered in the correct order.

  • animation frame This variable returns the current animation frame of your playing animation

Debugging/Drawing Issues

Is your character disappearing for no reason? Having problems seeing your character animation playback? Try these variables for some debugging:

  • creature_debug_draw This renders out the wireframe bounding sphere of your character

  • creature bounds scale This scales the bounding sphere of your character. Scale this up to a large value to avoid your character getting culled by the rendering engine

  • creature bounds offset Put a vector value to offset the bounding sphere is for any reason it is not centered on your character due to a double transform down your pipeline setup

Finally, go into Settings->Rendering and turn off occlusion culling.

Animation Functions

  • SetBluePrintActiveAnimation(AnimationName) This is a Blueprints function that can be called with a given animation name. Calling this function results in the character switching to a new animation with the given name.

  • SetBluePrintBlendActiveAnimation(AnimationName, Delta) This is a Blueprints function that can be called with a given animation name. Calling this function results in the character switching to a new animation with the given name in a smooth transition. The second parameter is how quickly the current animation will transition to the next. Set 0 < Value < 1 for Delta.

  • SetBluePrintAnimationCustomTimeRange(AnimationName, startTime, endTime) This is a Blueprints function that sets custom start and end times for an animation clip.

  • SetBluePrintAnimationLoop(flag) This turns on/off animation looping

  • SetBluePrintAnimationPlay(flag) This starts/stops the animation.

  • GetBluePrintAnimationFrame() Returns the current animation frame

  • SetBluePrintAnimationResetToStart() Resets the animation to the start frame

  • SetBluePrintAnimationPlayFromStart() Plays the animation from the start frame

  • SetBluePrintRegionAlpha(FString region_name_in, uint8 alpha_in) Sets the alpha/opacity value of the region given the name

  • void SetBluePrintRegionCustomOrder(TArray order_in) Allows you to re-order the regions of your character. Provide an input array of strings containing all the region names but ordered in your custom order. Make sure the size of this array is the same as the total number of regions you have in your character

Animation Events

  • BlueprintAnimationStart(frame) A Blueprint event that you can override. This is triggered when the animation starts.

  • BlueprintAnimationEnd(frame) A Blueprint event that you can override. This is triggered when the animation finishes.

Custom Animation Frame Callback Events/Triggers

You can create and trigger customm frame callback events for use in character sound effects like footsteps or the mission of particles at a specific frame of the animation.

The setup for this is very straightforward. There are 2 types of callbacks available:

  • CreatureFrameCallback This allows you to trigger a callback at a specific frame in the animation
  • CreatureRepeatFrameCallback This is a repeating callback, it repeats every specified X frames with a given offset

Here is the Blueprint Layout to create the regular frame callback:

UE4

As you can see, it involves calling the SetBluePrintFrameCallbacks function with an array of CreatureFrameCallback structs. Remember to fill in the correct details for each struct in the array.

Similary, the repeat callback trigger is setup the following way:

UE4

After the callbacks are setup, it is simply a matter of making your gameplay code aware of the callback delegates:

UE4

You should make the gameplay code aware of the input Name parameter in the delegate. Use the Name parameter to parse and determine how/what behaviour to run when the callback is triggered.

Custom Animation Frame Callback Events/Triggers using the Creature Editor

You can now author events directly in the Creature Editor itself and export them out into the engine.

  1. First, open up the Event Triggers edit window in Creature: UE4

  2. Now Add in your custom events. Events are added at the current time/frame you are on in the animation: UE4

  3. Export your character and import in the .mdata Creature Meta Asset into the scene. This is covered in the animated region ordering section of this document.

  4. Point your CreatureMeshComponent to the new Creature Meta Asset.

  5. Now in EventBeginPlay(), call the LoadBlueprintFramCallBacksAsset() to load the events you authored.

Proceed as normal by connecting up the required callback delegate documented in the previous section.

Bone Introspection and Collisions

  • GetBluePrintBoneXform(boneName, doWorldTransform) This is a Blueprints function that returns the transform of the bone in the character. Toggling the doWorldTransform flag will return the positions in world space. The return data is in the form of a FTransform. You can use this to place a collider object.

Step 3.1: ACreatureActor Object

The first step is to add the ACreatureActor Object into your scene. Properties for this object are listed under the Materials group:

UE4

Set the Creature Filename to the path of your character JSON file. Set the Start Animation Name to the starting animation clip that you want to play. You can set different values in Animation Speed to determine how quickly the animation plays back. If you want to use smooth transitions during animation switching, remember to toggle Smooth Transitions ON.

Finally, remember to set an appropriate material with your character's texture so as to display the character properly on screen.

Step 3.2: Switching Animations in Blueprint

UE4

The ACreatureActor object has methods exposed to the Blueprints system. Of particular interest is the function SetBluePrintBlendActiveAnimation(AnimationName, Delta). This is the function that allows you to smoothly transition the character between animation clips.

Using the system is very simple. Just add the function referenced to your character into BluePrints. Then set the following:

  • Name In The name of the animation clip to switch to
  • Factor How quickly the current animation is blended into the target animation (0 < value < 1)

Link up the exec slot of this function to the rest of your gameplay logic and watch your character change animation states.

Step 3.3 Swapping out In Game Items/Regions

You can switch/swap out specific mesh regions of your character using the runtimes. This is useful if you want to perform switching of character items like hats, shoes and weapons.

Preparation for switching

  1. Make sure Enable Image Swap is enabled for the region you want to swap in the Creature Editor

  2. Also make sure you have A Single Image Swap Knot set from the Sprite Frame Manager for the region of interest

  3. For the other sprites that you want to swap in for the region, open up the Sprite Frame Manager and take note of the Tag value:

UE4

The tag value is always Negative. This value is the number you will put into the runtime later on for in game item swapping.

At run time

UE4

Call the SetBluePrintRegionItemSwap function with the Tag you want from the previous section. When this function is trigged, the region mesh will have its image swapped out for the new image corresponding to the desired Tag value.

Step 3.4 Combining Multiple Character Animations

CreatureCollectionActor is an Actor object that allows you to string together a collection of CreatureActors and play their animations one after another as if they were just part of a Single Animation Clip.

This is useful if you have 2 or more different rigs representing one character and want to play their animations from the 2 different animations back to back as if it was one single merged animation clip.

How to use

1) You will need to create the individual CreatureActors first that you want to control.

2) Create a CreatureCollectionActor, and then put the CreatureActors that you want under the CreatureCollectionActor. This is so that the child actors transform according the the parent CreatureCollectionActor.

UE4

3) Start building up your collection clip: A Collection Clip is composed of a sequence of (CreatureActor, CreatureActorAnimationClip) pairs. That is you are basically saying, “I want to add this actor playing this clip first, then that actor playing that clip second etc.”

The CollectionClip itself is given a name. This name is the name you use to represent this collection of animation clips from different actors.

In this picture below, I am making a new CollectionClip called “clip1”. I am adding the first Child1 actor with its animation Cyclops_IDLE to “clip1” Then I am adding the second Child2 actor with its animation Cyclops_HIT_1 to “clip2”

UE4

4) Now actually start playing your new CollectionClip, treating it like one big animation clip! Normal clip setting Blueprint functions like SetBluePrintActiveClip as available to you to enable you to switch between your composed clips. There are also events like CreatureAnimationEndEvent that triggers when the collection clip has finished playing:

UE4

Step 3.5 Anchor Points

Often times when you are animating your character, you might not have centered them at the same origin. Anchor Points allow you to specify the character's relative origin for a specific Animation Clip. This allows you to center different character animations if so desired.

  1. In the Creature Editor, run Animate -> Set Anchor Point

  2. Set the Anchor Point by dragging the Red Spinning Box around the character: UE4

  3. In UE4:

UE4

Activate the anchor point function by calling SetBluePrintUseAnchorPoints to true.

Step 3.6 In Game Realtime IK

You can run IK live on 2 bones in your character by setting up IK Constraints in UE4. Take a look at the reference Blueprints Graph:

UE4

  1. First, make a CreatureBoneIK struct. The struct has the following properties:
  • First Bone Name : Name of the first bone for your IK system

  • Second Bone Name : Name of the second bone of your IK system

  • Target Pos : The target position to pose the 2 bones in world space

  • Positive Angle : Whether the IK system targets positive or negative angles

  1. Next, call the function SetBlueprintBonesIKConstraint with the CreatureBoneIK struct you created in the previous step. Calling this function during the event tick trigger will update the bones to the target position live during gameplay.

To remove your ik constraint, simply call RemoveBlueprintBonesIKConstraint.

Step 3.7 In Game Live Bones Positional Override

You can override the positions of your bones live in the game with the following setup:

UE4

  1. First, make a CreatureBoneOverride struct. It has the following properties:
  • Bone Name : Name of the bone

  • Start Pos : The starting postion of the bone in world space

  • End Pos : The ending postion of the bone in world space

  1. Next, create a new array and connect up the bone override structs you created in the previous step to this array.

  2. Call the function SetBlueprintBonesOverride with array created in step 2.

Animated Region Ordering

If you authored your character from Creature with Animated Region Ordering, there will be a extra file generated. This file has the extension of type .mdata. Create a Creature Meta Asset and then point your new asset at the .mdata file. After that, go into your CreatureMeshComponent of the Blueprint Actor and set the Creature Meta Asset slot to point to your newly created asset. This activates animated region ordering for your character.

Advanced Users: Understanding the Core Creature Runtime Library

The above went through a description of using the Creature UE4 runtimes. There are times when you might want to modify or add additional functionality to the core Creature Static Runtime library. Below is an in depth look into how the Static Runtime Library works.

Here are the files you should focus on:

  • MeshBone.cpp & MeshBone.h - This is the posing engine of the Creature runtime. You can access the entire skeletal hierarchy, its corresponding bones as well as mesh regions here.

  • CreatureModule.cpp & CreatureModule.h - This is the layer that is responsible for managing Creature characters and posing the characters with loaded in animation files.

You will notice that the UE4 ** ACreatureActor** class calls into CreatureModule in order to playback Creature animations.

How Loading and Initialization is done

Let us assume we have an exported dragon animation file called dragonTest.json. We also have its corresponding texture atlas called character-dragon.png. We start off by first loading the file assets:

CreatureModule::CreatureLoadDataPacket json_data;
CreatureModule::LoadCreatureJSONData(filename, json_data);

The above will load the JSON data from disk and into memory. Next, let us create the actual objects that can make use of these loaded assets:

auto cur_creature = std::shared_ptr<CreatureModule::Creature>(new CreatureModule::Creature(json_data));

creature_manager = new CreatureModule::CreatureManager(cur_creature);
creature_manager->CreateAnimation(json_data, "default");
creature_manager->CreateAnimation(json_data, "second");    

In the example above, the JSON file has 2 animation clips: default and second. Hence, we will need to create 2 animations from the creature_manager object to make them available for playback.

Now that we are done loading, we can set the active animation for playback:

creature_manager->SetActiveAnimationName("default");

This sets default as the currently active animation. If you take a look at LoadAnimation(), AddLoadedAnimation and LoadCreature in ** ACreatureActor**, you will notice that they perform exactly these steps in order to setup the character for playback.

CreatureManager and Creature classes

In order to playback and display a Creature character in your engine, you need to have access to the Playback/Posing mechanism as well as the topology, points and texture coordinates of the final Creature character mesh.

-CreatureManager - This is the class that manages animations and the loaded character. You will be using this class to advance the animation by a certain delta time. Your game loop or renderer should interact with the update function of this class at each "game tick".

-Creature - This is the class that contains the loaded creature. It can be accessed from the CreatureManager. This class will contain all the geometry/rendering data required for your game engine to display on screen.

Update and Advancing the animation

Every game engine has an update/game tick call. In this case we run:

void ACreatureActor::Tick(float DeltaTime)
{
	if (creature_manager)
	{
		creature_manager->Update(DeltaTime * animation_speed);
		UpdateCreatureRender();
	}
}

to tell the manager to update its Creature character by a certain delta time. This will advance the animation forwards by the value delta.

Display the mesh

The Creature object returns the following: indices, uvs and points. These attributes are sufficient for you to render out the character mesh. Here is how it is done:

void ACreatureActor::UpdateCreatureRender()
{
	auto cur_creature = creature_manager->GetCreature();
	int num_triangles = cur_creature->GetTotalNumIndices() / 3;
	glm::uint32 * cur_idx = cur_creature->GetGlobalIndices();
	glm::float32 * cur_pts = cur_creature->GetRenderPts();
	glm::float32 * cur_uvs = cur_creature->GetGlobalUvs();

	TArray<FProceduralMeshTriangle>& write_triangles = mesh->GetProceduralTriangles();

	static const FColor White(255, 255, 255, 255);
	int cur_pt_idx = 0, cur_uv_idx = 0;

	for (int i = 0; i < num_triangles; i++)
	{
		int real_idx_1 = cur_idx[0];
		int real_idx_2 = cur_idx[1];
		int real_idx_3 = cur_idx[2];

		FProceduralMeshTriangle triangle;

		cur_pt_idx = real_idx_1 * 3;
		cur_uv_idx = real_idx_1 * 2;
		triangle.Vertex0.Position.Set(cur_pts[cur_pt_idx], cur_pts[cur_pt_idx + 1], cur_pts[cur_pt_idx + 2]);
		triangle.Vertex0.Color = White;
		triangle.Vertex0.U = cur_uvs[cur_uv_idx];
		triangle.Vertex0.V = cur_uvs[cur_uv_idx + 1];

		cur_pt_idx = real_idx_2 * 3;
		cur_uv_idx = real_idx_2 * 2;
		triangle.Vertex1.Position.Set(cur_pts[cur_pt_idx], cur_pts[cur_pt_idx + 1], cur_pts[cur_pt_idx + 2]);
		triangle.Vertex1.Color = White;
		triangle.Vertex1.U = cur_uvs[cur_uv_idx];
		triangle.Vertex1.V = cur_uvs[cur_uv_idx + 1];

		cur_pt_idx = real_idx_3 * 3;
		cur_uv_idx = real_idx_3 * 2;
		triangle.Vertex2.Position.Set(cur_pts[cur_pt_idx], cur_pts[cur_pt_idx + 1], cur_pts[cur_pt_idx + 2]);
		triangle.Vertex2.Color = White;
		triangle.Vertex2.U = cur_uvs[cur_uv_idx];
		triangle.Vertex2.V = cur_uvs[cur_uv_idx + 1];

		write_triangles[i] = triangle;

		cur_idx += 3;
	}

	mesh->ForceAnUpdate();
} 

As you can see, it is a simple act of sending the points, uvs and indices into your rendering pipeline and drawing the data as a series of triangles.

Custom Time/Frame Range

You can set custom time/frame ranges for the currently active animation. Say you wanted to limit the playback to the frame range of 10 to 20, you would do the following:

auto all_animations = creature_manager->GetAllAnimations();
all_animations["myAnimation"]->setStartTime(10);
all_animations["myAnimation"]->setEndTime(20);

Animation Blending

You can blend between 2 animation clips by doing the following:

creature_manager->SetBlending(true);
creature_manager->SetBlendingAnimations("default", "second");
creature_manager->SetBlendingFactor(0.5); // 0 to 1 blends between the 2 clips

Smooth Transitions with Auto Blending

If you want to smoothly transition from one animation state to the next, you can use the Auto Blending feature in the runtimes.

To enable Auto Blending, first do:

creature_manager->SetAutoBlending(true);

after you have added all animations into your creature manager object.

To use Auto Blending, a single call to:

creature_manager->AutoBlendTo(NEW_ANIMATION_NAME, 0.1f);

will result in the character smoothly transitioning to the target animation called NEW_ANIMATION_NAME. The second parameter determines the time delta of the transition, with a range of 0.0 to 1.0.

Overriding/Modifying Bone Positions

Sometimes you need to modify the bone positions of the character directly. For example, you might want the positions of the bones to follow a number of rigid bodies connected with springs/joints for ragdoll physics. In the cases where you need to set bone positions for your own custom requirements, you should do the following. First, write your custom bone override method. Here is an example that displaces the bones in y by some amount:

// This is an example of how you can use a callback function to modify the position of the bones
// on your character. In this example, we will displace all the bones by a fixed amount in y.
void
HelloWorld::bonesModifyCallback(std::unordered_map<std::string, meshBone *>& bones_map)
{
    for(auto& bone_data : bones_map)
    {
        auto cur_bone = bone_data.second;
        auto cur_start_pos = cur_bone->getWorldStartPt();
        auto cur_end_pos = cur_bone->getWorldEndPt();
    
        cur_start_pos.y -= 5;
        cur_end_pos.y -= 5;
    
        cur_bone->setWorldStartPt(cur_start_pos);
        cur_bone->setWorldEndPt(cur_end_pos);
    }
}

You will also need to tell the CreatureManager to use your custom bone modify callback like this:

// Example of how to register a callback function to modify the bones
std::function<void (std::unordered_map<std::string, meshBone *>&) > cur_callback =
    std::bind(&HelloWorld::bonesModifyCallback, this, std::placeholders::_1);
creature_manager_2->SetBonesOverrideCallback(cur_callback);

Speeding up Playback Performance with Point Caches

If you want to increase the speed of the playback animations and not pay the penalty for the posing engine, you can enable point caches on specific animations.

Point caching essentially stores away the vertices of the animation, allowing the playback to bypass the posing engine. Character rigs with very complex skeletons and deformations benefit greatly from point caching.

To enable point caching on a specific animation, do this:

my_creature_manager->MakePointCache("myAnimationName");

That is all that is required to create the cache. Remember that the cache is specific to the animation, not character. This means that if you have instanced multiple characters, you only pay the cost of a single cache for that animation. This results in both memory savings and playback performance speedups.