Characters: Designing Customizable Colors in Blender & Three.js

How we look and how we feel about ourselves is intimately connected. In Relm, we want you to feel at home, and firmly in your own shoes while building and being part of your community online.

To that end, we’ve put a lot of thought into character customization, and the technology that goes into it. This is part 1 in a series of posts explaining why and how we’re making customizable characters for you. And if you’re building similar technology, we hope you can benefit from our lessons learned.

The Low-Poly, Low-Bandwidth Constraint

One of the major constraints for character design in a web browser environment is the size of the character assets. We want the world to load quickly, and we want it to render at 60 fps for people with regular phones or computers (let’s say, 2015 or more recent).

We could design a whole bunch of characters to give variety and self-expression to people who take up an avatar in Relm; however, each character would be a new asset file and a new set of data to download for each participant. If there are 100 characters to choose from, that means everyone in the world needs to download 100 character assets so they can see everyone else’s avatars.

One Character, Many Colors

Instead, it would be ideal if we could create one character design that lets a participant choose their character’s colors: skin color, hair color, clothing colors, etc.

The challenge is that a 3D model has many surfaces (“faces” in 3D modeling lingo) and vertices, but it’s unclear to the game engine which surfaces or vertices belong to what “parts” of the 3D model. If we want the player to be able to change their skin color, for example, the game engine doesn’t know how do differentiate surfaces that are skin from those that are clothing. We need a way to “annotate” the model and say “these vertices belong to the feet” or, “this surface belong to the torso”.

Blender already has the tools we need to create the type of 3d model that would enable color customizations:

• colored vertices
• vertex groups, and
• face maps

However, exporting to glTF (the native format supported by three.js) doesn’t necessarily keep all of that information. The current built-in glTF exporter throws it away.

We needed to add some capabilities to the glTF exporter, as well as to the Relm engine (which uses three.js), so that participants can select colors for their avatar’s skin, hair, clothing and other aspects of self-expression.

Custom Blender Exporter

If you download & install the following addon to Blender (replacing the default glTF Import/Export addon), it will include Vertex Group & Face Map metadata from Blender in the glTF (or .glb) files you export:

io_scene_gltf2Download

If you’re new to addons, see this tutorial on how to install the addon in Blender.

The source code is also available if you’d like to see inside. Our Github repository is a fork of the official glTF-Blender-IO addon, and is available at https://github.com/canadaduane/glTF-Blender-IO.

As you can see in the Blender screenshot above, you can create named Vertex Groups or Face Maps. Vertex Groups can be used for coloration, but we found that they were less than ideal because a colored vertex will “blend” its color with adjacent vertices. In other words, you won’t get clean lines of separation between colors as you’d want between, say, a white shirt and a black belt (it would be white-grey-black).

We recommend using Face Maps for custom colorization because you’ll get both a slightly smaller file size, as well as clean, separate colors from face to face (see “viewer” below).

If you’re interested in the technical details:

• Each Vertex Group is added to the three.js geometry attributes as a _vg_[vertex_group_name] custom attribute (e.g. you can access the four Vertex Group shown in the screenshot as mesh.geometry.getAttribute('_vg_four')). It contains what is essentially a “mask” where 0 means a vertex is “not in the set” and 1 means a vertex is “in the set”. Thanks to scurest‘s work, the exporter is smart enough to use a sparse index if there are just a few vertices in the group.
• All Face Maps are added to the three.js geometry attribute as a single _facemaps attribute. Each vertex is given a Face Map group ID from 0 to n (where n is the number of Face Maps being exported). If a vertex does not belong to a face with an assigned group, it is given the value -1. There is an additional userData.facemaps property in the userData field of the mesh that contains the geometry. This property holds the names of the Face Maps, in an array. For example, ["top", "bottom"] would mean that Face Map group 0 is named top and Face Map group 1 is named bottom.

See this Github issue for further details.

Custom glTF Viewer

Don McCurdy’s glTF viewer is the gold standard in knowing, “does this glTF work in three.js?” However, in order to support these custom attributes, we needed to modify the viewer to see the _vg_[vertex_group_name] and _facemaps attributes and teach it how to use them to color a section of the model.

You can clone or download our custom glTF viewer and run it locally on your machine. In addition, you can try it out here if we still have it running (no guarantees on this fast-paced project, sorry!)

https://viewer.relm.us

Putting it All Together

With Vertex Groups or Face Maps as annotations for various parts of the character model, your character designer can annotate the “hair”, “face”, “shirt”, “boots” or whatever else you need to be able to color as separate parts. Then, the game engine can tell which parts of the model are semantically separate and take the vertices’ colors and lerp them towards a new target color (e.g. change the hair to “blonde”).

Finally, using these basic building blocks, we’ll be able to design a character creator in Relm. Next, we’ll explore how to use morph targets and animations to make the character have different body shapes and genders, all while being able to re-use the same animation data.

We can’t wait for you to be able to express your presence the way that you would like to, and be with others in the Relm community!