Lathreas, Kistaro, SiberDrac, Athamanatha, Gear, Zennith
Quite some time has passed since our last post on the Anatomy CAD Tool project. It’s high time to give you an update on what we have been working on the past year! As with all major projects, the first big steps have mostly involved planning and designing the system, and form one of the most important stages of the project. Now that we are getting closer and closer to a first visual demonstration and a proof of concept, we are excited to give you an update on where we are at, and what the future will hold for this project. To catch you up, this post will be quite long, but hopefully interesting at the same time. Enjoy!
The FFF seeks to find ways to enable full body modifications, to allow people to be what they want to be. This is particularly challenging in multiple ways. For example, any transformation must satisfy two requirements: the end result must satisfactorily affirm the individual’s identity, including species identity, and the end result must be biologically viable. These two goals partially oppose each other, since more intense body alterations will be more likely to have adverse effects. The brain, for example, must not be harmed. Blood flow must not be constricted. Joints must be able to hinge, and muscles must be able to move unconstrained.
The human body is a machine, with each part present for a specific purpose. Each joint in your body has a specific shape, guided by the laws of physics, to ensure they can move correctly. Simply changing the looks of bone structures, such as adding a snout or changing limb length, will have an impact on the function of these ‘machine parts’. As such, we cannot go in blindly and change things at will; any artist seeking to model a transformation outcome must ensure that the body will work as intended. Just like we must model an airplane before we can build it, we must model any major body modifications before attempting to implement them.
Project outline & research question
For a full transformation, many tissue types will need to be changed. This includes bones, muscles, tendons, ligaments, the integument, and many others. To ensure that engineering of those tissues can proceed in a satisfactory and safe way, we need a Computer Aided Design (CAD) tool that allows bioengineers and artists to design and view body alterations. This system should give researchers and engineers the artistic freedom to change the shapes of bones, muscles, and other tissues while maintaining their physiological functions.
Present-day CAD tools tend to focus on industrial applications, such as the design of vehicles, machinery, or other non-organic objects. A few CAD tools focus on implants and prosthetics, but these do not provide fully extensible and parameterized modeling of anatomical systems.
Such modeling is critical for our needs, as are a few other requirements we have identified. Therefore, we are developing a novel tool for this purpose. We expect that this program will provide artists and bioengineers an important tool in their toolkit, so that they can flexibly define the outcome of a biologically viable transformation. Many other challenges will still exist, but will be covered in other projects.
To start building this CAD tool, we decided to first focus on modeling a relatively small structure of the human body: the elbow region. We intentionally chose the elbow due to its complexity, expecting to rapidly confront and solve important technical risks. For example, the way that the radius and ulna pronate and supinate, while the elbow itself acts as a hinge joint, represents an important biomechanical challenge for modeling. The musculature that makes movement of those bones possible is even more sophisticated. Understanding and accounting for variation, including the influence of muscle size on the shapes of developing bones, increases the complexity further.
If done correctly, modeling the elbow will lay the foundations for CAD of the entire musculoskeletal system, allowing us to later expand the engine to regions with even higher complexity, such as the face. Let’s dive in.
Developmental biology of anatomy
Before modeling something, you have to understand it. So, let’s take a crash course on the important determinants of anatomy we want to model.
Much of our research has been into what aspects of developmental and skeletal biology are most useful for our program. In order to aid future efforts in affecting genetic pathways and morphogenetic processes and to best design prototypical bones, we start with a discussion of “pattern formation”.
Pattern formation is how the shapes of animals (and other multicellular organisms) are ultimately determined and fleshed out from a single fertilized egg. The vertebrate skeleton doesn’t have signals that say “here’s a finger” and “here’s a shin”. Instead, it has protein pathways that will lay out a spatial or temporal pattern, such as spots or stripes, which say “yes skeleton” at these spots or “no skeleton” in-between the spots.
Once that pattern is laid down, the cells that get the “yes skeleton” signal become flexible cartilage. That cartilage grows and gives way to rigid bones. The spaces between become joints to allow mobility. And while all this is happening, the bone is being encased in muscle. That muscle starts to contract on its own, which sends signals to cells in the space between that get chemical and physical cues to become tendons and ligaments.
That’s when the real morphogenesis begins. These cues pull and tug at the skeleton like hard clay to bend and deform it into familiar shapes: the club-like head of the humerus; the cylindrical capitulum at the other end; the protrusion on your elbow called an olecranon. All these shapes are defined by and connect to muscles; and where and how these muscles attach plus how the bones fit into one another, equals what kinds and what strengths of movement the skeleton can achieve. See the below figure (by SiberDrac) for an example of how muscles influence bone shape, gathered from scientific literature.
Mechanisms of muscle influence on bone shape
Going this deep into developmental biology allows us to predict which output shapes are indeed realistic, and adapt our model accordingly. Putting all this together in general terms helps to plan and generate a model that can best reflect the desired shapes and abilities a user might desire. Furthermore, by applying this approach throughout the body, we ensure that whatever we display on screen is something that can actually exist in the physical body as well.
Early in this project, we compared humeri from various diverse species, from Australopithecus to moles, from elephants to foxes. The team used this to get a sense of where the variables are in humerus development and evolution. For each bone, we collected data focused primarily on variability among primates because the earlier phases of achieving morphological freedom will be anthropomorphic; thus, the arms are likely to still be used similarly to the way we and other primates use them now. Data from other species is also being collected, of course, for use down the road.
By gathering data from studies and meta-studies that have closely followed both the evolutionary history of specific bones and compared the anatomy within and across various clades, we have identified a set of those variables that we believe to be the most valuable “control points.” Control points are, simply put, the first set of parameters a user can change to design the shape they want to see. Need more flexible shoulders? Shift the position of the head of the humerus. Bulkier, stronger shoulders? A more pronounced deltoid tuberosity with subtly different shape will do. Control points also include more basic parameters, like length and diameter, and prior data from clinical and paleontological studies allows us to set safe ranges for these parameters.
We have been making good progress on defining these control points. The humerus has been very well defined by comparative anatomists and we have been able to synthesize their data into what we believe is a complete set. Most of the fine-tuning of the humerus is done at the distal end, where complex geometry dictates elbow flexibility and strength. There is less readily-available data on radius and ulna variability, though. Control points in these two bones are length and diameter as expected, plus the length and depth of the olecranon of the ulna, but there are likely others we have yet to define.
Interestingly, when it comes to actual numerical relationships and bounding, length and diameter of the long bones in the arm scale isotropically among primate species (though this scaling changes by degrees and become anisotropic in other clades), and the same studies that provided our set of control points also provided subsets of relevant measurements and interdependencies among them. These relationships are incomplete, though, and may require further work such as in silico experimentation with models of mechanical stability. Such experimentation would be a good reason for this project to continue after the initial release of the software.
Concept of our model’s design
Using the underlying fundamentals of developmental biology and anatomic diversity across species, we have been carefully designing and implementing a system capable of parametrically representing humanoid anatomy. The design takes biological mechanisms and functions of tissues into account, and will help ensure that any change made by the user will result in a functional anatomy.
In embryonic development, the precursors of bones are initially defined in embryonic development as capsule-shaped structures. The tissue, as it develops into a bone, is predisposed to create a bone of a particular length and overall size, without any input from the surrounding environment. Similarly, our model initially defines bones as roughly capsule-shaped structures that are later deformed. This unique design ensures that the bones remain smooth, while remaining biomechanically correct.
Important details, such as the places muscles attach, and where bones interact with other tissues to form joints, are created from complex signaling and interactions between those tissues and their neighbors. Similarly, in our model, the user can change the deformation of the bone into any shape they like, while the program ensures that the shape doesn’t occlude other bones or hinder rotation.
Since much of the complexity of a bone’s shape is at joints, we have focused extensively on how to properly model joints. For example, the hinge joint between the humerus (upper arm bone) and the ulna (lower arm bone) enables a single, controlled direction of rotation. Biologically, the ulna rotates around the cylindrical shape formed by the humerus. To model this hinge joint, our program creates a cylindrical shape around the axis of rotation, then ensures that the tip of the ulna is far enough away from the humerus to rotate freely (see figure). We’re using similar processes, as appropriate, for other joint types.
Left: Shape-fitting around a joint’s axis. Right: Two bones (purple) being shaped in two different ways by a HingeJoint (green) in our program.
Subsequently, muscles have attachment sites that will be modeled as deformations on the bone. The muscles themselves will be represented as deformed cylinders, similar to the way we currently generate bone geometry. All of these components together allow the user to focus purely on placing down all the organs and features where they want to, without them worrying about the details of anatomic correctness.
Combining it into a musculoskeletal system
To organize the loose components into a full body, we group the objects in a “Character” object. This Character object contains all instances of “Bone”, “Muscle”, and similar objects. Different types of Bone are represented with subclasses, such as “LongBone”, “FlatBone”, “ShortBone”, “SesamoidBone”, or “IrregularBone”, allowing us to use the appropriate geometric model for each bone type. For example, in the elbow, the humerus, radius, and ulna are all instances of LongBone.
This data structure lends itself to parametrization. In the humerus, we set the initial length and diameter with a curved, cylindrical shape, without considering the influence of external factors. Subsequently, the program refines the humerus’s contact surfaces with the shoulder joint (head of humerus) and the elbow joint (trochlea and surroundings) by shaping them around user-defined Joint objects. This overall framework is on track to let us represent a broad range of biological diversity, and is powerful enough for us to implement any type of bone inside the vertebrate body. By editing values in either intrinsic or extrinsic parameters, we are indeed able to represent changes in bone shape in predictable and meaningful ways, without changing the relationships between objects.
Our next challenge for representing information is tackling how to represent the influence of muscles and tendons on bone shape. The deltoid tuberosity is a great example of this: the deltoid muscle “pulls” on part of the humerus, deforming the developing outwards during embryonic development. In animals with larger deltoids, this translates into a larger deltoid tuberosity! In the 3D Anatomy application, we’ll need to link muscle/ligament data to a specific corresponding site on bones, and use that information to deform the bone’s shell outwards. This might, or might not, be similar in implementation to how we implemented joint deformation of bones. We have a few ideas, and we’ll let you know what works!
Left: a work-in-progress example bone generated using our engine, using a HingeJoint and a LongBone object. Right: real bone for comparison.
Best practices and maintainability
“Software engineering is what happens to programming when you add time and other programmers.” — Russ Cox
The software we are building is ambitious, and intended for deployment – not simply proof of concept. While small, single-person projects may be completed in one or two sittings, software projects like ours are expected to grow in complexity across the course of development. Following best-practices is essential for maintainability.
For example, software engineers create bugs when their expectations about how the program works are incorrect, so a sustainable long-term software project must be designed to be as unsurprising as possible. We have onboarded Kistaro, whose experience with big software projects helps to ensure we use the best code practices to keep the software maintainable as it grows. A big emphasis in maintainability is on testing, including automated tests, which we increasingly employ to catch those pesky bugs before they become too hard to find. To ensure we keep the program open to everyone, we are regularly testing the program on multiple operating systems, including different distributions of Linux, to ensure we avoid unknowingly building on top of a platform-specific dependency.
Developer documentation of our project
In these and other ways, we are working hard to follow standard practices such as good design patterns, clear class hierarchies, automated unit test coverage, documentation, refactoring along the way, running the program on different platforms, and conducting manual/directed testing.
At the same time, we are developing a user interface to go alongside the program’s mathematical engine. UI components, currently developed in the Godot open-source game engine, are separated from the model and engine, so model use and construction is not tied to any particular frontend framework. This allows us to easily maintain the program, and switch frontend if necessary.
After a long time planning, designing, researching, and developing the backend engine, the building blocks for the project are now beginning to fall in place. The next steps will be to finalize the design of single bones; once these are finalized, we can start building the rest of the musculoskeletal system.
As a brief overview of the future timeline and questions to be solved, here we show some future features that will be included in the application.
- Different bone types and variants
- Muscle attachments that deform the bone
- Basic physical representation of muscles
- Skin tissue
Together, these features will form the main components that will allow us to focus on the fun stuff, namely modeling a complete body in reasonable detail, and editing it into different shapes and species! Additionally, once these features are implemented, we can look deeper into structures such as the vascular system, the nerve system, and other tissue types. All of these components will reuse mechanisms of the core system we are currently developing, so whatever time we spend designing the system now will pay off once we get there.
We’re currently working towards a first demo that demonstrates the program. We’ll keep you posted on new features that enter the system as they come! And of course, in case you have any questions, feel free to reach out to us. Stay tuned!
The Project Team
Freedom of Form Foundation
501(c)(3) EIN 82-4415111