Learning gut feeling in software design: The Conceptual Complexity Framework
This framework is a step-by-step process that has the purpose of improving the software design capabilities of the developer. Specially in the early years of development.

This framework is a step-by-step process that has the purpose of improving the software design capabilities of the developer. Specially in the early years of development.

When we are learning to code, we quickly learn some tools to organise our code like functions and classes. And just like that, we have to start deciding what to put where. And that happens to be difficult. Throw too many things into the same component and you create a mess. Split it a lot into shallow classes and you create a potentially even bigger one.

Is there a way to accelerate this learning individually? Can we have a set of very specific steps for early developers to compare two designs. If we had, they could exercise themselves in this process, quickly improving their criteria.

This is what this framework tries to be. It gives a series of very specific steps that allows them to evaluate the complexity of a system. This will allow them to compare the complexity between two designs, aiming for the simplest possible.

THE BASICS

Imagine a software system that is composed of multiple components (say classes, functions, etc.). We will define the conceptual complexity of the whole system as the sum of the complexity of each component.

code

system_complexity = unit_1_complexity + unit_2_complexity ...


Where the conceptual complexity of each component is the number of different concepts contained to the power of two:

code

unit_1_complexity = number_of_unique_concepts ^ 2

As you can see, mixing a lot of different concepts inside the same unit doesn't scale well, as it grows in a quadratically. This force will drive you to split the system in units that manage similar concepts together. And as you'll see during the exercise, splitting indefinitely into shallow classes won't work either.

IDENTIFYING CONCEPTS

Every single node of the abstract syntax tree can be introducing one several concepts that we should identify. See the following example in Ruby:

code

require 'net/http'
Net::HTTP.get('example.com', '/index.html')

In the Ruby script above, we can identify the following nodes in the tree:

  • The #require method.
  • The 'net/http' string.
  • The Net constant.
  • The HTTP constant.
  • The #get method.
  • The 'example.com' string.
  • The '/index.html' string.

Let's separate the code in lines so we can write down the concepts we identify next to the corresponding nodes:

code

require(            # requiring libraries
 'net/http'         # HTTP client, the HTTP protocol, the Net::HTTP library, the name of the Net::HTTP library is 'net/http'
)

Net::HTTP.          # the Net module, the Net::HTTP class, HTTP client, the HTTP protocol
 get(               # the Net::HTTP#get method, HTTP request, HTTP GET verb, HTTP client, the HTTP protocol
   'example.com',   # the host name of the example.com, the Example website, hostname
   '/index.html'    # the "index.html" page, path
 )


Crazy, right? That two-line ruby script above accumulates 19 different concepts (duplicates don't count). This makes a conceptual complexity of 19**2 = 361 for the whole script. Now think for a moment about that 1000-lines-of-code class that you have in that infamous project.

Let's take a minute to analyse the concepts I identified to illustrate process. This the most subjective part of this framework, and subsequently, the most difficult:

  • The require method introduces the concept of requiring libraries: This one is easy, you make use of require because you wanted to require a library.
  • The 'net/http' string introduces:
  • The Net::HTTP library itself.
  • The fact that the name of the Net::HTTP library is 'net/http': Seems redundant with the above but it's not. If the name of the library would change, you would have to change your code too.
  • The concept of an HTTP client: That's what Net::HTTP library is. You are requiring it because you want an HTTP client.
  • The concept of the HTTP protocol: By making your script aware of the concept of an HTTP client you are also making it aware of the HTTP protocol. No way out. Again, seems redundant but trust me, it's not. When refactoring you might manage to extract one without the other you need to count them separately.

This is the level of depth that you need in your analysis. That's why doing this exercise for a very small app can take a long time. But remember, we are exercising to change the way we think. It takes time and effort.

These are the things that most experienced developers have crystalized in them. In the same way that we do not think hard on the English grammar of every English sentence that we say. But when you are learning English, doing the exercise of understanding English grammar well allows you to learn English faster and speak English better. And eventually, it just flows.

Here are the I use to identify concepts:

  • What is this node coupled to? What can change so it breaks? This is an easy one. It's very explicit too.
  • Why is this called like that? Units of code are usually named after the main concepts they bring with them. That's why this question works.
  • Why is this node here? What happens if I remove it? This is one of the most useful ones. It will help you identify concepts that are part of the business logic of your application. They usually come in the form of a sentence and the more explicit they are in your code the better. Some examples could be the user must be validated before being able to sign in. Or the account is blocked after X invalid attempts of logging in.

THE EXERCISE

In order to crystalize the concepts above, I propose the following exercise:

  1. Write an application in one single function. Write code as imperative as possible, instruction after instruction. Something small, maybe around 50-100 lines of code I.e. given a user name retrieve his last 5 tweets from the Tweeter API. That will be your system, and it will originally consist in only one unit. Some integration tests will help during the refactor phase.
  2. List all of the concepts in your system and calculate it's Conceptual Complexity. Such small application should have a complexity in the order of thousands.
  3. Refactor it extracting the logic into smaller units, recalculating the Conceptual Complexity on each extraction. Note how extracting a very small number of concepts into a lot of units won't work. As every time you extract something to a unit, you are introducing at least one new concept, the new unit that the original unit would still likely mention.
  4. Repeat until you cannot reduce the complexity anymore.

As you'll see, it's a balancing act. And the framework is designed to help you finding this balance on your own, with numbers that makes it easy to compare. Hopefully this can have an impact in the way you see classes, functions and modules. And it helps helps you producing a pragmatic design.

NOTES

This is the Ruby code that I used to calculate the number of concepts of the properly tagged example:

code

puts ss.split("\n").select{|s| s =~ /#/ }.flat_map{|s| s.gsub(/^[^#]+# /,'').split(', ') }.compact.length

Manuel Morales headshot.

About me

I'm a fractional CTO that enjoys working with AI-related technologies. I have dedicated more than 15 years to serving SaaS companies. I  worked with small startups and international scale-ups in Europe, UK and USA, including renowned companies like Typeform.

I now work helping startups achieving high growth and performance through best practices and Generative AI.

l x i m