Visual automation: how we create the newest feature in our 5-year history

Visual automation: how we create the newest feature in our 5-year history

Visual Automations (VA) is at the center of the user experience within ConvertKit. As an email marketing tool, our job is to help our creators (clients) send the right messages to the right audience at the right time, at the same time that the communication feels personal and personalized.

Visual automation is the biggest feature we have launched since ConvertKit was founded and will continue to drive our business (and our creators) over the next few years. Building something so big is an incomparable task, and as an engineer, I want to share what exactly is involved in a project like this.

For the context, this is what a visual automation looks like in ConvertKit:

As part of the team that worked to create VA, I will use this publication to explain how our application obtains the content of our server and creates a data structure that is useful for rendering a visual automation. In other words, I'll cover everything from when the data is received in the browser until we start displaying the graph on the screen.

Data that come in

One thing we decided from the beginning is that all communication would be done through JSON. After a few minutes of discussion, we thought JSON would look for something like this:

The final JSON was a bit more complex than that, but that's essentially what I was going to work with. I like to build features with small targets in sight. The ultimate goal is to represent a functional VA, but if you try to move from JSON to rendered graphics in your first step, it is very easy to feel overwhelmed.

One approach is to obtain validation early and often. Validations can come in different forms and there are no silver bullets. I knew from the previous discussion that VA could be super complex. In Ruby, we can manage this complexity with POROs (old ruby ?? objects) and I thought I could do the same with JavaScript. In this case, I would get the JSON, run it through an analyzer and, at the end of it, I would have something that I could use to help me represent the data on the screen.

The steps as I saw them were the following:

A linker that links nodes through edges.

A generator that would take the linker and build a useful data structure.

A walker that would make it possible to represent the graphic in some way.

These were the initial objectives. First, create the linker, then use the linker to create the Generator, then use the constructor to create the Walker. I decided on this approach from the beginning, not because it was the best solution, but because it allowed me to compartmentalize complexity.

He knew how to build the linker, but he had no idea how to build the other two components. I needed a small victory.

Step 1: Creating the linker

The linker seemed to be simple. It would return an array of N objects where the objects would be the leading edges of a graph. Then, if a graph has 3 entry points, the matrix will have a count of 3.

In the initial JSON, the borders only have identifiers of the nodes. The linker would replace those identifiers with Node and reuse the same object when a node has the same identifier.

It looks something like this:

Once the process method is called, a new sensor with a frozen matrix is ?? created. No, frozen arrangements are not bulletproof, but it makes sense to restrict things to force decoupling as much as possible. In general, the version with which we end up is quite similar to the one seen above. Objective reached!

With this first step, I realized that things would not be easy, after all. Multiple edges can start from the same node and also join the same node. I also started thinking about the broader concept of the graphic, the basic concepts we need when creating automatic visualizations on the screen.

The Builder class started to make more sense because now it had a purpose. His job would be to create a structure that would make it possible to find melting/branching points on the chart and help Walker determine which nodes to represent.

Step 2: Creating the Generator and solving the graph as a whole

The Constructor is the bridge between the linked data and the next Walker. In other words, the Builder is the glue that holds everything together. The goal is to provide information about the graph in a way that is actionable in the future to present the VA to the user.

Visual automations are represented row by row. When one side of the graph ends before the other, there is space allocated to allow the other side to finish. While it is easy for humans to visualize this, it is harder for a computer to realize that. Then, at some point, it would take some logic to figure out when to pause on one side and when not.

I had a heated debate in my head about where this should happen:

Should it be built natively in the graphic through the Constructor?

Should it be taken into account only when rendering?

It turns out that this is mostly subjective. It depends on how you approach the problem. I chose to continue with my strategy of creating small victories. If I were to incorporate the spacing in the graph natively, the Builder class would have to deal with much more logic and states. For that reason, I chose to find out what kind of information I would need to add the appropriate space.

Maybe organizing the data structure would make it easier to add spaces below?

While thinking about that question, I had the idea of ?? naming the substructure of the graph with different terms in order to think more clearly about what I wanted to do.

I drew a graph on a piece of paper:

Looking at that drawing, I realized that I could divide the graph into 3 parts:

Individual nodes

Branches that group linear nodes.

Lines, which are unique routes from the top of the graph to the bottom.

The way it works is that branches are created from nodes and lines are created from branches. No branch can have the same set of nodes, no line can have the same set of branches. That means that no line can have exactly the same set of nodes.

To make this easier, here is another image of a simpler graphics.

This graph has 7 nodes, 5 branches, and 4 lines. Here is the breakdown of the branches in this chart:

1-3

2-3

3-4-5

5-6

5-7

And the lines are the ordered combinations of those branches, where the branch 3-4-5 is present in all the lines.

With that information, you could write the Builder class with one goal in mind: move from a list of linked nodes to an array of line objects. It turns out that building that process was complicated, even after reducing the set of features for that particular class. It was a transverse route of the edge, branches were cut as branch points were found, new branches were created when melting points appeared and much more.

While writing this logic, many search / cut / add / delete operations emerged. Although I did not create the concept of Rama with this in mind, it was a perfect home for them.

While my focus was on building the Builder class, this kind of branch took most of my time. Each time I needed to perform an operation on the chart, the Rama object had all the tools to do the job.

The Builder class was going through nodes, observing the current state of the graph and discovering if a node should be added to a branch, removed from an existing branch, etc.

Once that was determined, the mission of doing the actual work was delegated to the object of Rama. Because he was using the same object, any operation on a branch on one line would automatically occur on all other lines.

By the time the Builder class was completed and I was returning the Line array, I had all the things I needed to start building the Walker. Soon it would be possible to render a VA on the screen!

Step 3: Put all the things together with the walker

I mentioned earlier that our visual automation tool has unique characteristics. One of those characteristics is that we represent the graph row by row. Sometimes that means you need some space to let another line end. The Walker is where the magic happens.

The idea is to start at the index = 0 and traverse the graph row by row to the end of the graph. In each row, the walker can return a series of nodes and spacers (we call them placeholders) in the correct order. This allows the rendering class to pick up those objects and display them correctly on the screen.

At this point, the original JSON underwent a bit of transformation and we added a lot of information to the data through objects, compositions, etc. The idea is to take advantage of all this to discover when to pause the lines.

Returning to the Line object, one of the rules is that there must be a single route from the beginning of a graph to its end. This means that if there are two Line elements with the same node that are not in the same index in their internal matrix, the Shortest line must be paused until it reaches the same index so that it can be merged in the same row.

And that's how the VA decides when to pause the lines. Ensures that all line elements that include a particular node are in the same row before representing it. Those that have more nodes still represent nodes, others simply represent placeholders.

This is how we solve it in the code:

Obtain all the positions for a given node, take the maximum number, if that number is larger than the current index, this line must be paused. When it reaches equality, the walker returns the paused Node.

Now that all the blocks are working as planned, it is now possible to render a graphic using the Walker through a {} while;.

Here is a fragment of our production code that shows the graphic on the screen:

Voila, a visual automation

These small steps are all (hehe) that are required at the end to represent something like this:

Following the strategy of dividing the big job into smaller parts and making sure to get small profits along the way helped me enormously. By adopting this approach, I was able to find places to decouple the code between classes and use the composition to reach a final state.

That is a glimpse behind the curtain of how we approach the construction of Visual Automation, our largest release of functions in our history (to date). Have questions? Comment online and I will return to answer them.