The New Economies of Scale: Fundamentals are Back

Cloud computing has revolutionized information technology by making unlimited scalability available to every startup. With this power and simplicity, the temptation to skip traditional engineering fundamentals is strong. However, our decades of experience measuring and operationalizing technology throughout hundreds of organizations, indicates that there are several key steps that many firms bypass, increasing risks and costs.

Scalability is achieved in one of two ways: horizontal or vertical. Horizontal scalability is the ability to increase your compute resources by adding similar nodes or servers to a particular architecture tier, such as an application pool or database cluster. Vertical scalability is the ability to add more resources such as CPUs or RAM to individual servers. Horizontal scalability is essential to most businesses and the focus of this article.
​
The majority of technology company executives we encounter believe that they have successfully implemented horizontal scalability.  However, data from the Corsis Confidence Index (CCI), a proprietary technology benchmarking standard that measures confidence in a company’s IT operations, tells a different story.  Our assessment data shows that critical engineering control processes which make horizontal scalability efficient and reliable are frequently overlooked.

For this article’s analysis, we evaluated 65 technology-enabled companies in the healthcare, ecommerce, construction, data analytics and professional services industries with revenues ranging from $30 million to $100 million. We converted hundreds of data points, or TechIndicatorsTM, into a 1 to 5 scale, where one equates to the lowest level of confidence and five equals the greatest level of confidence.
Overall, companies evaluated in our study scored above average for their support of horizontal scalability (4.0). However, many of these high-achieving companies were missing the engineering controls to reap the full benefits. Scores were low for the following essential TechIndicators:
​

• System provisioning is automated (2.64). By not creating and configuring virtual machines automatically, companies are severely limiting scalability by putting people in the middle of the process. This limitation can directly impact user satisfaction by creating frustrating lag time before new nodes are added to the tier, as well as increasing costs by being slow to remove the nodes once the usage peak has passed.

• Performance testing benchmarks are defined (2.47). Without performance benchmarks, how do you know when more resources are needed? Clearly defined trends, such as leading indicators of a sustained demand that would warrant another horizontal node, are vital to making the most of powerful elastic computing. In order to optimize cost, the indicators must track both directions – adding as well as reducing resources. Typical patterns such as “Add a node when CPU usage exceeds 60% CPU usage for 20 seconds” can be starting points, however your application patterns will be unique. Time invested in understanding your formula can help optimize cost while meeting demand.

• Performance testing is conducted at appropriate intervals (2.57). As your application changes, your scalability triggers may change. Yet CCI data indicates many companies don’t have a process for when to conduct performance testing. Best practice is to automate some of this testing as part of the continuous integration and continuous delivery ecosystem, but at a minimum, Corsis recommends conducting performance testing as part of each major release.

• Critical databases are sufficiently redundant (3.33). The database tier is often the most difficult to scale horizontally. Traditional relational databases require savvy DBAs and often, expensive enterprise licensing. Cutting edge open source databases such as Apache Cassandra are built from the ground up to support horizontal scalability. These solutions often have significant fault tolerance advantages over their RDBMS counterparts such as improved Recovery Time Objective (RTO). While this score is higher than many in this section, it is surprising how many companies don’t have database redundancy or horizontal scalability.

• Application is architected with appropriate granularity (3.79). Best in class architectures identify performance or resource hotspots and segregate them to their own tier or virtual servers. Technology like elasticsearch, Amazon’s Lambda server-less mechanism, and specialty NoSQL databases such as Cassandra are examples where functionality with high swings in demand can be effectively scaled at optimal cost.
  
  

Cloud computing can be a great equalizer for your business. The features, flexibility and price points are unprecedented, with very low barriers to entry. However, without undertaking some traditional engineering steps, you can miss the target and create a more expensive and less flexible solution. Incorporate the items listed above and you can leverage this technology with confidence.