Infrastructure sizing for projects

Sizing the infrastructure is always important when you start new IT projects, which require their own set of systems, processes and information. When you start a CQ5 project from scratch, you will have a discussion about CPUs, RAM, storage, network infrastructure and so on. That’s a common pattern and not specific at all to CQ5 projects.

In many cases you do an upfront calculation how the situation will be in 3 years (that’s a common interval for hardware replacement), and then you go shopping for hardware. In cases, where you can qualify the needs quite well, this approach is a good way to go. Because you have the discussion only once and you can ask for the adequate infrastructure budget when you do the budget discussion for the project. Though it’s a pretty bad situation, when you need to apply for a new hardware budget after 18 months. Because then you admit, that the initial calculation was wrong and you did a bad job at project start. At least, that’s often the way how people (especially the controllers) see this.

But assumptions, that you can do all the math upfront, are not valid for every project. In fact in many projects you won’t know how the situation will be in 24 or 36 months. If the project is successfull or if it has been cancelled in the meantime (for whatever reason). Sometimes external factors influence your systems heavily. Or the requirements change drastically. For example the rise of Facebook or Twitter often reduced the requirements for dedicated community areas on the company’s website. Or it just turned out, that despite Facebook and Twitter your user ask for a community site for your products.

So, in these cases any initial calculation will be wrong. But you don’t know that when you do your initial budget and your initial system sizing. The only thing you know is: “In 6 months I understand much better the requirements for system performance and how my systems should look like“.Because after these 6 months, you learned a lot about the requirements, the project and also about its rise or fall. You don’t need to assume and guess any more, you know and you calculate. Any sizing you do at this point in time is much more accurate, because it is based on actual numbers. It fits better to the problems, because you already know some bottlenecks and you can react on them now. And in the end it is cheaper because you have only the hardware you need right now; you haven’t spend your dollars and Euros on hardware you will need in 12 months (or not at all …), because you buy that in 12 months.

So instead of planning ahead for 36 months you only plan for 3 to 6 months, and recheck after that time, if all assumptions still hold true and how the current systems are performing. If everything’s fine: You have done a great job! If the situation changed and you need to react on it: That’s ok, that’s what this model is for.

For me this looks pretty much like the “waterfall versus agile” discussion in the software development area. In the software development the benefits of being agile are often obvious, but in many projects the hardware needs to be allocated like in a traditional waterfall process. You do a sizing and you will never change it. And based on this sizing the hardware is provided.

Some reasons why this model of constant review of the system sizing is not taken:

• When the upfront design and implementation phase are over, the architects and system designer will leave the project. The project is in regular production and you will have just minor changes and maintenance. In this phases you often have people, which are less skilled for this topic. And there’s no budget to allocate an architect resource for a regular session.
• When the need for a change to the system sizing is detected and framed out, there is no budget available. Changes like this have never been planned.
• Any reaction to new or changed requirements have not planned also for a changed system sizing as a consequence of these changed requirements.

Some reasons why you should switch to a model of constant review of the system sizing:

•   Reduce the risk of planning wrong, be it too large or to small.
•   Reduce the risk of requirements changing your basic assumptions for system sizing fundamentally

For sure this approach is something you cannot do for your project but it needs support from management, because it heavily influences budgets and budget discussions. So here are some ideas how you can get start to get there:

• Clearly communicate any risk of the system sizing: unclear and changing requirements, wrong assumptions about external factors influencing this sizing calculation. Or just the uncertainty which any change to a yet unknown technology comes with.
• Offer a number of variants: A single sizing which might be too large and expensive (to accommodate for some risks) vs sizing which can be changed and is much more accurate for the situation. Outline the differences in the price tags.
• Start the project with a number of proof of concepts and try to explore the requirements which are considered the ones with the largest impact on infrastructure sizing.