Recently I pointed out some differences in the handling of the “everything is content” paradigma of Communique. A few days ago I found a posting of David Nüscheler over at dev.day.com, in which he explained the details of performance tuning.
(His 5 rules do not apply exclusively to Day Communique, but to every performance tuning session).
In Rule 2 he states:
Try to implement an agile validation process in the optimization phase rather than a heavy-weight full blow testing after each iteration. This largely means that the developer implementing the optimization has a quick way to tell if the optimization actually helped reach the goal.
In my experience this isn’t viable in many cases. Of course the developer can check quickly, if his new algorithm performs better ( = is faster) than the older one. But in many cases the developer doesn’t have all the ressources and infrastructure available and doesn’t have all the content in his test system; which is the central point why I do not trust tests performed on developer systems. So the project team relies on central environments which are built for load testing, which have loadbalancers, access to directories, production-ready sized machines and content which is comparable to the production system. Once the code is deployed, you can do loadtesting. Either using commercial software or just using something like jmeter. If you can use Continious integration and and a autodeployment system, you can do such tests every day.
Ok, where have I started? Right, “everything is content”. So you run your loadtest. You create handles, modify them, activiate and drop them, you just request a page to view, you perform acitivies in your site, and so on. Afterwards you look at your results and hopefully they are better than before. Ok. But …
But Communique is not built to forget data — of course it does sometimes, but that’s not the point here :-), so all these activities are stored. Just take a look at the default.map, the zombie.map, the cmgr.hist file, … So all your recent actions are persisted and CQ knows of them.
Of course handling more of such information doesn’t make CQ faster. In case you have long periods of time between your template updates: Check the performance data directly after a template update and compare them to the ones a few months after (assuming you don’t have CQ instances which aren’t used at all). You will see a decrease in performance, it can be small and nearly unmeasurable, but it is there. Some actions are slower.
Ok, back to our loadtest. If you run the loadtest again and again and again, a lot of actions are persisted. When you reproduce the code and the settings of the very first loadtest and run the very first loadtest again, and you do that on a system which already faced 100 loadtests, you will see a difference. The result of this 101st loadtest is different from the first one,although the code, the settings and the loadtest are essentially the same. All the same except CQ and its memory (default.map and friends).
So, you need a mechanism which allows you to undo all changes made by such a loadtest. Only then you can perfectly reproduce every loadtest and run them 10 times without any difference in the results. I’ll try to cover such methods (there are some of them, but not all equally suitable) in some of the next posts.
And to get back to the title: Everything is content, even the history. So in contrary to my older posting,where I said:
Older versions of a handle are not content.
They are indeed content, but only when it comes to slowing down the system 🙂
Things on a content management system 
Agreed: during normal usage CQ instances tend to swell and it must affect load testing in particular. I’ve got a simpler solution though: disable versioning. You can do it by editing /config/repository/repository.xml file (the element). Additionally, it should take one element out of the equation and sharpen your results (versioning is time-consuming).
The other thing is that load tests must be executed in exactly the same environment every time. I suggest creating a backup of prepared installation (archive / copy of installation directory) and then restoring it before every test. Only this way reproducible results can be ensured.
Cheers,
Jan
Jan, thank you for the comment; I will do a quick answer here and try to cover the topics later. Yes, doing a backup before the loadtests and restore it before every loadtest seems to be the easiest solution to the problem. But I disagree regarding the versioning. Simply because I want to have the configurations identical in my loadtest environment and the production system.
Good point – I shouldn’t just scratch the versioning. In fact, I just realised that load tests should be run in pairs: with and without versioning – to show how it affects given application. It would make it possible to suggest e.g. the optimal version limit or version purging strategy. Thanks for that!