Limits of dispatcher caching with AEM as a Cloud Service

In the last blog post I proposed 5 rules for Caching with AEM, how you should design your caching strategy. Today I want to show another aspect of rule 1: Prefer caching at the CDN over caching at the dispatcher.

I already explained that the CDN is always located closer to the consumer, so the latency is lower and the experience will be better. But when we limit the scope to AEM as a Cloud Service, the situation gets a bit complicated, because the dispatcher is not able to cache files for more than 24 hours.

This is caused by a few architectural decisions done for AEM as a Cloud Service:

These 2 decisions lead to the fact, that no dispatcher cache can hold files fore more than 24 hours because the instance is terminated after that time. And there are other situations where the publishs are to be re-created, for example during deployments and up/down-scaling situations, and then the cache does not contain files for 24 hours, but maybe just 3 hours.

This naturally can limit the cache-hit ratio in cases where you have content which is requested frequently but is not changed in days/weeks or even months. In an AEM as a Cloud Service setup these files are then rendered once per day (or more often, see above) per publish/dispatcher, while in other setups (for example AMS on on-prem setups where long-living dispatcher caches are pretty much default) it can delivered from the dispatcher cache without the need to re-render it every day.

The CDN does not have this limitation. It can hold for days and weeks and deliver them, if the TTL settings allow this. But as you can control the CDN only via TTL, you have to make a tradeoff between cache-hit ratio on the CDN and the accuracy of the delivered content regarding a potential change.

That means:

  • If you have files which do not change you just set a large TTL to them and then let the CDN handle them. A good example are clientlibs (JS and CSS files), because they have a unique name (an additional selector which is created as a hash over the content of the file.
  • If there’s a chance that you make changes to such content (mostly pages), you should set a reasonable TTL (and of course “stale-while-revalidate”) and accept that your publishs need to re-render these pages when the time has passed.

That’s a bit a drawback of the AEM as a Cloud Service setup, but on the hand side your dispatcher caches are regularly cleared.

Writing integration tests for AEM (part 3)

This a part of my ongoing series about writing integration tests with AEM.

In the last post on writing integration tests with AEM I quickly walked you through a simple test case for authoring instances, but I didn’t provide much context, what is going on exactly, and how it will be executed in Cloud Manager. That’s what I want to talk about today.

As we have seen, some relevant parameters for integration tests are provided are provided externally, most notable the URLs for the environment plus credentials.

In the pom.xml it looks like this:

Here you can see defaults, but you can simply override them by providing the exact values with the command line, as you already did in the previous post with overriding the URL of the authoring instance. The POM just introduces another indirection via properties which is technically not really necessary.

CloudManager works the same way: It invokes the maven-failsafe-plugin to execute the integration tests and provides overrides these default values with the correct data specific for that environment (including the admin credentials).

In detail, the urls are configured like this:

This means that your tests acess the loadbalanced author cluster and the loadbalanced publish farm (including dispatcher!).

This has 2 implications:

  • On your local installation you should have as well a dispatcher configured in front of the publish instance to have an identical setup
  • You can use integration tests also to validate your publish dispatcher rules!

And armed with this knowledge I will show you in the next post how you can validate with integration tests, that your domain setup is configured correctly.