Automation done right — a reliable process

(the subtitle could be: Why I don’t call simple curl scripts „automation“).

Jakub Wadalowski mentioned in his talk on CircuitConf 2016, that „Jörg doesn’t like curl“. Which is not totally true. I love curl as a command line http client (to avoid netcat).

But I don’t like the way how people are using curl in scripts and then call these scripts „automation“; I already blogged about this topic.

Good AEM automation requires much more than just package installation. True automation has to work in a way, that you don’t need to perform any manual intervention, supervision or control. The automation has to work in a reliable and repeatable fashion.
But we also know, that there will ever be errors and problems, which prevent a successful result. So a package installation can work, but the bundles might come not up. In such cases the system has to actively report problems back. But this has to be accurate in a way, that you can rely on it: If no messaging occurs, everything is fine! There are no manual checks anymore!
This really requires a process you can trust. You need to trust your automated process so much, that you can start it and then go home.

So to come back to the „Jörg doesn’t like curl“ statement of Jakub: When you just have the curl calls and no error checking (I am not aware of an easy way to determine the status code of the HTTP request done with curl) and no proper error handling and reporting, it will never be reliable. It might save you a few clicks, but in the end you still have to perform a lot of manual steps. And to get away from these manual steps just with shell scripting, it requires a good amount of scripting.

And to be honest: I’ve rarely seen „good shell scripts“.

One thought on “Automation done right — a reliable process

  1. I’ve never thought I’d end up being a blog post hero, but apparently it just happened 🙂

    Neither curl by itself nor shell scripts that wrap it allow you to handle complex HTTP flows that include multiple requests and status/response body validation. It’s of course possible, but quickly becomes cumbersome. Everyone that has ever tried to write something more advanced in pure bash exactly knows what I mean. I fully agree with Jorg’s opinion about curl.

    That was one of the reasons I use just Ruby in my Chef cookbook: cq_package resource is a great example that eliminates all the flaws simple curl execution has. It even goes beyond simple validation of Package Manager HTTP responses and makes sure the entire post processing, that may get triggered sometimes (due to new OSGi bundles or configs), is over before next item gets processed.

    All in all, curl is great, but people need to be aware of its limitations. 10-line bash script that assumes happy path scenario may not be the best option for AEM deployment. And a bunch of sleep executions won’t make it any better.

Comments are closed.