Question about immutable servers and environment/node specific configuration

Let's start here:

https://www.chef.io/blog/2014/06/23/immutable-infrastructure-practical-or-not/

Personally, I hate the "immutable infrastructure" fad that has been sweeping technology lately. And for exactly the reasons Julian points out in that article, it flies in the face of DevOps culture and creates more problems than it solves. For instance, we have a certain vendor who used to provide physical hardware appliances. In the early days of my industry, that was how things were done, call the vendor, get a black box, hook it up in your DC, when things go wrong, call the vendor. We also didn't require five nines... having a black box increases troubleshooting times greatly, especially if your vendor isn't open 24x7.

Well, we're in the 21st century now, we have extremely low SLAs, and need to be able to debug our own infrastructure. But this same vendor is now providing docker images. That's great that the image can run on any hardware platform (including virtual hardware), but what happens when something goes wrong? Sure, we can destroy the container and redeploy, but we keep running into this same bug. Redeploying that container multiple times a day is not really an acceptable solution. But without access to the container, there's really nothing we can do except go back and forth with the vendor, who, believe it or not, can't reproduce the issue! If you're not logging the right things, no amount of logging is going to help either.

I realize this is all anecdotal so perhaps not what you were looking for, my point was to illustrate the problems we're currently experiencing with so called "immutable infrastructure".

My question to you is, what are you trying to accomplish or what problem are you trying to solve with immutable infrastructure? I would do an experiment. Find a semi-complex app, and build it with immutable infrastructure, then build it with standard DevOps principles. Which is easier to maintain? Break something. Which is easier to resolve the issue? Which is easier to scale?

In your infrastructure, you talk about per-environment configs. So app01 and app02 have config files that point them to eachother. What happens when you need to add an app03? You'll have to create a new image, right? Then destroy your existing deployment and redeploy all three servers? Wouldn't it be quicker just to deploy app03, run whatever Ansible playbooks you need to update the specific config settings, then restart the service?

Anyway, I had more but I have to run.

/r/devops Thread