Purpose
To make good Changes and SOP's (Standard Operating Procedures) - its similar to being a chef that need to learn how to make a good dish, or a marine that need to perform a certain function really good, as in: repetition, repetition....
In short, the goal is to end up with a solid recipe, to make the same "good meal" every time - That the Cookbook principle for me
First, lets have a look at the environments we (commonly) deal with in an every days Service Operation environment:
| System | Purpose | Who can deploy here? | The good and bad stuff |
|---|---|---|---|
| Development | Basic or custom development, this can be on the developers own systems or a development setup | Developer | A voliatile environment with a potentionally high number of changes and no control. Very flexible and agile for high productivity |
| Test | Testing what has been released via development | Developer | A voliatile environment with a potentionally high number of changes |
| Stage | Deployment and testing of the release after test in Test | Sysadmin (And/or Developer if DevOps is used) | A known/controlled environment |
| Production | Sysadmin (And/or Developer if DevOps is used) | A known/controlled environment - few and controlled changes that has be have been performed in Stage. |
The environments from above is not the only ones, in many (large) setups we can have several others:
| System | Purpose | Who can deploy here? | The good and bad stuff |
|---|---|---|---|
| UAT | User Acceptance Test | Sysadmin (And/or Developer if DevOps is used) | A known/controlled environment - few and controlled changes that has be have been performed in Stage. |
| integration | Special integrations tests, typically with 3rd party systems outside our control. Import/Export/Sync | Sysadmin (And/or Developer if DevOps is used) | A known/controlled environment - few and controlled changes that has be have been performed in Stage. |
DevOps
DevOps and Continues Delivery concepts extending the traditional concepts of the traditional Developer vs. Sysadmin thinking, where a common stage is that Sysadmins deploys, while Developers does not interact with Stage- or Production-environments.
These are not nesesarily colliding ways of thinking, but requires some thinking and implementation that enforces special rules and approval-procedures, since (learned over 10 years as Sysadmin):
The Developers Point Of View on classic vertues and procedures are typically focused in a complete other direction than those of the Sysadmin - The Developer has a focus on getting quickly to the next finish line with features and deployment - sometimes the cost are giving a little slack on test and deployment diciplin.
Giving the Developer access to Deployment in the Stage or Production without enforcing special rules and approval-procedures, do consider the battle lost
During DevOps or Continues Delivery one should implement rules, procedures and technically based access/approval workflows where deployment depends on successfully in the previous environments, to prevent unintended or hasty deployment from the Developers side.
Do also give data confidentiallity and data security a thought, when Developers are given any form of access to Stage or Prod, as these environments often contains confidential or classified content; just by giving Developers access, the numbers of people with possible access rises, secondly because the Developer [potentially] has the possibility to use the software to extract confidential or classified informations more directly.
One of the developers common reasons to require Stage or Production access is the need for debug or log informations, here I do recommend using a facility like splunk for collecting data (in a controlled manner) off the servers/systems.
Also, seperation of dutys are often a requirement for IT-Revision
Environments
The minimum should be:
Even better:
Creating the recipe and doing deployment
Seeing beside the number og levels of environments on our platform, we do have the oppertunity to improve our recipe through the environments from Development to Production, and with todays techonologies for virtualizing like VMware or snapshotting filesystems (like ZFS) within a short timeframe - we can repeat and improve the recipe several times, even within each environment:
Fra opskrift til SOP
En opskrift i kogebogen er ofte kun fremadrettet, dvs. tanken om at gå tilbage er ikke altid med i opskriften, ofte har vi ved fejl bare rullet tilbage til et snapshot og startet forfra.
Men der er mange situationer, hvor denne rollback metode ikke er mulig eller ønskelig, specielt i systemer hvor nedetiden skal være minimal, eller der er integration til andre systemer, der ikke tillader rollback, f.eks kunne et fortløbende ID smadres.
Samtidig har opskriften måske mange objecter med navne og bestemte ting for netop denne opskrift.
Det kan derfor være ønskeligt at "rense" opskriften for disse ting og lave en SOP - hvor alle navngivninger er ændret i nomenklateren til placeholders for objekter etc. Det er så også ønskeligt at der med eller i SOP'en følger en forklaring og/eller liste af hvad der skal i disse placeholders når SOP'en bruges.
Det sidste skridt der derfor at SOP'en laves:
