Sidehistorik
| Indholdsfortegnelse |
|---|
Purpose
To make good Changes and SOP's (Standard Operating Procedures) - its similar to beeing being a chef that need to learn how to make a good dish, or a marine that need to perform a certain function really good, as in: repetition, repetition....
In short, the goal is to end up with a solid recipe, to make the same "good meal" every time - That the Cookbook principle for me
...
The environments from above is not the only ones, in many (large) setups we can have several others:
| System | Formål | Hvem kan deploye her | Purpose | Who can deploy here? | The good and bad stuffGode/dårlige ting | |||
|---|---|---|---|---|---|---|---|---|
| UAT | User Acceptance Test | Sysadmin (Evt. Developer ved DevOps)And/or Developer if DevOps is used) | A known/controlled environment - few and controlled changes that has be have been performed in Stage. | |||||
| integration | Special integrations tests, typically with 3rd party systems outside our control. Import/Export/Sync | Sysadmin (And/or Developer if DevOps is used) | A known/controlled environment - few and controlled changes that has be have been performed in Stage. | Kendt miljø under Change kontrol. Få eller ingen ændringer og skal til enhver tid matche Stage miljøet | integration | Test omkring integrationer med andre systemer, f.eks import/export/synkronisering | Sysadmin (Evt. Developer ved DevOps) | Kendt miljø under Change kontrol. Få eller ingen ændringer og skal til enhver tid matche Stage miljøet |
DevOps
DevOps og Continues Delivery koncepterne bryder lidt med den traditionelle Udvikler and Continues Delivery concepts extending the traditional concepts of the Developer vs. Sysadmin tanke, idet at man ofte antager at Sysadmins blot kan deploye, mens udviklere ikke kan arbejde (overhovedet) i Stage eller Produktionsmiljøer.
Dette er ikke nødvendigvis koliderende tankegange, men man bør overvejer og implementere lidt anderledes regler og godkendelsesprocedurer, da jeg (belært af små 10 år på Sysadmin siden) må erkende:
| Info |
|---|
Udvikleres synsvinkel på klassiske dyder og procedurer er som regel koncentreret om helt andre aspekter end dem som en Sysadmin eller driftsansvarlig person fokuserer på - for udvikleren har typisk et fokus på at komme hurtigt i mål med nye features og/eller deployments, - somme tider på bekostning af tests og datadiciplin. Giver man udvikleren - eller tvinger han sig adgang - til Produktionsmiljøet kan man som Sysadmin godt anse slaget for tabt. |
Ved DevOps eller Continues Delivery bør man evt implementere regler, procedurer eller mere tekniske funderede adgange som godkendelser eller workflows hvor deployment afhænger af success tests i de miljøer der er før det ønskede miljø for deployment, for at forhindre utilsigtet eller forhastet deployment fra udviklerens side.
| Advarsel |
|---|
Der er også datasikker- og datafortrolighed at tænke over, når udviklere gives nogen som helst form for adgang til Stage eller Prod, som ofte indeholder personhenførbar eller fortrolig data, både fordi at antallet af personer med tilgang simpelthen udviddes, sekundært fordi udvikleren [potentielt] har mulighed for at udnytte softwaren til at udtrække fortroligt data udenom normale foranstaltninger.
Sekundært er der IT Revisionsmæssigt gode grunde til at funktionsadskille. |
Miljøer
approach, where a common sence is that Sysadmins deploys, while Developers does not interact with Stage- or Production-environments.
These are not necessarily colliding ways of thinking, but requires some thinking and implementation that enforces special rules and approval-procedures, since (learned over 10 years as Sysadmin):
| Info |
|---|
The Developers Point Of View on classic virtues and procedures are typically focused in a complete other direction than those of the Sysadmin - The Developer has a focus on getting quickly to the next finish line with features and deployment - sometimes the cost are giving a little slack on test and deployment diciplin. Giving the Developer access to Deployment in the Stage or Production without enforcing special rules and approval-procedures, do consider the battle lost |
During DevOps or Continues Delivery You should implement rules, procedures and technically based access/approval workflows where deployment depends on successfully deployment/test in the previous environments, to prevent unintended or hasty deployment from the Developers side.
| Advarsel |
|---|
Do also give data confidentiality and data security a thought, when Developers are given any form of access to Stage or Prod, as these environments often contains confidential or classified content; just by giving Developers access, the numbers of people with possible access rises, secondly because the Developer [potentially] has the possibility to use the software to extract confidential or classified informations more directly. One of the developers common reasons to require Stage or Production access is the need for debug or log informations, here I do recommend using a facility like splunk for collecting data (in a controlled manner) off the servers/systems. Also, separation of duties are often a requirement for IT-Revision and data security/control |
Environments
The minimum should beVores absolutte minimum må være:
| Gliffy Diagram | ||||
|---|---|---|---|---|
|
Endnu eller endnu bedreEven better:
| Gliffy Diagram | ||||
|---|---|---|---|---|
|
Oprettelse af opskrift og gennemførsel
Creating the recipe and doing deployment
Seeing beside the number og levels of environments on our platform, we do have the oppertunity to improve our recipe through the environments from Development to Production, and with todays techonologies for virtualizing like VMware or snapshotting filesystems (like ZFS) within a short timeframe - we can repeat and improve the recipe several times, even within each environmentLigemeget hvor mange systemer eller rettere niveauer der er på vores platform, har vi mulighed for at forbedre vores opskrift op gennem de forskellige niveauer fra Development til Production, og med dagens teknologier indenfor virtualisering hvor vi kan rulle en server eller et system tilbage til et tidligere state på meget kort tid (via snapshot teknologi) - kan man indenfor de forskellige systemer sagtens gentage processen flere gange:
| Gliffy Diagram | ||||
|---|---|---|---|---|
|
Fra opskrift til SOP
En opskrift i kogebogen er ofte kun fremadrettet, dvs. tanken om at gå tilbage er ikke altid med i opskriften, ofte har vi ved fejl bare rullet tilbage til et snapshot og startet forfra.
Men der er mange situationer, hvor denne rollback metode ikke er mulig eller ønskelig, specielt i systemer hvor nedetiden skal være minimal, eller der er integration til andre systemer, der ikke tillader rollback, f.eks kunne et fortløbende ID smadres.
Samtidig har opskriften måske mange objecter med navne og bestemte ting for netop denne opskrift.
Det kan derfor være ønskeligt at "rense" opskriften for disse ting og lave en SOP - hvor alle navngivninger er ændret i nomenklateren til placeholders for objekter etc. Det er så også ønskeligt at der med eller i SOP'en følger en forklaring og/eller liste af hvad der skal i disse placeholders når SOP'en bruges.
Det sidste skridt der derfor at SOP'en laves:
From recipe to SOP
A recipe in typically going forward from A to B, it does not (always) imply the thoughts of getting back to the base, besides rolling back to a snapshot.
In many situations this type of rollback in not possible, allowed or comply with real-life (prod) - this can be for systems required to have low or zero downtime, or to maintain 100% audit or date/time/id integrity - for example when interacting with 3rd party systems.
At the same time, the recipe is typically very specific regarding objects and entities, with actual host and database names in it.
Hence, we can have a wish for cleaning the recipe and making a SOP - where all specific names and references in the nomenclature are becoming placeholders instead. The SOP then needs to have a section explainng what the placeholders should contain (Pre-requisites).
So final step is to make the SOP:
| Gliffy Diagram | ||||
|---|---|---|---|---|
|
Divide and conquer
The Divide and conquer principle is crucial for good Changes and SOPs - as opposite to havin a big Change with many steps and types of implementation (Application, Firewall, Database, OS), You should divide the Change into smaller Changes that are independant and/or sequentiel to be done separately. This also means that a rollback of a Change is not highly critical for the complete scope of the Changes.Gliffy Diagram