## Wrote a book Author of [Infrastructure as Code: Dynamic Systems for the Cloud Age](https://learning.oreilly.com/library/view/infrastructure-as-code/9781098114664) which is about [[Infrastructure as code]]. The foreword is by [[Martin Fowler]]. [Building Evolutionary Infrastructure • Kief Morris • GOTO 2019](https://www.youtube.com/watch?v=1f_P8ZH5BC8). ## Patterns of infrastructure as code > [!TIP] > Build small, independently releasable components. [Strive to build small](https://youtu.be/1f_P8ZH5BC8?t=1408), **independently releasable** components. They should play well with others. Small surface area. Intention revealing. Minimal dependencies. ## Patterns and Antipatterns for Structuring Stacks This is about the size and structure of the stack as a whole. - [[Monilithic stack pattern]] - A monolithic stack is an infrastructure stack that includes too many elements, making it difficult to maintain - [[Micro stack pattern]] - The micro stack pattern divides the infrastructure for a single **service** across multiple stacks - [[Application group stack pattern]] - An application group stack includes the infrastructure for multiple related applications or services. The infrastructure for all of these applications is provisioned and managed as a group - [[Service stack pattern]] - A service stack manages the infrastructure for each deployable application component in a separate infrastructure stack ## Patterns for Building Environments Environments, like production and testing. - [[Multiple-environment stack antipattern]] - A multiple-environment stack defines and manages the infrastructure for multiple environments as a single stack instance. - [[Copy-Paste Environments antipattern]] - The copy-paste environments antipattern uses a separate stack source code project for each infrastructure stack instance. They avoid the [[Blast radius]] problem of the multiheaded stack antipattern. You can also easily customize each stack instance. - [[Reusable stack pattern]] - A reusable stack is an infrastructure source code project that is used to create multiple instances of a stack - A stack is usually the highest-level component in an infrastructure system. It’s the largest unit that can be defined, provisioned, and changed independently. The reusable stack pattern encourages you to treat the stack as the main unit for sharing and reusing infrastructure. - Infrastructures composed of small stacks are more nimble than large stacks composed of modules and libraries. You can change a small stack more quickly, easily, and safely than a large stack. So this strategy supports the virtuous cycle of using speed of change to improve quality, and high quality to enable fast change. - Building a system from multiple stacks requires keeping each stack well-sized and well-designed, cohesive and loosely coupled. > [!QUOTE] > Reusable stacks should be the workhorse pattern for most teams who need to manage large infrastructures. A stack is a useful unit for testing and delivering changes. It provides assurance that each instance of an environment is defined and built consistently. The comprehensiveness of a stack, rather than modules, as a unit of change enhances the ability to easily deliver changes quickly and frequently. > [!TIP] > Divide infrastructure into multiple, independently changeable stacks. To have a [monolithic application cluster](https://youtu.be/1f_P8ZH5BC8?t=1899). ## Patterns for Configuring Stacks - Manual Stack Parameters antipattern - Stack Environment Variables pattern - Scripted Parameters pattern - Scripted parameters involves hardcoding the parameter values into a script that runs the stack tool. You can write a separate script for each environment or a single script that includes the values for all of your environments - Stack Configuration Files - Stack configuration files manage parameter values for each instance in a separate file, which you manage in version control with your stack code - This is what we usually do - Wrapper stack pattern - A wrapper stack uses an infrastructure stack project for each instance as a wrapper to import a stack code component. Each wrapper project defines the parameter values for one instance of the stack. It then imports a component shared by all of the stack instances - Pipeline stack parameters pattern - With the pipeline stack parameters pattern, you define values for each instance in the configuration of a delivery pipeline. This is dumb and [[GitHub Actions]] barely supports configuring anything - Stack parameter registry pattern - A stack parameter registry manages the parameter values for stack instances in a central location, rather than with your stack code. The stack tool retrieves the relevant values when it applies the stack code to a given instance ## About DRY If you ever want a different value for something hardcoded, you are screwed. > [!QUOTE] Duplication Considered Useful > The [[Don't repeat yourself]] principle discourages duplicating the implementation of a concept, which is not the same as duplicating literal lines of code. Having multiple components depend on shared code can create tight coupling, making it hard to change. > > I’ve seen teams insist on centralizing any code that looks similar; for example, having all virtual servers created using a single module. In practice, servers created for different purposes, such as application servers, web servers, and build servers, usually need to be defined differently. A module that needs to create all of these different types of servers can become overly complicated. > > When considering whether code is duplicated and should be centralized, consider whether the code truly represents the same concept. Does changing one instance of the code always mean the other instance should change as well? > > Also consider whether it’s a good idea to lock the two instances of code together **into the same change cycle**. Forcing every application server in the organization to upgrade at the same time may be unrealistic. ## Patterns for Stack Components - [[Facade module pattern]] - A _facade module_ creates a simplified interface to a resource from the stack tool language or the infrastructure platform. The module exposes a few parameters to the calling code - Like making a super simple interface to the EC2 instance module (just expose disk size for example) - It seems to me that we use the facade module pattern - Also known as the wrapper module - [terraform-aws-security-group](https://github.com/terraform-aws-modules/terraform-aws-security-group) is a facade module - [[Obfuscation module antipattern]] - An obfuscation module wraps the code for an infrastructure element defined by the stack language or infrastructure platform, but does not simplify it or add any particular value. In the worst cases, the module complicates the code. - [[Unshared module antipattern]] - An unshared module is only used once in a codebase, rather than being reused by multiple stacks. - People usually create unshared modules as a way to organize the code within a stack project. - When a stack project becomes too large, there are several alternatives to moving its code into modules. It’s often better to split the stack into multiple stacks, using an appropriate stack structural pattern - If the stack is fairly cohesive, you could instead simply organize the code into different files and, if necessary, different folders. Doing this can make the code easier to navigate and understand without the overhead of the other options. - The [[Rule of Three]] for software reuse suggests that you should turn something into a reusable component when you find three places that you need to use it. - [[Bundle module pattern]] - A bundle of related things - A _bundle module_ declares a collection of related infrastructure resources with a simplified interface. - [terraform-aws-atlantis](https://github.com/terraform-aws-modules/terraform-aws-atlantis) is a bundle module - [[Spaghetti module antipattern]] - A _spaghetti module_ is configurable to the point where it creates significantly different results depending on the parameters given to it. The implementation of the module is messy and difficult to understand, because it has too many moving parts - A module that does too many things is less maintainable than one with a tighter scope. The more things a module does, and the more variations there are in the infrastructure that it can create, the harder it is to change it without breaking something. - A spaghetti module’s code often contains conditionals that apply different specifications in different situations. - A spaghetti module is often an attempt at building an infrastructure domain entity using declarative code. - [[Infrastructure domain entity pattern]] - Is this it? Networking, storage and so on? - A bundle module is similar to a domain entity in that it creates a cohesive collection of infrastructure resources. But a bundle module normally creates a fairly static set of resources, without much variation. The mindset of a bundle module is usually bottom-up, starting with the infrastructure resources to create. A domain entity is a top-down approach, starting with what’s required for the use case. - Most spaghetti modules for infrastructure stacks are a result of pushing declarative code to implement dynamic logic. But sometimes an infrastructure domain entity becomes overly complicated. A domain entity with poor cohesion becomes a spaghetti module. The [[Rule of Three]]: - It is three times as difficult to build reusable components as single use components - A reusable component should be tried out in three different applications before it will be sufficiently general to accept into a reuse library ## Using Stacks as Components > A stack is usually the highest-level component in an infrastructure system. It’s the largest unit that can be defined, provisioned, and changed independently. The reusable stack pattern (see “Pattern: Reusable Stack”) encourages you to treat the stack as the main unit for sharing and reusing infrastructure. > > Infrastructures composed of small stacks are more nimble than large stacks composed of modules and libraries. You can change a small stack more quickly, easily, and safely than a large stack. So this strategy supports the virtuous cycle of using speed of change to improve quality, and high quality to enable fast change. > > Building a system from multiple stacks requires keeping each stack well-sized and well-designed, cohesive and loosely coupled. ## Discovering Dependencies Across Stacks - [[Resource matching pattern]] - A consumer stack uses resource matching to discover a dependency by looking for infrastructure resources that match names, tags, or other identifying characteristics. - Most stack languages have features to match other attributes than the resource name. Terraform has data sources and AWS CDK’s supports resource importing. You can create [data only modules](https://www.terraform.io/language/modules/develop/composition#data-only-modules) in Terraform too. - [[State data lookup pattern]] - State data lookup is a common pattern when using [[Terragrunt]] ([dependency outputs](https://terragrunt.gruntwork.io/docs/reference/config-blocks-and-attributes/#dependency)) - Also known as: remote statefile lookup, stack reference lookup, or stack resource lookup. - Stack data lookup finds provider resources using data structures maintained by the tool that manages the provider stack. - Many stack management tools maintain data structures for each stack instance, which include values exported by the stack code. Examples include Terraform and Pulumi remote state files. - Terraform stores output values in a remote state file. Pulumi also stores resource details in a state file that can be referenced in a consumer stack using a StackReference. CloudFormation can export and import stack output values across stacks, which AWS CDK can also access. - [[Integration registry lookup pattern]] - A consumer stack can use integration registry lookup to discover a resource published by a provider stack. Both stacks refer to a registry, using a known location to store and read values. ## Dependency injection > Dependency injection (DI) is a technique where a component receives its dependencies, rather than discovering them itself. > > An infrastructure stack project would declare the resources it depends on as parameters, the same as instance configuration parameters described in Chapter 7. > > A script or other tool that orchestrates the stack management tool (“Using Scripts to Wrap Infrastructure Tools”) would be responsible for discovering dependencies and passing them to the stack. [[Terragrunt]] works well for this. > > **So instead of doing [data lookups](https://www.terraform.io/language/data-sources) inside the stack, it happens on the outer edge through a wrapper script or similar**. This makes the code more general purpose and easier to test. This actually is kinda how it would work in Terragrunt because the DI is done from the outside of all other modules. > > At the very least, make the data sources optional with input variables. ## Issues with mixing dependency and definition code > Combining dependency discovery with stack definition code adds cognitive overhead to reading or working with the code. Although it doesn’t stop you from getting things done, the subtle friction adds up. > > You can remove the cognitive overhead by splitting the code into separate files in your stack project. But another issue with including discovery code in a stack definition project is that it couples the stack to the dependency mechanism. > > Coupling dependency management and definition can make it hard to create and test instances of the stack. > > Hardcoding specific assumptions about dependency discovery into stack code can make it less reusable. ## Using Infrastructure Code to Deploy Applications > Infrastructure code defines what goes onto servers. Deploying an application involves putting things onto servers. So it may seem sensible to write infrastructure code to automate an application’s deployment process. In practice, mixing the concerns of application deployment and infrastructure configuration becomes messy. The interface between application and infrastructure should be simple and clear.