DevOps

• Continuous integration pipelines automate building applications:

  • The process starts with checking code into a repository where all the unit tests are run. On successful passing of the tests, a deployment package is built as a Docker image. This image is then saved in a container registry from where it can be deployed. Each microservice should have its own repository.
  • Typical extra steps include linting of code/quality analysis by tools such as SonarQube, integration tests, generating test reports, and image scanning.

  
• • The Cloud Source Repositories service provides private Git repositories hosted on Google Cloud. These repositories let you develop and deploy an app or service in a space that provides collaboration and version control for your code. Cloud Source Repositories is integrated with Google Cloud, so it provides a seamless developer experience.

  
• CI/CD Components:

  • Cloud Build executes your builds on Google Cloud infrastructure. It can import source code from Cloud Storage, Cloud Source Repositories, GitHub, or Bitbucket, execute a build to your specifications, and produce artifacts such as Docker containers or Java archives. Cloud Build executes your build as a series of build steps, where each build step is run in a Docker container. A build step can do anything that can be done from a container, irrespective of the environment. There are standard steps, or you can define your own steps.
• CI/CD Components:

  • A Cloud Build trigger automatically starts a build whenever you make any changes to your source code. You can configure the trigger to build your code on any changes to the source repository or only changes that match certain criteria.
• CI/CD Components:

  • Container Registry is a single place for your team to manage Docker images or deployment packages, perform vulnerability analysis, and decide who can access what with fine-grained access control.
• Cloud Source Repositories provides managed Git repositories:

  • Control access to your repos using IAM within your Google Cloud projects.
  • Cloud Source Repositories provides managed Git repositories. You can use IAM to add team members to your project and to grant them permissions to create, view, and update repositories. Repositories can be configured to publish messages to a specified Pub/Sub topic. Messages can be published when a user creates or deletes a repository or pushes a commit.
• Cloud Source Repositories

  • Some other features of Cloud Source Repositories include the ability to debug in production using Cloud Debugger, audit logging to provide insights into what actions were performed where and when, and direct deployment to App Engine. It is also possible to connect an existing GitHub or Bitbucket repository to Cloud Source Repositories. Connected repositories are synchronized with Cloud Source Repositories automatically.

  
• Cloud Build:

  • There is no more need to provision or maintain build environments: all is handled by Cloud Build. You write a build config to provide instructions to Cloud Build on what tasks to perform. These are defined as a series of steps. Each step is executed by a cloud builder. Cloud builders are containers with common languages and tools installed in them
  • Builders can be configured to fetch dependencies, run unit tests, static analyses and integration tests, and create artifacts with build tools such as docker, gradle, maven, bazel, and gulp.
• Cloud Build:

  • Cloud Build executes the build steps you define. Executing build steps is similar to executing commands in a script.
  • You can either use the build steps provided by Cloud Build and the Cloud Build community or write your own custom build steps.

  
• Build Triggers:

  • Build triggers watch a repository and build a container whenever code is pushed. Google’s build triggers support Maven, custom builds, and Docker.
  • A Cloud Build trigger automatically starts a build whenever a change is made to source code. It can be set to start a build on commits to a particular branch or on commits that contain a particular tag. You can specify a regular expression to match the branch or tag value.

  
• Build Triggers:

  • The build configuration can be specified either in a Dockerfile or a Cloud Build file. The configuration required is shown on this slide.
  • First, a source is selected. This can be Cloud Source Repositories, Github, or Bitbucket. In the next stage, a source repository is selected, followed by the trigger settings. The trigger settings include information like the branch or tag to use for trigger, and the build configuration, for example the Dockerfile or Cloud Build file.
• Container Registry

  • Provides a secure, private Docker image repository on Google Cloud. Image layer files and tags are stored in Cloud Storage. Cloud Storage IAM allows configuration of who can access, view, or download images. Container Registry provides native Docker support. This means you can push and pull Docker images to your private registry using the standard Docker command line interface. Multiple registries can be defined.

  
• Container Registry:

  • Container Analysis is a service that can also perform vulnerability scanning of images against a constantly updated database of known vulnerabilities. Container Analysis can be integrated with binary authorization to prevent the deployment of images with known security issues. Container Analysis must be explicitly enabled for a project
• Binary Authorization |

  • Allows you to enforce deployment of only trusted containers into GKE. Binary authorization is a Google Cloud service and is based on the Kritis specification.
  • For this to work, you must enable binary authorization on your GKE cluster where your deployment will be made. A policy is required to sign the images. When an image is built by Cloud Build, an attestor verifies that it was from a trusted repository; for example, Source Repositories. Container Registry includes a vulnerability scanner that scans containers. A typical workflow is shown in the diagram.

  
• Binary Authorization:

  • Checkin of code triggers a Cloud Build. As part of the build, Container Registry will perform a vulnerability scan when a new image is uploaded. The scanner publishes messages to Pub/Sub. The Kritis Signer listens to Pub/Sub notifications from a container registry vulnerability scanner and makes an attestation if the image scanning passed the vulnerability scan. Google Cloud binary authorization service then enforces the policy requiring attestations by the Kritis signer before a container image can be deployed.

  
• IaC:

  • In the cloud, all infrastructure needs to be disposable. The key to this is infrastructure as code (IaC), which allows for the provisioning, configuration, and deployment activities to be automated.
  • Having the process automated minimizes risks, eliminates manual mistakes, and supports repeatable deployments and scale and speed. Deploying one or one hundred machines is the same effort. The automation can be achieved using scripts or declarative tools such as Terraform which we will discuss later.
• IaC:

  • Terraform is one of the tools used for Infrastructure as Code or IaC
  • In essence, infrastructure as code allows for the quick provisioning and removing of infrastructures.
  • The on-demand provisioning of a deployment is extremely powerful. This can be integrated into a continuous integration pipeline that smoothes the path to continuous deployment.

  
• IaC:

  • Automated infrastructure provisioning means that the infrastructure can be provisioned on demand, and the deployment complexity is managed in code. This provides the flexibility to change infrastructure as requirements change. And all the changes are in one place. Infrastructure for environments such as development and test can now easily replicate production and can be deleted immediately when not in use. All because of infrastructure as code.
• IaC:

  • Several tools can be used for IaC. Google Cloud supports Terraform, where deployments are described in a file known as a configuration. This details all the resources that should be provisioned. Configurations can be modularized using templates, which allows the abstraction of resources into reusable components across deployments. In addition to Terraform, Google Cloud also provides support for other IaC tools, including:

  ● Chef ● Puppet ● Ansible ● Packe
• Terraform:

  • Terraform is an open source tool that lets you provision Google Cloud resources.
  • Terraform lets you provision Google Cloud resources—such as virtual machines, containers, storage, and networking—with declarative configuration files. You just specify all the resources needed for your application in a declarative format and deploy your configuration. HashiCorp Configuration Language (HCL) allows for concise descriptions of resources using blocks, arguments, and expressions.

  
• Terraform:

  • Deployments can be repeated over and over with consistent results, and you can delete an entire deployment with one command or click. The benefit of a declarative approach is that it allows you to specify what the configuration should be and let the system figure out the steps to take.
  • Instead of deploying each resource separately, you specify the set of resources that compose the application or service, which allows you to focus on the application. Unlike Cloud Shell, Terraform will deploy resources in parallel.
• Terraform:

  • Terraform uses the underlying APIs of each Google Cloud service to deploy your resources. This enables you to deploy almost everything we have seen so far, from instances, instance templates, and groups, to VPC networks, firewall rules, VPN tunnels, Cloud Routers, and load balancers.
• Terraform Language:

  • The Terraform language is the user interface to declare resources. Resources are infrastructure objects such as Compute Engine virtual machines, storage buckets, containers, or networks. A Terraform configuration is a complete document in the Terraform language that tells Terraform how to manage a given collection of infrastructure. A configuration can consist of multiple files and directories.

  
• The syntax of the Terraform language includes:

  ● Blocks that represent objects and can have zero or more labels. A block has a body that enables you to declare arguments and nested blocks.

  ● Arguments are used to assign a value to a name.

  ● An expression represents a value that can be assigned to an identifier.
• Terraform can be used on multiple public and private clouds:

  • Terraform is already installed in Cloud Shell. The example Terraform configuration file shown starts with a provider block that indicates that Google Cloud is the provider.
  • The region for the deployment is specified inside the provider block.
  • The resource block specifies a Google Cloud Compute Engine instance, or virtual machine. The details of the instance to be created are specified inside the resource block.

  
• Terraform:

  • The output block specifies an output variable for the Terraform module. In this case, a value will be assigned to the output variable "instance_ip."