Como trabajar con DevOps en AWS

02/11/2024 · by · in AWS, Cloud, Git, Sistemas · Leave a comment

Esto es un resumen de recursos que podemos encontrar, sobre todo, en la web de AWS sobre DevOps.

La información la he dividido en:

  1. Cultura, Procedimientos y Procesos
  2. Herramientas y configuraciones
  3. Otros Links de interés

Básicamente esta organización es básica aunque posteriormente se podrá ampliar internamente con temas como integración continua o similares.

Introducción

Para empezar nuestro viaje por DevOps en AWS tenemos su Web DevOps y AWS donde se puede encontrar toda la información o links a otros recursos sobre el tema.

Posteriormente a ello nos encontramos con los siguientes documentos fundamentales:

  1. Introduction to DevOps on AWS: Whitepaper base donde se puede encontrar toda la información relativa a esta entrada. Tanto a herramientas, procesos, infraestructuras, monitorización o seguridad.
  2. DevOps Guidance: Mucho mas teórico que el documento anterior, definir los procedimientos y puntos fundamentales que forman parte del DevOps en AWS.
  3. Practicing Continuous Integration and Continuous Delivery on AWS: This paper explains the features and benefits of using continuous integration and continuous delivery (CI/CD) along with Amazon Web Services (AWS) tooling in your software development environment. Continuous integration and continuous delivery are best practices and a vital part of a DevOps initiative.

A parte de este tipo de links tenemos otros mas genéricos en AWS como el indicado como: What is DevOps?, donde se hace un análisis a alto nivel sobre el tema. Donde nos ayuda a entender como DevOps define su modelo, como trabaja o sus beneficios.

Para ampliar el conocimiento y datos sobre temas de DevOps, te recomiendo buscar mas información al respecto, como sería la Web de 12Factor App o similares. Pero en este caso aquí solamente quiero centrarme en la información que nos ofrece AWS.

Cultura, Procedimientos y Procesos

Este post es básicamente una recopilación de artículos y documentos que AWS pone a nuestra disposición para desplegar este tipo de arquitecturas/metodologías en su plataforma.

Pero para aquellos que ya trabajéis en este formato, sabéis perfectamente que las herramientas no son nada sin una cultura que la sustente, unos procedimientos a seguir y unos procesos a ejecutar.

Este apartado se basa en la recopilación de datos y links correspondientes a la información que nos ofrece Amazon para mejorar nuestra implementación/ejecución de DevOps en nuestros proyectos.

Como se especifica en el apartado Communication and Collaboration de la documentación de AWS: Whether you are adopting DevOps Culture in your organization or going through a DevOps cultural transformation, communication and collaboration are an important part of your approach. At Amazon, we have realized that there was a need to bring a change to the mindset of our teams and thus adopted the concept of Two-Pizza Teams.

We try to create teams that are no larger than can be fed by two pizzas,” said Bezos. “We call that the two-pizza team rule.

Y teniendo como este mantra la ejecución de los procedimientos de DevOps, se deja en este apartado los siguientes enlaces de interés:

  • DevOps Guidance (AWS Well-Architected): Drawing from Amazon’s own transformative journey and the expertise gained by AWS in managing cloud services at global scale, the AWS Well-Architected Framework DevOps Guidance offers a structured approach that organizations of all sizes can follow to cultivate a high-velocity, security-focused culture capable of delivering substantial business value using modern technologies and DevOps best practices.
    • DevOps Sagas: Collection of modern capabilities that together form a comprehensive approach to designing, developing, securing, and efficiently operating software at cloud scale.
    • Organizational adoption: Provides a prescriptive approach to creating a more customer-focused culture that can quickly respond to changing business needs.
    • Development lifecycle: Provides a prescriptive approach to optimizing an organization’s ability to develop, review, build, and release workloads to improve delivery speed and create safer deployments.
    • Quality assurance: Emphasizes the integration of test-driven methodologies into every phase of the software development process.
    • Automated governance: Encapsulates the strategic implementation of policies, processes, and tools that allow organizations to manage and control their IT operations effectively and efficiently.
    • Observability: Ability to understand the internal state of your systems through external outputs.
  • Implementing continuous integration and continuous delivery: This section discusses the ways in which you can begin to implement a CI/CD model in your organization.
  • Deployment methods: You can consider multiple deployment strategies and variations for rolling out new versions of software in a continuous delivery process.
  • Testing stages in continuous integration and continuous delivery: Testing should start as early as possible.
  • Deployment Pipeline Reference Architecture:
    The Deployment Pipeline Reference Architecture (DPRA) for AWS workloads describes the stages and actions for different types of pipelines that exist in modern systems.

Herramientas y configuraciones

En base la información que nos presenta AWS se organiza las herramientas aquí indicadas. Aparte se incluirán otras que a medida se han ido implementando soluciones similares se han experimentado o utilizado como parte de la solución final.

Todas ellas están recogidas como Developer Tools console así como otra documentación destinada tanto a desarrollo como gestión de infraestructuras.

Integración Continua

Según AWS: “Continuous integration (CI) is a software development practice where developers regularly merge their code changes into a central code repository, after which automated builds and tests are run. CI helps find and address bugs quicker, improve software quality, and reduce the time it takes to validate and release new software updates.”

Herramientas:

  • AWS CodeCommit: IMPORTANTE: Herramienta descontinuada por AWS desde Julio del 2024.
    • Connections: Sustituye a CodeCommit para integrar las herramientas de AWS con otros repositorios de código. Importante: The service name AWS CodeStar Connections has been renamed. Resources created with the previous namespace codestar-connections will still be supported.
  • AWS CodeBuild: Fully managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy. You don’t need to provision, manage, and scale your own build servers.
  • AWS CodeArtifact: Fully managed artifact repository service that can be used by organizations to securely store, publish, and share software packages used in their software development process. CodeArtifact can be configured to automatically fetch software packages and dependencies from public artifact repositories so developers have access to the latest versions.

Entrega Continua

Nos presenta AWS: “Continuous delivery (CD) is a software development practice where code changes are automatically prepared for a release to production. A pillar of modern application development, continuous delivery expands upon continuous integration by deploying all code changes to a testing environment and/or a production environment after the build stage. When properly implemented, developers will always have a deployment-ready build artifact that has passed through a standardized test process.”

Herramientas:

  • AWS CodeDeploy: Fully managed deployment service that automates software deployments to a variety of compute services such as Amazon Elastic Compute Cloud (Amazon EC2), AWS Fargate, AWS Lambda, and your on-premises servers. AWS CodeDeploy makes it easier for you to rapidly release new features, helps you avoid downtime during application deployment, and handles the complexity of updating your applications. You can use CodeDeploy to automate software deployments, reducing the need for error-prone manual operations. The service scales to match your deployment needs.
  • AWS CodePipeline: continuous delivery service that you can use to model, visualize, and automate the steps required to release your software.

Infraestructura como código

AWS nos introduce: “Practicing infrastructure as code means applying the same rigor of application code development to infrastructure provisioning. All configurations should be defined in a declarative way and stored in a source control system such as AWS CodeCommit, the same as application code. Infrastructure provisioning, orchestration, and deployment should also support the use of the infrastructure as code.”

Herramientas:

  • AWS CloudFormation: Service that enables developers to create AWS resources in an orderly and predictable fashion. Resources are written in text files using JSON or YAML format.
  • AWS Serverless Application Model: The AWS Serverless Application Model (AWS SAM) is an open-source framework that you can use to build serverless applications on AWS.
    AWS SAM integrates with other AWS services, so creating serverless applications with AWS SAM provides the big benefits.
  • AWS Cloud Development Kit (AWS CDK): Open source software development framework to model and provision your cloud application resources using familiar programming languages.
  • AWS Cloud Control API: New AWS capability that introduces a common set of Create, Read, Update, Delete, and List (CRUDL) APIs to help developers manage their cloud infrastructure in an easy and consistent way. The Cloud Control API common APIs allow developers to uniformly manage the lifecycle of AWS and third-party services.

Automatización y desarrollo

Another core philosophy and practice of DevOps is automation. Automation focuses on the setup, configuration, deployment, and support of infrastructure and the applications that run on it. By using automation, you can set up environments more rapidly in a standardized and repeatable manner. The removal of manual processes is key to a successful DevOps strategy. 

Herramientas:

  • AWS OpsWorks: takes the principles of DevOps even further than AWS Elastic Beanstalk. It can be considered an application management service rather than simply an application container. AWS OpsWorks provides even more levels of automation, with additional features such as integration with configuration management software (Chef) and application lifecycle management. You can use application lifecycle management to define when resources are set up, configured, deployed, un-deployed, or ended.
  • AWS Elastic Beanstalk: Service to rapidly deploy and scale web applications developed with Java, .NET, PHP, Node.js, Python, Ruby, Go, and Docker on familiar servers such as Apache, NGINX, Passenger, and IIS.
  • EC2 Image Builder: Fully managed AWS service that helps you to automate the creation, maintenance, validation, sharing, and deployment of customized, secure, and up-to-date Linux or Windows custom AMI. EC2 Image Builder can also be used to create container images. You can use the AWS Management Console, the AWS CLI, or APIs to create custom images in your AWS account.
  • AWS Proton: Enables platform teams to connect and coordinate all the different tools your development teams need for infrastructure provisioning, code deployments, monitoring, and updates. AWS Proton enables automated infrastructure as code provisioning and deployment of serverless and container-based applications.
  • AWS Service Catalog: Enables organizations to create and manage catalogs of IT services that are approved for AWS. These IT services can include everything from virtual machine images, servers, software, databases, and more to complete multi-tier application architectures. AWS Service Catalog lets you centrally manage deployed IT services, applications, resources, and metadata to achieve consistent governance of your IaC templates.
  • AWS Cloud9: Cloud-based IDE that lets you write, run, and debug your code with just a browser. It includes a code editor, debugger, and terminal. AWS Cloud9 comes prepackaged with essential tools for popular programming languages, including JavaScript, Python, PHP, and more, so you don’t need to install files or configure your development machine to start new projects. Because your AWS Cloud9 IDE is cloud-based, you can work on your projects from your office, home, or anywhere using an internet-connected machine. IMPORTANTE: After careful consideration, we have made the decision to close new customer access to AWS Cloud9, effective July 25, 2024. AWS Cloud9 existing customers can continue to use the service as normal. AWS continues to invest in security, availability, and performance improvements for AWS Cloud9, but we do not plan to introduce new features.
  • AWS CloudShell: Browser-based shell that makes it easier to securely manage, explore, and interact with your AWS resources. AWS CloudShell is pre-authenticated with your console credentials. Common development and operations tools are pre-installed, so there’s no need to install or configure software on your local machine.
  • Amazon CodeGuru: Developer tool that provides intelligent recommendations to improve code quality and identify an application’s most expensive lines of code.
  • Amazon CodeCatalyst: Amazon CodeCatalyst is a unified software development service that makes it easy for development teams to quickly build and deliver scalable applications on AWS, while adhering to organizational best practices.

Monitorización y Observabilidad

Communication and collaboration are fundamental in a DevOps philosophy. To facilitate this, feedback is critical. This feedback is provided by our suite of monitoring and observability services.

Herramientas:

  • Amazon CloudWatch: Amazon CloudWatch monitors your Amazon Web Services (AWS) resources and the applications you run on AWS in real time. You can use CloudWatch to collect and track metrics, which are variables you can measure for your resources and applications.
    • Metrics: Automatically collect data from AWS services such as Amazon EC2 instances, Amazon EBS volumes, and Amazon RDS database (DB) instances. These metrics can then be organized as dashboards and alarms or events can be created to trigger events or perform Auto Scaling actions.
    • Alarms: You can set up alarms using Amazon CloudWatch alarms based on the metrics collected by Amazon CloudWatch metrics. The alarm can then send a notification to Amazon SNS topic, or initiate Auto Scaling actions. An alarm requires period (length of the time to evaluate a metric), evaluation period (number of the most recent data points), and datapoints to alarm (number of data points within the evaluation period).
    • Logs: Log aggregation and monitoring service. AWS CodeBuild, CodeCommit, CodeDeploy and CodePipeline provide integrations with CloudWatch logs so that all of the logs can be centrally monitored. In addition, the previously mentioned services various other AWS services provide direct integration with CloudWatch.
    • Logs Insights: scans your logs and enables you to perform interactive queries and visualizations. It understands various log formats and auto-discovers fields from JSON logs.
    • Events: Delivers a near real-time stream of system events that describe changes in AWS resources. Using simple rules that you can quickly set up, you can match events and route them to one or more target functions or streams. You can configure rules in Amazon CloudWatch Events to alert you to changes in AWS services and integrate these events with other third-party systems using Amazon EventBridge. 
  • Amazon EventBridge: Serverless event bus that enables integrations between AWS services, Software as a services (SaaS), and your applications. In addition to build event driven applications, EventBridge can be used to notify about the events from the services such as CodeBuild, CodeDeploy, CodePipeline, and CodeCommit. Amazon CloudWatch Events and EventBridge are the same underlying service and API, however, EventBridge provides more features.
  • AWS CloudTrail: All AWS interactions are handled through AWS API calls that are monitored and logged by AWS CloudTrail.
  • Amazon DevOps Guru: Service powered by machine learning (ML) that is designed to make it easy to improve an application’s operational performance and availability. DevOps Guru helps detect behaviors that deviate from normal operating patterns, so you can identify operational issues long before they impact your customers.
  • AWS X-Ray: Helps developers analyze and debug production, distributed applications, such as those built using a microservices architecture. With X-Ray, you can understand how your application and its underlying services are performing to identify and troubleshoot the root cause of performance issues and errors. X-Ray provides an end-to-end view of requests as they travel through your application, and shows a map of your application’s underlying components. X-Ray makes it easy for you to: Create a service map, Identify errors and bugs and Build your own analysis and visualization apps.
  • Amazon Managed Service for Prometheus: Serverless monitoring service for metrics compatible with open-source Prometheus, making it easier for you to securely monitor and alert on container environments. Amazon Managed Service for Prometheus reduces the heavy lifting required to get started with monitoring applications across Amazon Elastic Kubernetes Service, Amazon Elastic Container Service, and AWS Fargate, as well as self-managed Kubernetes clusters.
  • Amazon Managed Grafana: Fully managed service with rich, interactive data visualizations to help customers analyze, monitor, and alarm on metrics, logs, and traces across multiple data sources. You can create interactive dashboards and share them with anyone in your organization with an automatically scaled, highly available, and enterprise-secure service.
  • DevOps Monitoring Dashboard on AWS: Automate the process of ingesting, analyzing, and visualizing continuous integration/continuous delivery (CI/CD) metrics.

Seguridad

Whether you are going through a DevOps transformation or implementing DevOps principles for the first time, you should think about Security as integrated in your DevOps processes. This should be cross cutting concern across your build, test deployment stages.

Herramientas:

  • Identity and Access Management: Defines the controls and polices that are used to manage access to AWS resources. Using IAM you can create users and groups and define permissions to various DevOps services.
  • Security Hub: Security Hub collects security data across AWS accounts, AWS services, and supported third-party products and helps you analyze your security trends and identify the highest priority security issues.

Otros Links de interés

Leave a comment