Developing a Simple Python Application and Automating the CI/CD and Containerization Process [Part 3A]
Constructing the CI/CD Framework
Previously, I talked about how to set up a simple Python application and create infrastructure on the cloud using Terraform. For a brief refresher on how we accomplished this, check out the resources section below to visit my latest articles. Now let’s move into the third phase of this project, which is to construct the continuous integration/continuous deployment (CI/CD) framework to automate this build process. We’ll be heading back to Microsoft Azure, now utilizing their comprehensive CI/CD platform, Azure Devops, to build out our example application. I like Azure Devops for a couple of reasons — its interface is very similar to any other CI/CD platforms you’ve probably worked with (like Gitlab, Bitbucket, etc.), but its also superior in the sense that it provides a lot of plugins and sample pipeline code fragments to assist in creating build pipelines for a variety of use cases.
Alright, first let’s dive into some of the basic requirements to set up a project on the Azure DevOps framework. We need to start by creating an organization, or a structure within which we will be able to house all of our projects. If you’re new to Azure DevOps or unfamiliar with how to create an organization, check out the resources section at the end of this article for a helpful tutorial on this from Microsoft Azure. Once we’ve got the organization set up, we will be redirected to a page similar to Figure 1, where an option to add a new project will be shown. We can title this project, provide a description for its purpose, and determine whether we want it publicly or privately accessible within our organization. Once we’ve added our new projects, we should see them all listed under our organization.
Clicking on the project of interest will reroute us to the CI/CD platform for that project, where we can upload our code to our project repository, set up build and release pipelines, manage artifacts, generate test plans, etc, as shown in Figure 2.
The first thing we’ll do now is upload our code to the project by creating a repository and cloning it to our local. I’m assuming my readers here are familiar with git commits and pushes so I’m not going to go over that process in detail, but I will offer a tip — always make sure that you do a pull on your remote repository to clone the latest changes into your local, before you do a commit and push of your local files to the remote origin. This ensures that any merge conflicts and inconsistencies between the local and remote origins are resolved before the code merge is approved and completed. After our code has been merged into the remote project repository, we should see an interface similar to Figure 3.
Next, we can go ahead and start developing the continuous integration build process. So what is continuous integration? It’s a process through which our CI/CD platform performs checks on the code within our remote repository to determine if new changes have been contributed by developers in our organization. If there are changes, the process builds them out to alter or create new cloud infrastructure or simply just run the modified application code to perform its functionality. Figures 4–8 go through the steps of setting up a simple build pipeline with YAML syntax to illustrate this process visually.
The above process is pretty easy to follow, but one note I’d like to make here is regarding the configuring of tasks itself in the pipeline. Some of you may know how to write YAML syntax so it may not be an issue for you to set up build pipeline tasks to accomplish what you’re looking to do, as shown in the images above. But for those of us that are not familiar with YAML and need a little bit of help in setting up tasks to configure a build process, there are two options to utilize — 1) searching for specific tasks on the Azure DevOps Build Pipeline Interactive Editor and 2) using the classic editor instead. Here, in Figure 7, to utilize the former option, I searched for the docker build task since I’m looking to package my application code into a Docker image that I can then run as a container. Clicking on this docker build task helps me set up a sample YAML format for this task, which I can then cater to add in my Docker repository and registry details. The latter option makes my life even easier, because I can exit out of this build editor and instead, in step 1 (shown in Figure 5), I can choose to use the classic editor (right below Subversion) which creates the pipeline without me having to write any YAML syntax at all. How is that even possible? Take a look at Figures 9–14 below. We are not required to specify any YAML code to set up the pipelines because Azure DevOps automatically supplies the YAML syntax based on the selections we make to configure the pipeline agent and build tasks, behind the scenes. This makes it easy for anyone, even with very limited experience in CI/CD to get pipelines set up and running on the Azure DevOps framework.
If we follow the instructions on the interactive editor to configure the build tasks, we should end up with a build pipeline that contains two stages, one to actually build the Docker image from the application code and Dockerfile, and the other to push the image to the repository we created on Azure Container Registry (which I discussed in my previous article). Let’s delve deeper into this to show what needs to be configured for this process to properly work.
The first task deals with building the Docker image and tagging it to our Azure Container Registry. To do this, we must select our registry from the list of registries associated with our subscription and choose the ‘Build an Image’ option. Now, in order for an image to be actually built, we need to supply a Dockerfile with instructions to package our application code and dependencies. Figure 15 shows the Dockerfile that I created and supplied to this build step in order to package the simple Python banking application.
Writing a Dockerfile is pretty straightforward as long as you know what you want your container to run and what you require to run it. In our case, the application was coded in Python, and required a few dependencies for the Microsoft ODBC SQL driver (discussed in my first article), so I had to use the Python base image and a bunch of run commands to install these dependencies onto the image. The last few steps of a Dockerfile are normally the same across most images, in that we have to copy over our working directory files onto the image, switch to the directory we copied to, and run our application.
Once we have created this file, we would normally build it using the docker build -t imageName command from our current working directory. Since we have specified the docker build option on our pipeline, it will perform the same build command behind the scenes to create our Docker image. We will just need to provide a tag name for the image so that we know what to look for in our ACR repository. With just these few preliminary steps, the image is built and stored locally, but now if we want it to be stored on Azure, we need to actually push it to our ACR repository. Pushing an image is way easier than actually building it as all we need to supply is the name of the image (the tag we just created in the previous step) so that it can match to our repository on ACR and push the image to that tag. Choosing the ‘Push an Image’ option will commit and push the image to ACR. To see the exact details of how my build pipeline steps were configured, check out the link below this article to take you to my project on Azure Devops.
That wraps up the build portion for the application code. There is one more part to the build process and that is the infrastructure side. We need to create the resources we set up via Terraform (in the previous article) on the cloud, through another build pipeline that can run the terraform apply command all without manual input for us. Now since this is a whole other topic that I want to devote more time and clarification to, I’ve decided to split the CI/CD process articles up and discuss this in my next piece (Part 3B)! Until then, get some more practice on your coding and the concepts I’ve introduced in these three articles! See y’all soon with more content :)
Find my code here 👉 https://abujji.visualstudio.com/banking-app-python
Resources:
- https://medium.com/@madhuvanthi-sridhar/developing-a-simple-python-application-and-automating-the-ci-cd-and-containerization-process-part-5d11509cbe9d
- https://medium.com/@madhuvanthi-sridhar/developing-a-simple-python-application-and-automating-the-ci-cd-and-containerization-process-part-19b39ccda807
- https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/create-organization?view=azure-devops
- https://stackoverflow.com/ (If you’re in Tech, then you know how big of a lifesaver this forum is 😂)
- Can’t forget, my Brain 😎
