For our team automation is a starting point, we always design automated development processes. Let’s find out how we do this.
When we take care of the automation in the first stages of the project, we can save a lot of time and money in the following stages. Our goal is to automate everything and don’t use human resources for repeatable and expensive activities like deploying, creating infrastructure, testing, and checking our software. Nowadays, we have many great tools and techniques to achieve this goal of providing and maintaining software with possibly the best quality.
Source code projects
We used to use GitLab as a source code repository based on Git technology. Starting a new project we prepare the right projects and group structure depending on project requirements.
Projects structure
The structure of our example project looks like the below:
-
device - Sub group for IoT projects, deployed to electronic boards
-
internal - Sub group for UX, project informations, documentations
-
dashboard - Sub group for dashboard application, UI part, microservices, and infrastructures creation
-
public - Sub group for public resources like a web page and infrastructure creation for this area
Of course, the naming convention is very important and we need to remember that bad names, in the beginning, can have a big impact later on understanding by other team members and customers. A good practice is to prepare and maintain example projects to speed up new project creation by forking base projects.
Project sources
Project structure needs to be compliant with the standard to be easily extended by another team or developer.
Every project has to have its README documentation and CHANGELOG file to easily identify how to run and maintain source code. By keeping our projects always stable and well documented, we can easily avoid misunderstandings.
Pipelines
This is definitely the heart of our project’s ecosystem. Well designed and used can be a solution to keep all projects and systems stable, secure, and maintenance-free. On the market there are various CI/CD systems, we used to use GitLab CI/CD in our projects.
Development flow
Every framework is different, we need to provide a suitable continuous integration process for every project and application.
Java microservice pipeline
For example, our Java microservice pipeline can look like the below:
-
Build - compile Java source code, make JUnit tests and prepare JAR file
-
Test - module tests with TestContainers framework
-
Deploy - prepare Docker image, deploy docker to AWS EKS registry and deploy new ECS TaskDefinition to ECS container
We can easily extend our pipeline and add other stages like:
-
Anchor security scanner verification
-
Sonar Quality Gate verification
Infrastructure pipeline
Another pipeline example is Terraform project automation based on CI/CD scripts.
-
Plan - create a plan for Terraform scripts. Comparing with current infrastructure and prepare ready to run Terraform plan
-
Apply - make changes for AWS infrastructure
-
Start and stop - optional step to start and stop our test environment in the cloud
Test results
Every pipeline can export test results and provide this for users and simply show them in our merge requests.
Group variables
To not repeat variables between projects is a good idea to gather all variables under group.
Cloud Automation
Cloud automation is strictly related to the Infrastructure as a Code pattern where we can reflect our whole architecture on the source code. We used to use Terraform framework to onboard complete environments to the cloud. Terraform scripts can be integrated with CI/CD scripts and infrastructure creation can be also an automated process without any human activities.
GitFlow
GitFlow is a current standard for developing and maintaining projects by development teams. Everybody uses it to plan and release software without conflicts and problems due to concurrency.
This methodology can be used to drive our pipeline flow. The most common use cases:
-
develop branch - flow dedicated to compile and deploy our changes merged from feature branches
-
feature branch - flow dedicated to compile developer changes ad hoc
-
release branch - flow dedicated to provide our build-to-stage environment
-
master branch - flow dedicated to production environment
-
merge requests - flow dedicated to compile, check and test developer changes from feature branches
We can easily design our own behavior for the above cases.
Useful information
Summary
CI/CD is a very large topic, as always a good idea is to stay close to the standards, track Technology Radars, and practice ideas on real projects. Because of current architecture trends like microservices and cloud - DevOps is a key skill for a good working development team.
