AWS ECS service is an excellent solution for running CPU-intensive tasks.
Firstly, it doesn’t cost much; you pay for what you use and for the computing time. No need to hold a vast virtual machine running and pay for air.
Secondly, isolating tasks. You know that each task is isolated and doesn’t block or steal compute resources from other tasks. Each task’s resources are defined and limited by your own desire.
Finally, easy updating and management. For updating your task, one thing you should do is update your ECS Task Definition and push a new image to the ECR repository. Management can cause problems only if you are unfamiliar with AWS Cloud. In other cases, it will be easy.
So, as a result, we get an economical and effective mechanism of CPU-intensive task execution.
This blog post will help you to choose the best solution for CPU-intensive tasks. Also, you learn about ECS service implementation and why it is so good for this kind of task. When our team faced CPU-intensive tasks, we tried different solutions. So, take our experience and learn new things about AWS ECS!
AWS ECS Investigation Process
Firstly, we tried to use a Docker container on AWS EC2 Instance, but the main process consumes all EC2 processor time for this task. If there are two or more processes, the EC2 instance will just stack. On our board, we have the opportunity to split CPU-intensive tasks to CPU threads on the EC2 Instance, but it also didn’t make much profit (we can run more tasks, but there is no option to handle scaling EC2 Instance resources in the right way).
After that, we decided to use serverless solutions. We chose AWS Lambda functions. After preparing all the stuff for deploying code as Lambda, we discovered that Lambda allows only 15 minutes of function execution. It was a critical blocker for us because if some process is long enough, the lambda function can't finish it properly and will stop the process because of an unchangeable timeout. So, we should refuse this kind of solution.
Gradually, we tried the ECS service and had success with it. Asynchronous CPU-intensive tasks work great and can execute separately from each other, so there is no need to implement some scheduler that looks for a task queue.
Understanding AWS ECS
What is AWS ECS?
Amazon Elastic Container Service (ECS) is a fully managed service offered by Amazon Web Services (AWS) that makes it simple to run, manage, and scale Docker containers in the cloud. ECS provides high scalability, security, and performance for running containerized applications and microservices.
ECS offers many benefits over traditional infrastructure management approaches, including automatic scaling, automatic task placement, and self-healing. You can easily scale your applications as demand increases or decreases, and ECS automatically manages the distribution of tasks across available resources. Additionally, ECS will automatically replace failed containers with healthy ones, helping to ensure that your applications are highly available and always running.
Get inspired by our recent blog post - HOW TO REDUCE AWS COSTS: TOP STRATEGIES!
Product Discovery Lab
Free product discovery workshop to clarify your software idea, define requirements, and outline the scope of work. Request for free now.
Main components of the AWS ECS
AWS ECS works by abstracting the infrastructure required to run Docker containers and providing a high-level API for managing containers. The following are the basic components and steps involved in the functioning of AWS ECS:
- Cluster: A group of EC2 instances that host the Docker containers. You can create a new cluster or use an existing EC2 instance as the cluster.
- Task Definition: A blueprint for your application that specifies the Docker image to use and the resources required by the containers in your task.
- Task: A running instance of a task definition. A task can contain one or more containers.
- Container Instance: An EC2 instance that runs one or more Docker containers.
- Service: A long-running task that represents a deployment of a specific version of an application. A service schedules and maintains a desired number of tasks.
- Load Balancer: An optional component that distributes incoming traffic across multiple tasks within a service.
To run a Docker container using ECS, you create a task definition that specifies the Docker image to use, configure the desired resources, and then launch the task in a cluster. The ECS service then automatically places the task on an available EC2 instance, launches the containers in the task, and monitors the running containers. You can then use the ECS API to manage and scale your containers as needed without managing the underlying infrastructure.
With ECS, you also can scale your containers automatically based on demand and automate the deployment of new versions of your application. The service provides several tools and features to help you manage and monitor your containers, including real-time metrics, logs, and alerts. This allows you to identify and resolve issues quickly and ensure that your applications run smoothly.
GET GUIDE ON AWS COST OPTIMIZATION FOR FREE!
Async Processing With ECS
The process of setting up async processing with ECS looks like the general setting up task in ECS service, but there are some differences.
Firstly, you need to build and push a Docker image from your source code to the ECR repository. For example, upload your task’s source code to the Github repository, create an AWS Codebuild project, and with its help, build and push your Docker image to the ECR repository.
After that, you should create an ECS cluster as a basement for future ECS tasks executing.
When you have an ECS cluster, you can start building your ECS Task Definition. You need to define the IAM role, Security Group, network, ephemeral storage, and environment variables for your task. One more important thing is monitoring the status of your running task. So you should decide how you will handle the status of the finished or failed task. We use a database for solving this issue, so if a task fails (no matter what reason causing this), the server will reinvoke this task after some time when it checks the status in the database.
Your initiator, in our case, is an application server, and should be able to run ECS tasks. Create an IAM role for this and provide a server’s SDK with AWS Programmatic access with this attached role.
Conclusion
Finally, we got a great solution using ECS. For business, it is really economical, you spend money only on what you use. ECS is the issue-killer that works pretty effectively. In addition, you get hosting that is always working, and AWS is more than a reliable сloud service. ECS has no restrictions which can block development processes, and it is managed easily.
If you have some troubles with ECS service implementation, don't hesitate to contact us and get all the expertise from Axon software engineers for your project! Get in touch in one click.
