From 259e8a848c5a292ef34bb1fe8ff3d44f8c25acbb Mon Sep 17 00:00:00 2001 From: omagdy7 Date: Sun, 26 May 2024 01:07:23 +0300 Subject: Added Readme --- README.md | 84 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..f0b2f8a --- /dev/null +++ b/README.md @@ -0,0 +1,84 @@ +# CloudRender Project + +## Overview + +CloudRender is a distributed image processing system that leverages cloud computing resources to efficiently process large volumes of images. The project uses Terraform for Infrastructure as Code (IaC) and Kubernetes for container orchestration, ensuring scalability and fault tolerance. The system comprises several components, including an application server, worker nodes, a load balancer, Kubernetes, Terraform, Amazon S3, and RabbitMQ, working together to provide a robust, scalable, and efficient image processing service. + +## Project Structure + +1. **Application Server:** Manages incoming image processing requests and communicates with worker nodes. +2. **Worker Nodes:** Perform the actual image processing tasks. +3. **Load Balancer:** Distributes incoming requests evenly across worker nodes. +4. **Kubernetes:** Orchestrates the containerized applications, including deployment, scaling, and management. +5. **Terraform:** Manages the setup of the project's underlying infrastructure. +6. **Amazon S3:** Stores the original and processed images. +7. **RabbitMQ:** Acts as the message broker, managing the task queues between the application server and worker nodes. + +## Prerequisites + +- AWS account +- Terraform installed +- Kubernetes installed +- Docker installed +- AWS CLI configured + +## Installation and Setup + +### 1. Clone the Repository + +```sh +git clone https://github.com/omagdy7/CloudRender +cd CloudRender +``` + +### 2. Clone the infra Repository +git clone https://github.com/omagdy7/CloudRender-Infra + +### 3. Set Up Infrastructure with Terraform +Navigate to the terraform directory and initialize Terraform. + +```sh +cd CloudRender-Infra/terraform +terraform init +``` +Apply the Terraform configuration to set up the infrastructure. + +```sh + +terraform apply +``` + +### 4. Build Docker Images +Build the Docker images for the application server and worker nodes. + +```sh +docker build -t cloudrender-app ./app +docker build -t cloudrender-worker ./worker +``` + +### 5. Push Docker Images to AWS ECR +Tag and push the Docker images to your AWS Elastic Container Registry (ECR). + +```sh +# Tag the images +docker tag cloudrender-app:latest .dkr.ecr..amazonaws.com/cloudrender-app:latest +docker tag cloudrender-worker:latest .dkr.ecr..amazonaws.com/cloudrender-worker:latest +# Push the images +docker push .dkr.ecr..amazonaws.com/cloudrender-app:latest +docker push .dkr.ecr..amazonaws.com/cloudrender-worker:latest +``` + +### 6. Deploy to Kubernetes +Navigate to the k8s directory and apply the Kubernetes configurations. +```sh +cd CloudRender-Infra/kubernetes +kubectl apply -f app-deployment.yaml +kubectl apply -f worker-deployment.yaml +kubectl apply -f auto-hpa.yaml +``` + +## Usage +- Upload images to be processed by sending requests to the application server. +- The application server stores the images in Amazon S3 and sends task notifications to RabbitMQ. +- Worker nodes retrieve tasks from RabbitMQ, fetch images from S3, process them, and store the results back in S3. +- The load balancer distributes incoming requests to ensure no single node is overwhelmed. -- cgit v1.2.3