Describe the architecture you would use to move large files
Sigiloso
Architecture Design: Client-Side Upload Handling: Use chunked uploads to split large files into smaller parts. Implement parallel uploads to maximize bandwidth. Support resumable uploads in case of failures. API Gateway & Load Balancer: Route requests to the appropriate services. Ensure authentication & authorization via OAuth or JWT. File Processing Service (Microservice-based approach): Asynchronous processing using event-driven architecture (Kafka/SQS). Store metadata (file name, size, user, checksum) in a database (PostgreSQL, MongoDB). Storage Layer: Use cloud-based storage (AWS S3, GCS, Azure Blob) for scalability. Enable multi-region replication for redundancy. Utilize tiered storage (hot/cold storage) based on access frequency. Streaming & CDN for Distribution: Use CDN (CloudFront, Akamai) for global distribution. Implement range requests to allow partial downloads. Security & Integrity: Encrypt files at rest (AES-256) & in transit (TLS). Implement checksum (MD5, SHA-256) for data integrity validation.