The Developer Space

Developer's Cafe

  • Cloud
  • Database
  • Open Source
  • Programming
  • Web Dev
  • Mobile
  • Security
  • QuickRef
  • Home
  • Cloud
  • Minimize Non-Critical Database Workload costs in AWS

Minimize Non-Critical Database Workload costs in AWS

Shameel Ahmed - AWS, Cloud, Database, Open Source, PostgreSQL, SQL Server
April 23, 2020April 24, 2020 4 Comments
5 0
Read Time9 Minute, 2 Second

One of the major challenges in Cloud is controlling costs effectively. With an on-premise data center infrastructure, the costs are known ahead and planned in advance and usually predictable. On the other hand, Cloud platforms usually do not have an upfront cost and you pay as you use. While the Cloud seems to be cost effective in the long run, there are elements of uncertainty which might give us a shock if we do not plan and use Cloud resources judiciously. The objective of this article is to set up and operate non-critical database workloads while keeping their costs within reasonable limits.

This article assumes that the reader is familiar with AWS and Database concepts and terminologies. While this article uses PostgreSQL as an example, this can be applied to other Database Engines as well.

There are various ways to set up a relational database in AWS. The cost depends on how you set it up and what Database Engine you use. Note that the techniques discussed here are applicable only to non-critical workloads where high-availability, automatic failover, etc. are not required. The topics discussed here may not be relevant to critical workloads which need to be always on due to continuous development, deployment or testing.

AWS provides different options to set up a Relational Database. And the available options depend on the Database Engine you choose.

  1. Aurora
  2. Relational Database Service (RDS)
  3. Databases hosted on EC2 Virtual Machines

Aurora

Aurora is Amazon’s own fully managed Database Engine that runs on top of either PostgreSQL or MySQL. It is a drop-in replacement for the respective database engines and has many features built on top of what the underlying Engines provide.

Of all the available options, Aurora provides the most features and is the most expensive. It provides built-in high availability by replicating data to two nodes in each Availability Zone (AZ) within the region. Most regions typically have 3 AZs which means it keeps six copies of your data and is available even if an entire AZ goes down. It provides automatic failover to secondary replica with zero data loss when the primary goes down. The failover is transparent and client applications can continue to access the database with the same endpoint. Aurora supports up to 15 read replicas with milliseconds replication latency. The primary writer instance and the read replicas share the same underlying storage and therefore there is no real replication happening. This makes it a very good solution for read heavy workloads.

Aurora is well suited for Production workloads with variable load as well as continuous peak loads, where high availability is needed. It may not be suited for non-critical workloads due to its high cost.

Relational Database Service (RDS)

RDS is an AWS service that makes it easy to set up and operate a Relational Database in AWS at the click of a button. It supports different Database Engines like SQL Server, Oracle, PostgreSQL, MySQL, MariaDB, etc. Unlike Aurora, RDS does not provide high availability and automatic failover but these features can to be set up manually. It comes at a relatively lower cost compared to Aurora for the same configuration.

RDS suits Production workloads with variable load as well as continuous peak loads, where high availability is not a major concern and downtime of few minutes to few hours is acceptable. Note that you can still set up high availability manually by having a cold or warm standby instance and manually failing over to it when primary instance goes down. Failovers may result in data loss.

RDS is only a little less expensive than Aurora and therefore may not be the right choice for non-critical workloads. It doesn’t provide any added benefits compared to Databases hosted on EC2 other than the ease with which you create and manage the database. RDS supports up to 5 read replicas, and the replication latency is a little higher than that of Aurora.

Databases hosted on EC2 Virtual Machines

While the whole point of using a Cloud Database is to take advantage of the simplified set up and operation provided by Cloud platforms, there may be times when this might be an overkill and may not provide a reasonable cost-benefit ratio. In such cases, you can spin EC2 instances and manually set up and administer the databases. EC2 Virtual Machines are very cost effective when compared to Aurora and RDS. Examples of such workloads could be non-critical databases which doesn’t need to be always on, databases that are used for development purposes, Staging, Pre-Production and UAT databases that are used only for testing & validation, etc.

If you are using open source databases like PostgreSQL, MySQL or MariaDB, then there is no need to worry about licensing. However, if you are using proprietary databases like Oracle or SQL Server, you can use the Express editions which are free to use but come with database size limitations. Alternatively, you can also use Licensed versions of these databases if you already own the licenses.

Hosting Databases on EC2 also allows you to use all command line tools, plugins, etc. that are not fully available on Aurora and RDS. It also gives you complete control over the database and the underlying operating system tools. This method can also be used to host databases that are not supported by Aurora and RDS. For example, MongoDB, CouchBase, HazelCast etc.

Caveats

Hosting Databases on EC2 requires that you have a Database Administrator to set up, configure, and to continuously monitor and maintain the Databases. Aurora and RDS provide GUI tools to create and manage Databases at the click of a button. Since this article is specifically focused on non-critical workloads, this might not be an important point of consideration.

If you are using proprietary Database Engines like Oracle and SQL Server, you may have to incur additional licensing costs which are not covered here. Alternatively, you can use the free Express editions of these databases to avoid licensing costs. Express editions come with certain limitations in terms of storage and performance, but should be enought for non-critical workloads. Aurora and RDS include the license for proprietary Database Engines and you don’t have to worry about it. In short:

Cost Comparison

Let’s take an example of a indicative price comparison for PostgreSQL Database (US East – N Virginia) as of April 2020. These are On-Demand prices and Reserved prices are far less than the ones mentioned here. Note that there are other charges like Storage, Data transfer, etc. The comparisons are only indicative and may change drastically depending on how the services are used.

Note: Prices mentioned are for the EC2 r5 and RDS db.r5 Instance Families.

ConfigurationAuroraRDS (Single AZ)EC2 (Linux)
2 CPU, 4 GB RAM$0.082/hr$0.072/hr$0.0416/hr
4 CPU, 32 GB RAM$0.58/hr$0.50/hr$0.252/hr
16 CPU, 128 GB RAM$2.32/hr$2.00/hr$1.008/hr

Cost Exercise

Let’s calculate the monthly costs for roughly equivalent configurations for the three approaches using AWS Pricing Calculator.

ConfigurationAuroraRDS (Single AZ)EC2 (Linux)
Instance Typedb.r5.xlarge
(4 CPU, 32 GB Memory)
db.r5.xlarge
(4 CPU, 32 GB Memory)
r5.xlarge
(4 CPU, 32 GB Memory)
Storage100 GB100 GB100 GB (EBS)
Backup Storage20 GB20 GB20 GB
Data Transfer10 GB10 GB10 GB
On-Demand Price$434$378$175
Standard Reserved Price (1 year – no upfront)$288$224$114
Spot InstanceN/AN/A$49

Summary

As you can see, you can easily save a minimum of 50% in EC2 VMs compared to Aurora and RDS. If your Databases need not be available all the time, you can opt for Spot instances that allows you to save up to 90% of your overall costs. Note that the these comparisons are very approximate and may vary drastically in real world usage.

Aurora if:

  1. Your Database Engine is PostgreSQL or MySQL
  2. You don’t have a dedicated DBA
  3. Your Database needs to be always on and no downtime is allowed during an outage.

RDS if:

  1. Your Database Engine is PostgreSQL, MySQL, MariaDB, Oracle, SQL Server.
  2. You don’t have a dedicated DBA
  3. Your Database needs to be available most of the time, but it is okay to have a few minutes of downtime during an outage.

EC2 if:

  1. Your Database Engine is anything, even those outside of what Aurora/RDS offers, i.e., Redis, Hazelcast, MongoDB, CouchDB, CouchBase, Cassandra, etc.
  2. You have a dedicated DBA who can create & configure the databases and set up backup scripts initially and be available when things go wrong.
  3. Your Database availability is not a big deal and it is okay to have a few hours of downtime during an outage.

Share

Facebook
Twitter
LinkedIn
Email

Post navigation

C# 8 New Features
Book: Migrating your SQL Server Workloads to PostgreSQL

Related Articles

Migrate your SQL Server Workloads to PostgreSQL: Quick Reference: Second Edition

Book: Migrate your SQL Server Workloads to PostgreSQL: Quick Reference – Second Edition

Shameel Ahmed
October 18, 2022October 22, 2022 1 Comment

How to fix Redis Error “ERR DEBUG command not allowed.”

Shameel Ahmed
July 4, 2022July 5, 2022 No Comments

Cross Region Lambda invocation in AWS

Shameel Ahmed
February 27, 2022February 27, 2022 No Comments

Average Rating

5 Star
0%
4 Star
0%
3 Star
0%
2 Star
0%
1 Star
0%
(Add your review)

4 thoughts on “Minimize Non-Critical Database Workload costs in AWS”

  1. Arunkumar says:
    April 25, 2020 at 6:13 pm

    Excellent Shameel ! Nice Article , keep going …

    Reply
    1. Shameel Ahmed says:
      April 26, 2020 at 12:40 pm

      Thank you Arunkumar.

      Reply
  2. Sivashakar says:
    April 25, 2020 at 8:55 pm

    Nice Article

    Reply
    1. Shameel Ahmed says:
      April 26, 2020 at 12:40 pm

      Thank you Sivashankar.

      Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Categories

.NET Architecture Artificial Intelligence ASP.NET AWS Azure Books C# Career Cloud CodeProject Conversational Bots Database Data Security Facade IDEs Java Mobile MongoDB MySQL Open Source Patterns PostgreSQL Programming Python Redis Security SQL Server Tools Uncategorized Web Development Windows Phone

Recent Posts

  • Developer to Architect Series (Red Hat Enable Architect) January 16, 2023
  • Can ChatGPT replace Google Search? January 11, 2023
  • Learn Python with ChatGPT December 26, 2022
  • Book: Migrate your SQL Server Workloads to PostgreSQL: Quick Reference – Second Edition October 18, 2022
  • Increase Visual Studio Code Terminal Buffer Size July 14, 2022

Archives

  • January 2023 (2)
  • December 2022 (1)
  • October 2022 (1)
  • July 2022 (2)
  • February 2022 (1)
  • November 2021 (1)
  • July 2021 (1)
  • June 2021 (1)
  • September 2020 (1)
  • May 2020 (2)
  • April 2020 (1)
  • October 2019 (1)
  • September 2019 (4)
  • July 2019 (2)
  • May 2018 (1)
  • September 2017 (1)
  • April 2017 (1)
  • April 2014 (1)
  • August 2011 (1)
  • June 2009 (1)
Copyright 2022. The Developer Space | Theme: OMag by LilyTurf Themes
  • DbStudio
  • Re:Link
  • shameel.net
  • Privacy Policy
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
Do not sell my personal information.
Cookie settingsACCEPT
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are as essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
SAVE & ACCEPT