{"id":9389,"date":"2022-10-05T08:43:58","date_gmt":"2022-10-05T15:43:58","guid":{"rendered":"https:\/\/www.pingcap.com\/?p=9389"},"modified":"2024-08-21T07:14:17","modified_gmt":"2024-08-21T14:14:17","slug":"simplify-database-geo-redundancy-backup-with-cloud-storage-services","status":"publish","type":"post","link":"https:\/\/www.pingcap.com\/ko\/blog\/simplify-database-geo-redundancy-backup-with-cloud-storage-services\/","title":{"rendered":"Simplify Database Geo-redundancy Backup with Cloud Storage Services"},"content":{"rendered":"

Geo-redundancy is a disaster recovery practice that backs up data in geographically distributed data centers. This approach safeguards against incidents and guarantees business continuity. For database systems, there are mainly two ways to be geo-redundant: cold (offline) backup and hot (online) backup. Cold backup is more widely used because it\u2019s cheaper and has less impact on online performance. <\/p>\n\n\n\n

However, it\u2019s not easy to build a cost-effective, scalable cold backup solution. There can be challenges in scalability, operations, and costs. In this article, I will offer a solution to these challenges: geo-redundancy backup with cloud storage service.<\/p>\n\n\n\n

<\/span>Challenges of geo-redundancy cold backup<\/strong><\/span><\/h2>\n\n\n\n

First, let\u2019s look at the limitations of the most commonly used cold backup solutions. <\/p>\n\n\n\n

Database built-in backup capabilities<\/strong><\/h3>\n\n\n\n

Databases are usually shipped with cold backup capabilities. They either provide tools to dump data into your local storage or to upload data to remote storage services. For databases, geo-redundancy backup is not much different from single destination backup. You simply add more remote storage destinations.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Traditional cold backup with databases<\/em><\/p>\n\n\n\n

However, dumping data during backup can use a lot of bandwidth. More the destinations you specify, the more bandwidth you consume. This may significantly impact database performance. In the worst case, dumping data may exhaust most of the bandwidth, and the database stops to respond to queries. Therefore, geo-redundancy using built-in database features is probably not a scalable solution. <\/p>\n\n\n\n

Database geo-redundancy + a message queue<\/strong><\/h3>\n\n\n\n

Using a message queue (MQ) can solve this bandwidth issue.  MQ is a mechanism for asynchronous interprocess communication and is widely accepted as a solution for saving bandwidth\u2014especially when your network is constrained by concurrent data . You can put an MQ service between the database instance and the remote storage.<\/p>\n\n\n\n

MQ lets you distribute data to multiple destinations to achieve geo-redundancy backup at scale. The impact on database performance will be the same as a single-destination backup. If the MQ reaches its performance bottleneck, you can add more instances to the chain.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Database geo-redundancy with MQ<\/em> <\/p>\n\n\n\n

The MQ solution sounds good, but there are other nuts to crack. Let\u2019s look at the steps for geo-redundancy with the MQ solution:<\/p>\n\n\n\n

    \n
  1. Break the target data into message queue payloads.<\/li>\n\n\n\n
  2. Feed the data to the MQ instance.<\/li>\n\n\n\n
  3. Read the data from the MQ instance.<\/li>\n\n\n\n
  4. Compose the payload to logical backup data.<\/li>\n\n\n\n
  5. Send the data to the remote storage service.<\/li>\n<\/ol>\n\n\n\n

    As you can see, you have to develop upstream and downstream data adapting components and host MQ instances. This adds to your operational and maintenance costs and requires extra domain expertise. <\/p>\n\n\n\n

    Is there a simpler, cheaper solution?<\/strong><\/h3>\n\n\n\n

    Based on the above solutions and the challenges with them, our requirements are clear: we want a geo-redundant solution that\u2019s scalable, simple, and less expensive. This naturally leads us to cloud services, which have a lower total cost of ownership, a simpler architecture, and easier maintenance. One option is a fully managed MQ service such as Confluent Cloud<\/a>. It could release you from the MQ processing procedures with a scalable service on the cloud, but that was only part of the issue. You still need to develop upstream and downstream adapting components. Actually, what we need is a Storage as a Service, or to be more precisely, Cross Region Replication as a Service (CRRaaS) solution. With CRRaaS, you can store data in one region and automatically replicate it to other ones.<\/p>\n\n\n\n

    \"\"<\/figure>\n\n\n\n

    Geo-redundancy with CRRaaS<\/em><\/p>\n\n\n\n

    <\/span>Comparing<\/strong> popular c<\/strong>loud solutions: S3 and GCS<\/strong><\/span><\/h2>\n\n\n\n

    Let\u2019s take a look at the most widely used object storage services from cloud vendors: Amazon Simple Storage Service (S3)<\/a> \uadf8\ub9ac\uace0 Google Cloud Storage<\/a> (GCS). They both have options for cross region replication. Moreover, most databases support backing up data to managed storage services, so you don\u2019t need any data adapting components. In this section, we will compare S3 and GCS and see if they can meet our goals for geo-redundant backup.<\/p>\n\n\n\n

    S3<\/strong><\/h3>\n\n\n\n

    The S3 replication<\/a> service includes cross-region replication. You can define replication rules between two storage buckets with versioning<\/a> enabled. Amazon S3 geo-redundant replication looks like this:<\/p>\n\n\n\n

    \"\"<\/figure>\n\n\n\n

    Geo-redundancy with Amazon S3<\/em><\/p>\n\n\n\n

    One Amazon Web Service (AWS) account can only support a maximum of 1,000 buckets. Therefore, when too many database instances require geo-redundant backup, you must define shared buckets among database instances. In addition, S3 lets you set the lifecycle configuration on a bucket for data retention to control cost.<\/p>\n\n\n\n

    \"\"<\/figure>\n\n\n\n

    <\/em>Shared buckets on Amazon S3<\/em><\/p>\n\n\n\n

    GCS<\/strong><\/h3>\n\n\n\n

    Instead of letting users define replication rules among storage buckets, GCS offers three bucket location types<\/a>: single-region, dual-region, and multi-region. You can select dual-region or multi-region buckets to get cross region replication. <\/p>\n\n\n\n

    \"\"<\/figure>\n\n\n\n

    Geo-redundancy with GCS<\/em><\/p>\n\n\n\n

    However, you can only scale dual-region buckets to two backup destinations. To achieve geo-redundancy across more than two destinations, you must choose multi-region buckets. <\/p>\n\n\n\n

    Comparison<\/strong><\/h3>\n\n\n\n

    Both S3 and GCS meet our requirements from a CRRaaS perspective, and each has their own unique attributes. This table compares both solutions. <\/p>\n\n\n\n

    Geo-redundancy option<\/strong><\/td>S3 replication<\/strong><\/td>GCS dual-region<\/strong><\/td>GCS multi-region<\/strong><\/td><\/tr>
    SLA<\/td>99.99% within 15 minutes<\/td>99.9% within 15 minutes (turbo replication<\/a> mode)<\/td>99.9% within 60 minutes<\/td><\/tr>
    Operation burden<\/td>Monitor replication performance
    Manage replication and lifecycle rules<\/td>    		Monitor replication performance<\/td>Monitor replication performance<\/td><\/tr>
    Scalability\/fault tolerance <\/td>Easily scale to any number of regions
    Can resist failure with N-1 regions <\/td>    		Data distributed in two fixed regions
    Can resist one-region failures<\/td>    		Data is distributed in two or more regions within the same continent
    Can resist failures with more than one regions<\/td><\/tr>
    Access latency<\/td>Low latency if you access data from the same region<\/td>Low latency if you access data from the same region <\/td>Unpredictable as data might not be stored in all regions, which is invisible to users.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

    <\/h2>\n\n\n\n

    S3 and GCS are managed storage services, so developers don\u2019t have to host replication infrastructure. Here are a few things to consider before you make a choice:<\/p>\n\n\n\n