Running a Startup on AWS? Get Funding With AWS JumpStart. Click Here to Learn More

2021 Fillmore Street #1128

}

24/7 solutions

Troubleshooting and migrating your secured AWS DocumentDB inside Kubernetes

In this article, we explain a few tools to manage your AWS DocumentDB inside Kubernetes. 1) Launching a MongoDB client pod to connect to AWS DocumentDB Using the following command You will get shell access to a pod inside the k8s environment that will allow you to connect into the AWS DocumentDB database. As the […]

In this article, we explain a few tools to manage your AWS DocumentDB inside Kubernetes.

1) Launching a MongoDB client pod to connect to AWS DocumentDB

Using the following command

kubectl run -i --rm --tty mongo-client --image=mvertes/alpine-mongo --restart=Never --command -- /bin/bash

You will get shell access to a pod inside the k8s environment that will allow you to connect into the AWS DocumentDB database.

As the database requires SSL connection we need to download the public key from AWS using the following command:

wget http://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem

this will download the public key file into the current folder, next, if we run the following command from the same folder

mongo --ssl
    --host <hostname>
    --sslCAFile rds-combined-ca-bundle.pem
    --username <mongo-user>
    --password <mongo-password>

After running the previous command we should be connected to the AWS DocumentDB and see a prompt like this:

MongoDB shell version v3.4.2
connecting to: mongodb://<server>:27017/
MongoDB server version: 3.6.0
WARNING: shell and server versions do not match
rs0:PRIMARY> 

Then, we can just switch to the desired db using the following command:

rs0:PRIMARY> use <db-name>;
switched to db <dn-name>

At this point we can run any mongoldb cli command we need to see/troubleshoot data issues.

2) Launching a Mongo Dump client pod to to create a backup from DocumentDB

Using the following command

kubectl run -i --rm --tty mongodump-client --image=bigtruedata/mongodump --restart=Never --command -- /bin/sh

You will get shell access to a pod inside the k8s environment that will allow you to execute mongodump against the AWS DocumentDB database.

As the database requires SSL connection we need to download the public key from AWS using the following command:

wget http://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem

We are ready now to dump the database using the following command:

mongodump --ssl      --host <host>     --sslCAFile rds-combined-ca-bundle.pem     --username <db-user>     --password <db-password> --db <db-name>

At the end of the dump we will see a confirmation message like:
2019-08-14T20:22:46.547+0000 done dumping <db-name>.collectionX (12775 documents)

This should have created a new folder named dump/ with all the dump data. This means we are ready to create a tar/gz file with that folder using the following command:
tar -zcvf db-dump.tar.gz dump

Now we have the tar/gz file with all the information:

ls -ltrh *.tar.gz
-rw-r--r--    1 root     root        9.9M Aug 14 20:37 db-dump.tar.gz

Now without closing the current bash shell, open a new terminal tab and run the following command:

kubectl default/mongo-client3:/dump/db-dump.tar.gz /tmp 

This will copy the backup db-dump.tar.gz file into the tmp folder of the computer you are connected from tmp (hopefully still inside a secure network)

At this point, we can run the same cp command to copy to your second k8s environment, but we need to first launch a pod instance of the mongodump pod to be able to do a mongorestore so let’s launch it.

Finally, we can exit the mongodump pod and it will be automatically deleted.

NOTE: At this point you should switch kubectl to point to the destination k8s cluster where you want your backup to be restored.

3) Launching a Mongo Dump client pod to restore the backup from DocumentDB

Using the following command we will launch a pod that will be used to restore the database.

kubectl run -i --rm --tty mongorestore-client --image=bigtruedata/mongodump --restart=Never --command -- /bin/sh

You will get shell access to a pod inside the k8s environment that will allow you to execute mongorestore against the AWS DocumentDB database.

As the database requires SSL connection we need to download the public key from AWS using the following command:

wget http://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem

Open a new terminal tab to copy the backup file into the mongorestore pod by running:

kubectl cp /tmp/db-dump.tar.gz default/mongorestore-client:/dump

After it finishes uploading the file into our pod we can now delete our local copy of the backup.

Now, we need to extract the tar/gz file using the following command

tar xvzf db-dump.tar.gz

This will extract all the contents of the tar/gz file into our dump/ folder. We can check by running:

 ls -ltrh
total 10120
-rw-r--r--    1 501      wheel       9.9M Aug 14 20:44 db-dump.tar.gz
drwxr-xr-x    3 root     root          29 Aug 14 21:00 dump

Finally we can run the restore:

mongorestore dump/ --ssl  --host <host>  --sslCAFile rds-combined-ca-bundle.pem  --username <db-user> --password <db-password> --db <db-name>

At this point our database should have been fully restored into our new k8s environment. You can refer to the first section fo this document to connect to your AWS DocumentDB and see your data.

allcode_admin
allcode_admin

Related Articles

Here’s Why You Should Work with an AWS Partner

Here’s Why You Should Work with an AWS Partner

Amazon Web Services is understandably a difficult platform to adapt to and utilize fully upon first getting started. Some organizations can be selected to become certified partners to indirectly extend services to help build on the Amazon Cloud. Finding a certified company to help build out is undoubtedly the best way to significantly simplify, streamline, and reduce the cost of utilizing AWS.

Amazon Web Services – CodeCatalyst

Amazon Web Services – CodeCatalyst

When a development team is building out an application, it helps to have access to the same resources, have the tools for planning and testing, and to have access to the application all in one place. CodeCatalyst comes with a slew of continuous integration/continuous development (CI/CD) tools and can leverage other AWS services and be connected to other AWS projects on an account. As a collaborative tool, it is easy to introduce new members into the project and to log all activity or all tests from a single dashboard. It’s a complete package of all the tools needed to securely work on every step of an application’s lifecycle.

The Definitive Guide to AWS Pricing

The Definitive Guide to AWS Pricing

Perhaps the biggest issue with AWS that its competitors edge out on is the confusing pricing model. It does promise the capacity to help users save significantly on funds that otherwise by avoiding spending on unnecessary resources, but getting to that point isn’t always clear. We will be covering in greater detail how this works.

Download our 10-Step Cloud Migration ChecklistYou'll get direct access to our full-length guide on Google Docs. From here, you will be able to make a copy, download the content, and share it with your team.