As businesses and individuals continue to generate vast amounts of data, the need for efficient and reliable cloud storage solutions has become increasingly important. Amazon S3 has long been a popular choice for many, offering scalability, durability, and accessibility However, as the amount of data is growing, more and more people are willing to move to cheaper S3-compatible cloud storage providers. Contabo Object Storage is one such provider, offering an affordable and flexible alternative to Amazon S3. In this article, we will guide you through the process of migrating from Amazon S3 to Contabo Object Storage, outlining the steps you need to take to ensure a smooth transition without any data loss or downtime. Whether you are a small business or a large corporation, this tutorial will help you make an informed decision and enable you to take advantage of the benefits offered by Contabo Object Storage.
Prerequisites/Requirements
In order to flawlessly migrate from Amazon S3 to Contabo Object Storage a few things are required:
- An Amazon S3 Storge + Credentials (e.g. S3 URL, Access Key and Secret Key)
- A Contabo Object Storage + Credentials (e.g. S3 URL, Access Key and Secret Key)
- A Server with a Linux-based operating system (WSL would be enough)
- Rclone installed on your Linux-server
Hint: To install rclone use this command:
apt install rclone –y Configure rclone
To transfer the files from Amazon S3 to Contabo Object Storage we are going to use the tool rclone.
If you want to learn more about rclone and other tools that work with Contabo Object Storage check out our official documentation here.
After installing it on our Linux-server, we need to configure the source-server, as well as the destination server. Although your own local PC with WSL on it or a linux workstation could do the job, we recommend using a VPS (e.g. our VPS S SSD) because the migration process might take a while depending on the size of the files to copy. It’s crucial to execute the migration process uninterrupted.
Config for Amazon S3 (Source)
First, we are going to start with the source server, which in this case is the Amazon S3 storage:
To create a new config run this command:
rcone config- Now type in “n” to make a new remote and press [Enter]
- Now enter a name for this config (e.g. Amazon S3 or source).
- In our example I will use “source” as the name.
- Now type in “4” to select “Amazon S3 Compliant Storage Providers” and press [Enter]
- Type “1” to select “Amazon Web Services (AWS) S3”
- Again, type in “1” to enter your AWS Credentials
- Now fill in your S3-Access Key ID and press [Enter]
- Now paste in your S3-Secret Access Key
- Skip the next menus by pressing [Enter] until it asks you “Edit advanced config?”
- Now type in “n” and press [Enter]
- To finish the setup process, save everything by typing in “y” and hitting [Enter] and typing in “q” to leave to configuration menu.
Config for Contabo Object Storage (Destination)
Now we can create a second config for our destination server, which in this case will be our Contabo Object Storage.
To create this config, again run this command:
rcone config- Now type in “n” to make a new remote and press [Enter]
- Now enter a name for this config (e.g. Contabo S3 or destination).
- In our example I will use “dest” as the name.
- Now type in “4” to select “Amazon S3 Compliant Storage Providers” and press [Enter]
- In the next menu type “2” to select “Ceph Object Storage”
- Type in “1” to enter your AWS Credentials
- Now enter your Access Key and hit [Enter]
- Now enter your Secret Key and hit [Enter]
- Skip the region by hitting [Enter]
- For Endpoint paste in the S3-URL (e.g. https://eu2.contabostorage.com)
- Skip the next two menus by hitting [Enter] two times
- Now type in “n” to skip the advanced config
- Hit “y” to save the config and exit the config menu by typing in “q”
Start the migration process
Now after setting up both configs it’s finally time to start the actual migration process.
This can be done with one single command
The scheme for this command is:
rclone copy [Name of the Source config]:[Path] [Name of the Destination Config]:[Path]In our example the full command would be:
rclone copy source:source dest:destExplanation:
The first “source” is the name of the config we’ve just created and I named “source”. The second “source” is the name of the bucket I want copy.
The first “dest” is again the name of the second config we’ve just created and I named “dest”. The second “dest” is again the name of the bucket where I want my files to be copied to.
Depending on the size of your Bucket this process can take some time.
If you want to migrate the whole content at once and not just only a certain bucket, use this command instead:
rclone copy source:~ dest:~Verify the Migration Process
Please make sure to check if everything has been copied correctly bevor cancelling/deleting your Amazon S3 Storage!
This can be done in many different ways. You could for example compare the file counts of the source-server with the destination-server. To do so, use these commands:
For Amazon S3 (Source-Server):
rclone ls s3:~ --recursive | wc -lFor Contabo Object Storage (Destination-Server):
rclone ls contabo:~ --recursive | wc -lCongrats! You’ve successfully migrated from Amazon S3 to Contabo Object Storage.