Xia wu hao! 🙂
Hi reader, a belated Happy Chinese New Year to you! 🙂 and a Happy Valentine’s day as well!
For this blog post, we will be talking about the Amazon Elastic Block Store (EBS). This is actually a topic I came into late last December and actually came very handy early this January when I needed to setup a data environment with a copy of our latest database. It was a perfect timing, indeed! So let’s start rolling:
Amazon Elastic Block Store (EBS) provides persistent block level storage volumes that can act like hard drives or other storage devices to your EC2 instances. EBS volumes can be attached to instances, detached from one instance, and even re-attached to another.
EBS volumes are also automatically replicated in its availability zone to protect from component failures and provide high availability and durability. EBS snapshots could also be taken to replicate data either for attachment to another instance or for safekeeping as a backup. These snapshots are stored durably in Amazon S3.
In this blogpost, we will see how we can create volumes, attach them to instances, make them available for storage, detach them, reattach them to other instances and finally create snapshots.
I. Creating an EBS Volume
To start off, let us create an EBS volume. Go to the AWS console and then to the EC2 page. On the left pane, under Elastic Block Store, click Volumes. And then click Create Volume.
For this purpose, I’ll create a 1 GB volume in us-west-2a availability zone. You may choose a different size or availability zone, but just remember that to be able to attach a volume to an EC2 instance, they must exist in the same availability zone.
II. Attaching an EBS Volume to an EC2 Instance
To attach an EBS volume to an EC2 instance, ensure that they belong to the same availability zone. Select the available EBS volume that you want to attach.
And then choose the instance unto which you want to attach it:
Once you gave chosen the instance, you may also rename the device. But read the warning: the device name you enter here might not be the one used by the OS. 🙂 But don’t worry as there’s usually a pattern how device names are renamed and it won’t be difficult to figure out which is your newly mounted device out among all the available devices in your instance.
III. Mounting an EBS Volume to your EC2 Instance
Great! Now that we have attached the volume, it is now ready for use. But oops, not too fast! We first need to mount it and make it a part of our available file system.
You may issue the following command to see which file storages are currently mounted and the respective usage for each:
df
and the following command to list all available blocks (either mounted or not):
lsblk
From df’s results, we can see that our newly attached EBS is not included yet (since it’s not yet mounted). On the other hand, from lsblk’s results, we saw /dev/xvdf, our newly attached EBS but with an empty mount point.
Now let’s proceed to mounting our EBS,
First check if a file system is already available to our EBS as we need one before we can use it. Issuing the following command,
sudo file -s /dev/xvdf
we get ‘data’ as a result. This means that there is no file system installed yet. Don’t worry as this is normal since we have just created our EBS volume.
If you are using an existing EBS volume which already has a file system, skip the following step (unless you really want to override the current file system and format your block):
sudo mkfs -t ext4 /dev/xvdf
Once we have created a file system in our EBS, let us now mount it.
1. We first create our mount point:
Command format: sudo mkdir mount_point
sudo mkdir data-adelen
2. And then proceed mounting our EBS. Remember from above that our device_name is /dev/xvdf:
Command format: sudo mount device_name mount_point
sudo mount /dev/xvdf data-adelen
When we issue the df command again, we can now see /dev/xvdf part of our available disks.
Let us try adding contents to our EBS. To make our EBS writable by even non root users, we can change permissions and owner
cd /data-adelen chmod 777
sudo chown ubuntu:ubuntu -R .
And then proceed to creating text files and executable files.
Before we proceed, let us first check the persistence of our mounted device by turning of our instance and then turning it on again.
Persistence Check:
a. On our Instances page, right click and choose Stop on the EC2 instance unto which we attached our EBS earlier:
b. After it has successfully stopped, start your instance again.
c. Upon a successful reboot, when we issue a df, we see that our EBS is gone (it’s no longer mounted!). Why is this so? – The mount we made earlier was not persistent.
To make our mounted device persistent (i.e. our EBS volume is mounted on every system reboot), we can add an entry for the device to the/etc/fstab
file.
* Fun fact: fstab stands for File Systems Table
Create a backup of your /etc/fstab
file where you can refer to in case you accidentally destroy or delete your actual /etc/fstab while editing it.
sudo cp /etc/fstab /etc/fstab.orig
We now have our backup, let us proceed in editing it and adding the additional lines that mount our device on every system reboot:
sudo vim /etc/fstab
Add a new line to the end of the file for your volume using the following format.
device_name
mount_point
file_system_type
fs_mntops
fs_freq
fs_passno
From our example above, the corresponding line to add is:
/dev/xvdf /data ext4 defaults,nofail 0 2
nobootwait
After adding, we then save and exit from the file. To test if our configuration file is still valid, we can do:
sudo mount -a
If the above command did not produce any errors, then great! Else, try to find the cause of the error. Note that errors on our etc/fstab file can render our system unbootable so it’s important to solve first the existing problems before shutting down your instance.
After we have edited our /etc/fstab file and ensured that it does not produce any errors, we can do the same persistence check again by stopping and starting our instance:
Upon system reboot:
When we issue df right after system boot, we see /dev/xvdf available and mounted at /data-adelen.
IV. Detaching your EBS Volume from your EC2 Instance
In case we want to detach our EBS volume, we need to unmount first our EBS volume from our EC2 instance.
sudo umount -d /dev/xvdf
When you get an error that says your device is busy, you may issue the following command (only if it’s okay to force kill all process using your device). Basically what it does is it finds and kills all process running with an open file in /dev/xvdf/
sudo fuser -km /dev/xvdf sudo umount -d
/dev/xvdf
Once you have successfully issued the umount command, you may verify that your device is no longer mounted by the df command.
V. Re-Attaching your EBS Volume to another EC2 Instance
Say we have another Amazon Linux instance where we want to re-attach our EBS volume to:
Oh, it’s a different OS? No problem!
As you’ll see in this section, EBS attachments are OS independent. We initialiiy attahced our EBS volume to an Ubuntu 14.04 instance and as we’ll see, we’ll now attach it to a Amazon Linux instance.
From the above df, we saw that the EBS volume is not yet mounted. And doing a lsblk, we can see that xvdf is available but has an empty mount point.
Checking for the file system of /dev/xvdf, we see ext4, the one that we installed a while ago.
Now let’s proceed to creating our mountpoint and eventually mounting our EBS Volume:
Doing a df, we can now see it available and mounted at /adelen-mount-point-2.
Looking at the contents, we see that the files we have created and saved in our Ubuntu instance is also available here.
VI. Creating a Snapshot of your EBS Volume
In cases we want to create snapshots to replicate our data, AMazon EBS provides an easy way to do so!
Just go to the Volumes section of the EC2 page, and then cright click on your target volume and choose Create Snapshot.
From the same left panel, you may see the avaialble snapshots from the Snapshots option under Elastic Block Store:
VII.Creating a Volume from EBS Volume Snapshot
So on to the last section of this blog post. A while ago we saw how to create snapshots of our EBS volume. And now, we’ll be looking into how to create another volume from that snapshot.
So from the Snapshots page, right click on the snapshot where you want to create an EBS volume from. We can see that the minimum size you can allocate is the size of the volume where the snapshot is taken from.
You may also change the availability zone and other defaults. And then once you are satisfied with the settings, go click Create. From there, you may also see the progress of creating your EBS volume. Once it’s available, you can now attach it and use like any other normal EBS volumes.
So there, a really quick post on Amazon Elastic Block Store (EBS). 🙂 Hope this post was helpful. Surely, in one of your current and future projects, you might encounter Amazon EBS especially when dealing with a large sized files – OS, databases, and the likes. 🙂
Happy Valentine’s Day again and never forget to always keep your heart happy!
Sources:
Pingback: Restoring MongoDB from a Dump File in AWS S3 | blog()