Amazon EBS: Detachable Persistent Data Storage

Xia wu hao! 🙂

Hi reader, a belated Happy Chinese New Year to you! 🙂 and a Happy Valentine’s day as well! ❤

For this blog post, we will be talking about the Amazon Elastic Block Store (EBS). This is actually a topic I came into late last December and actually came very handy early this January  when I needed to setup a data environment with a copy of our latest database. It was a perfect timing, indeed! So let’s start rolling:

Amazon Elastic Block Store (EBS) provides persistent block level storage volumes that can act like hard drives or other storage devices to your EC2 instances. EBS volumes can be attached to instances, detached from one instance, and even re-attached to another.

EBS volumes are also automatically replicated in its availability zone to protect from component failures and provide high availability and durability. EBS snapshots could also be taken to replicate data either for attachment to another instance or for safekeeping as a backup. These snapshots are stored durably in Amazon S3.

In this blogpost, we will see how we can create volumes, attach them to instances, make them available for storage, detach them, reattach them to other instances and finally create snapshots.

I. Creating an EBS Volume

To start off, let us create an EBS volume. Go to the AWS console and then to the EC2 page. On the left pane, under Elastic Block Store, click Volumes. And then click Create Volume.

Screen Shot 2015-12-26 at 12.19.19 PM.png

For this purpose, I’ll create a 1 GB volume in us-west-2a availability zone. You may choose a different size or availability zone, but just remember that to be able to attach a volume to an EC2 instance, they must exist in the same availability zone.

II. Attaching an EBS Volume to an EC2 Instance

To attach an EBS volume to an EC2 instance, ensure that they belong to the same availability zone. Select the available EBS volume that you want to attach.

Screen Shot 2015-12-26 at 12.20.53 PM.png

And then choose the instance unto which you want to attach it:

Screen Shot 2015-12-26 at 12.21.11 PM.png

Once you gave chosen the instance, you may also rename the device. But read the warning: the device name you enter here might not be the one used by the OS. 🙂 But don’t worry as there’s usually a pattern how device names are renamed and it won’t be difficult to figure out which is your newly mounted device out among all the available devices in your instance.

Screen Shot 2015-12-26 at 12.21.36 PM.png

III. Mounting an EBS Volume to your EC2 Instance

Great! Now that we have attached the volume, it is now ready for use. But oops, not too fast! We first need to mount it and make it a part of our available file system.

You may issue the following command to see which file storages are currently mounted and the respective usage for each:

df

and the following command to list all available blocks (either mounted or not):

lsblk

From df’s results, we can see that our newly attached EBS is not included yet (since it’s not yet mounted). On the other hand, from lsblk’s results, we saw /dev/xvdf, our newly attached EBS but with an empty mount point.

Now let’s proceed to mounting our EBS,

First check if a file system is already available to our EBS as we need one before we can use it. Issuing the following command,

 

sudo file -s /dev/xvdf

we get ‘data’ as a result. This means that there is no file system installed yet. Don’t worry as this is normal since we have just created our EBS volume.

If you are using an existing EBS volume which already has a file system, skip the following step (unless you really want to override the current file system and format your block):

sudo mkfs -t ext4 /dev/xvdf

Once we have created a file system in our EBS, let us now mount it.

1. We first create our mount point:

Command format: sudo mkdir mount_point

sudo mkdir data-adelen

2. And then proceed mounting our EBS. Remember from above that our device_name is /dev/xvdf:

Command format: sudo mount device_name mount_point

sudo mount /dev/xvdf data-adelen

When we issue the df command again, we can now see /dev/xvdf part of our available disks.

Let us try adding contents to our EBS. To make our EBS writable by even non root users, we can change permissions and owner

cd /data-adelen chmod 777
sudo chown ubuntu:ubuntu -R  .

And then proceed to creating text files and executable files.

Screen Shot 2015-12-26 at 12.29.50 PM.png

Before we proceed, let us first check the persistence of our mounted device by turning of our instance and then turning it on again.

Persistence Check:

a. On our Instances page, right click and choose Stop on the EC2 instance unto which we attached our EBS earlier:

Screen Shot 2015-12-26 at 12.33.14 PM.png

 

b. After it has successfully stopped, start your instance again.

c. Upon a successful reboot, when we issue a df, we see that our EBS is gone (it’s no longer mounted!). Why is this so? – The mount we made earlier was not persistent.

Screen Shot 2015-12-26 at 12.40.23 PM.png

To make our mounted device persistent (i.e. our EBS volume is mounted on every system reboot), we can add an entry for the device to the/etc/fstab file.

* Fun fact: fstab stands for File Systems Table

Create a backup of your /etc/fstab file where you can refer to in case you accidentally destroy or delete your actual /etc/fstab while editing it.

sudo cp /etc/fstab /etc/fstab.orig

We now have our backup, let us proceed in editing it and adding the additional lines that mount our device on every system reboot:

sudo vim /etc/fstab

Add a new line to the end of the file for your volume using the following format.

device_name mount_point file_system_type fs_mntops fs_freq fs_passno

From our example above, the corresponding line to add is:

/dev/xvdf       /data   ext4    defaults,nofail        0       2

nobootwait

After adding, we then save and exit from the file. To test if our configuration file is still valid, we can do:

sudo mount -a

If the above command did not produce any errors, then great! Else, try to find the cause of the error. Note that errors on our etc/fstab file can render our system unbootable so it’s important to solve first the existing problems before shutting down your instance.

After we have edited our /etc/fstab file and ensured that it does not produce any errors, we can do the same persistence check again by stopping and starting our instance:  Screen Shot 2015-12-26 at 12.48.46 PM.png

Upon system reboot:Screen Shot 2015-12-26 at 12.52.51 PM.png

When we issue df right after system boot, we see /dev/xvdf available and mounted at /data-adelen.

IV. Detaching your EBS Volume from your EC2 Instance

In case we want to detach our EBS volume, we need to unmount first our EBS volume from our EC2 instance.

sudo umount -d /dev/xvdf

When you get an error that says your device is busy, you may issue the following command (only if it’s okay to force kill all process using your device). Basically what it does is it finds and kills all process running with an open file in /dev/xvdf/

sudo fuser -km /dev/xvdf sudo umount -d /dev/xvdf

Once you have successfully issued the umount command, you may verify that your device is no longer mounted by the df command.

V. Re-Attaching your EBS Volume to another EC2 Instance

Say we have another Amazon Linux instance where we want to re-attach our EBS volume to:

Screen Shot 2015-12-26 at 12.11.41 PM.png

Oh, it’s a different OS? No problem!

As you’ll see in this section, EBS attachments are OS independent. We initialiiy attahced our EBS volume to an Ubuntu 14.04 instance and as we’ll see, we’ll now attach it to a Amazon Linux instance.

From the above df, we saw that the EBS volume is not yet mounted. And doing a lsblk, we can see that xvdf is available but has an empty mount point.

Screen Shot 2015-12-26 at 12.12.11 PM.png

Checking for the file system of /dev/xvdf, we see ext4, the one that we installed a while ago.

Screen Shot 2015-12-26 at 12.13.10 PM.png

Now let’s proceed to creating our mountpoint and eventually mounting our EBS Volume:

Screen Shot 2015-12-26 at 12.14.50 PM.png

Doing a df, we can now see it available and mounted at /adelen-mount-point-2.

Screen Shot 2015-12-26 at 12.15.38 PM.png

Looking at the contents, we see that the files we have created and saved in our Ubuntu instance is also available here.

Screen Shot 2015-12-26 at 12.16.41 PM.png

 

VI. Creating a Snapshot of your EBS Volume

In cases we want to create snapshots to replicate our data, AMazon EBS provides an easy way to do so!

Just go to the Volumes section of the EC2 page, and then cright click on your target volume and choose Create Snapshot.

Screen Shot 2015-12-26 at 12.56.33 PM.png

From the same left panel, you may see the avaialble snapshots from the Snapshots option under Elastic Block Store:

Screen Shot 2015-12-26 at 12.57.44 PM.png

VII.Creating a Volume from EBS Volume Snapshot

So on to the last section of this blog post. A while ago we saw how to create snapshots of our EBS volume. And now, we’ll be looking into how to create another volume from that snapshot.

So from the Snapshots page, right click on the snapshot where you want to create an EBS volume from. We can see that the minimum size you can allocate is the size of the volume where the snapshot is taken from.

Screen Shot 2015-12-26 at 12.57.59 PM.png

You may also change the availability zone and other defaults. And then once you are satisfied with the settings, go click Create. From there, you may also see the progress of creating your EBS volume. Once it’s available, you can now attach it and use like any other normal EBS volumes.

Screen Shot 2015-12-26 at 12.59.11 PM.png

So there, a really quick post on Amazon Elastic Block Store (EBS). 🙂 Hope this post was helpful. Surely, in one of your current and future projects, you might encounter Amazon EBS especially when dealing with a large sized files – OS, databases, and the likes. 🙂

Happy Valentine’s Day again and never forget to always keep your heart happy! 

Sources:

Advertisements