Docker: You might loose your data if you do this mistake!

Docker is great, its container flexibility can give you the freedom to try many versions of the software you want without worrying about dependencies, or having to issues of “but works on my computer, i don't know why doesn't works on yours”, or leaving garbage's while installing / uninstalling different versions of the software.

But there is a newbie mistake that most people do when they start with Docker! and it can be a very dangerous mistake that might cause data loss!

Data persistence and Docker

Docker does not persist data if a container stop exists, note that many people got confused and believe that docker has data persistence without any configuration because they stop a container and they see that their data is still there when they restart the container, but in docker terms the container exists even if its stopped. A container stop exists when is deleted using the docker rm command.

Lets try to verify the above!

1. Create a docker container

$ docker run -it -d --name test01 ubuntu /bin/bash

run: starts a container
-i: keep STDIN of the container open, even if not attached
-t: allocate a pseudo-terminal
-d: start container de-attached
— name: give a name to the container
ubuntu: the image to use.
/bin/bash: start the container with process bash

so just remember, if you want to be able to attach to a de-attached container and type commands you need to start the container with the -i -t parameters.

2. Verify that the container runs

$ docker ps

It should produce an output similar to this

The STATUS column inform us how long the container is running, and the NAMES column the name of the container.

3. Attach to STDIN of the container

$ docker exec -ti test01 /bin/bash

4. Create a file and exit

Inside the container enter the following

# echo "hello world" >> ~/hello.txt
# exit

5. Stop the container

$ docker stop test01

6. Verify that the container stopped

$ docker ps -a

From the output we can see that the container is stopped, it has been exited since some seconds, but the point is to realize that the container still exists and the file hello.txt we created is still there

7. Restart the container

$ docker start test01

Verify if hello.txt still exists and has content

$ docker exec -ti test01 /bin/cat /root/hello.txt

It should produce the following output

hello world

8. Stop and delete the container

$ docker stop test01
$ docker rm test01

9. Verify that the container does not exist anymore

$ docker ps -a

We can see that the container is no more.

10. Recreate the container

$ docker run -it -d --name test01 ubuntu /bin/bash

11. Verify that hello.txt does not exist

Now we know that if we delete a container and stop the container the data is not lost after we start the container again, but if we stop and delete the container data is lost forever.

Why this is happening?

Despite it is logical that if you delete your container any saved data will be lost, its not for docker newbies, because it needs some time to realize that that the image that is used to create a container has not been updated with the data you created inside the container, so any containers based on the original image will not have any of the extra data that created on a running container of the same image.

How i can prevent this?

There are some ways to add data persistence even in the case that the container has been deleted, i will write an example only using bind mounts because i just tend to use it more often :)

What is a bind mount?

A bind mount is a host machine directory or file mounted inside a container, if the container is deleted the data of the container will be safe inside the specified host directory.

Example: create a container with binded volume

$ mkdir ~/myfiles
$ docker run -it -d -v ~/myfiles:/myfiles --name test01 ubuntu /bin/bash

With the mkdir command we created a directory named myfiles inside our home directory of the host.

All the parameters execpt the -v parameters are the same.

v: This parameter mounts the ~/myfiles directory of the host in the /myfiles directory of the container.

if i create a file inside the host ~/myfiles directory will be available in the container /myfiles directory?

On the host enter the following

$ echo "this is a test" >> ~/myfiles/test.txt

This command will create a file named test.txt inside the ~/myfiles directory with some text, lets verify if this file exists and is readable from the container mounted volume

$ docker exec -ti test01 /bin/cat /myfiles/test.txt

You should get the following output

this is a test

So we can see that even if the container is running we can read/write files from/to host/container.

if i delete the a file inside the /myfiles directory of the container will be dissapear from the host ~/myfiles directory?

On the host enter the following

$ docker exec -ti test01 /bin/rm /myfiles/test.txt

With this command we deleted the file from the container, lets see if the file exists on the host.

$ ls ~/myfiles/

It should not return test.txt. So any files inside the ~/mydata /mydata directories of the host and the container can manipulated from the host and the container.

if i delete the container and create a new container and bind the volume, the files will be available to the new container?

Lets stop and delete our containers to have a clean start for this example

$ docker stop test01
$ docker rm test01

Lets create some files inside the ~/mydata directory

$ touch "aaa" >> ~/myfiles/a.txt
$ touch "bbb" >> ~/myfiles/b.txt
$ touch "ccc" >> ~/myfiles/c.txt

Lets start a new container

$ docker run -it -d -v ~/myfiles:/myfiles --name test01 ubuntu /bin/bash

The files should be available from the container

$ docker exec -ti test01 /bin/ls /myfiles/
a.txt  b.txt  c.txt

Lets create a new file from the container

$ docker exec -ti test01 bash -c "echo 'ddd' >> /myfiles/e.txt"

Stop and Delete container test01

$ docker stop test01
$ docker rm test01

Create a new container and bind mount ~/mydata

$ docker run -it -d -v ~/myfiles:/myfiles --name test02 ubuntu /bin/bash

Verify that files are available for the container test02

$ docker exec -ti test02 /bin/ls /myfiles/
a.txt  b.txt  c.txt  e.txt

So we can see that using binded volumes the data persist even after a container that created the file does not exist no more.

Some useful notes on binded volumes

Backup often binded volumes, imagine the bad scenario that you use a new version of your container that does some transformation of your data but suddenly and you decide that you will not use the newer version, but instead you will use the older one, which cannot use the data anymore.
Its not a good idea to do write operations from the host or from another container while a container does some action on the data, you might end with corrupted files.

I hope you found the article easy and interesting :)