Contents
Today, we are going to learn how to setup an environment with the awesome tool that any DevOps should know, Vagrant.
Behind this name is a wonderful software which automates the use of virtual machines (VMs) to create reproducible environments on-the-fly.
By creating a simple configuration file, within minutes you will have a fresh virtual machine started and provisioning without any other human interaction. The Vagrant Dev Environment!
Since Vagrant is not a pure virtualization tool by itself, it makes use of providers to do the virtualization such as VMWare, VirtualBox, etc…
And what a better way to demonstrate the power of this tool if not by setting up a distributed file system!
And this is a good thing because RozoFS is exactly what we are going to need. RozoFS is a high performance and scalable open source distributed file system.
You may take a look at what they do, it’s pretty interesting! http://rozosystems.com/
This demonstration will go as follows: first we will setup the RozoFS cluster, then we will run some rozo commands to show how it works.
In this tutorial, I will suppose that Vagrant and VirtualBox are already installed in your computer, if not here are the download links:
That being said, let’s get to the heart of the matter!
The Vagrant file
The Vagrant file will be the heart of our environment. It contains all the instructions to setup our system.
So to start, create a text file named Vagrantfile and copy paste the code below:
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
file_to_disk = "./tmp/large_disk_rozo01.vdi"
node.vm.box = "bento/ubuntu-14.04"
node.vm.hostname = "rozo01"
node.vm.network "private_network", ip: "10.0.42.11"
node.vm.provider :virtualbox do |vb|
vb.cpus = 2
vb.memory = 2048
vb.customize ['createhd', '--filename', file_to_disk, '--size', 50 * 1024]
vb.customize ['storageattach', :id, '--storagectl', 'SATA Controller', '--port', 1, '--device', 0, '--type', 'hdd', '--medium', file_to_disk]
end
end
Basically, here we have defined all the basic stuff for a VM : Operating System, hostname, IP, disk and the provider (VirtualBox in our case).
But this will create only one VM, and it’s not really what we need. Since we are building a cluster, we will need more machines, so let’s modify this Vagrantfile to look more like our goal.
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
(2..4).each do |i|
file_to_disk = "./tmp/large_disk_rozo0#{i}.vdi"
config.vm.define "rozo0#{i}" do |node|
node.vm.box = "bento/ubuntu-14.04"
node.vm.hostname = "rozo0#{i}"
node.vm.network "private_network", ip: "10.0.42.1#{i}"
node.vm.provider :virtualbox do |vb|
vb.cpus = 2
vb.memory = 2048
unless File.exist?(file_to_disk)
vb.customize ['createhd', '--filename', file_to_disk, '--size', 50 * 1024]
end
vb.customize ['storageattach', :id, '--storagectl', 'SATA Controller', '--port', 1, '--device', 0, '--type', 'hdd', '--medium', file_to_disk]
end
end
end
end
So now it will deploy 3 VM’s with the same configuration except for the hostname and IP.
Then we will need to execute some shell commands into each VM. So for that, there is a vagrant instruction that allows us to run a shell command just after the VM creation;
node.vm.provision "shell", inline:
The shell commands that we need to run will install and configure everything for RozoFS.
Let’s see now how our Vagrantfile looks.
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
(2..4).each do |i|
file_to_disk = "./tmp/large_disk_rozo0#{i}.vdi"
config.vm.define "rozo0#{i}" do |node|
node.vm.box = "bento/ubuntu-14.04"
node.vm.hostname = "rozo0#{i}"
node.vm.network "private_network", ip: "10.0.42.1#{i}"
node.vm.provider :virtualbox do |vb|
vb.cpus = 2
vb.memory = 2048
unless File.exist?(file_to_disk)
vb.customize ['createhd', '--filename', file_to_disk, '--size', 50 * 1024]
end
vb.customize ['storageattach', :id, '--storagectl', 'SATA Controller', '--port', 1, '--device', 0, '--type', 'hdd', '--medium', file_to_disk]
end
node.vm.provision "shell", inline: <<-SHELL
wget -O - http://dl.rozofs.org/deb/devel@rozofs.com.gpg.key | apt-key add -
echo deb http://dl.rozofs.org/deb/master $(lsb_release -sc) main | tee /etc/apt/sources.list.d/rozofs.list
apt-get update
apt-get install -y rozofs-storaged
apt-get install -y rozofs-exportd
apt-get install -y rozofs-rozofsmount
apt-get install -y rozofs-manager-lib
apt-get install -y rozofs-manager-cli
apt-get install -y rozofs-manager-agent
apt-get install -y rozofs-rozodiag
apt-get install -y nagios-plugins-rozofs
apt-get install -y vim
mkdir -p /srv/rozofs/storages/storage_1_#{i}
mkfs.ext4 /dev/sdb -F
mount /dev/sdb /srv/rozofs/storages/storage_1_#{i}
/etc/init.d/rozofs-storaged restart
cp /vagrant/storage.conf /etc/rozofs/storage.conf
sed -i \'s/storage_1_1/storage_1_#{i}/\' /etc/rozofs/storage.conf
sed -i \'s/sid = 1/sid = #{i}/\' /etc/rozofs/storage.conf
/etc/init.d/rozofs-storaged restart
SHELL
end
end
end
Finally we need to declare one more VM for our master rozo node. The reason why it’s not included in the first loop is because this host needs a specific shell command to run in it.
So here is the final Vagrantfile:
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
(2..4).each do |i|
file_to_disk = "./tmp/large_disk_rozo0#{i}.vdi"
config.vm.define "rozo0#{i}" do |node|
node.vm.box = "bento/ubuntu-14.04"
node.vm.hostname = "rozo0#{i}"
node.vm.network "private_network", ip: "10.0.42.1#{i}"
node.vm.provider :virtualbox do |vb|
vb.cpus = 2
vb.memory = 2048
unless File.exist?(file_to_disk)
vb.customize ['createhd', '--filename', file_to_disk, '--size', 50 * 1024]
end
vb.customize ['storageattach', :id, '--storagectl', 'SATA Controller', '--port', 1, '--device', 0, '--type', 'hdd', '--medium', file_to_disk]
end
node.vm.provision "shell", inline: <<-SHELL
wget -O - http://dl.rozofs.org/deb/devel@rozofs.com.gpg.key | apt-key add -
echo deb http://dl.rozofs.org/deb/master $(lsb_release -sc) main | tee /etc/apt/sources.list.d/rozofs.list
apt-get update
apt-get install -y rozofs-storaged
apt-get install -y rozofs-exportd
apt-get install -y rozofs-rozofsmount
apt-get install -y rozofs-manager-lib
apt-get install -y rozofs-manager-cli
apt-get install -y rozofs-manager-agent
apt-get install -y rozofs-rozodiag
apt-get install -y nagios-plugins-rozofs
apt-get install -y vim
mkdir -p /srv/rozofs/storages/storage_1_#{i}
mkfs.ext4 /dev/sdb -F
mount /dev/sdb /srv/rozofs/storages/storage_1_#{i}
/etc/init.d/rozofs-storaged restart
cp /vagrant/storage.conf /etc/rozofs/storage.conf
sed -i \'s/storage_1_1/storage_1_#{i}/\' /etc/rozofs/storage.conf
sed -i \'s/sid = 1/sid = #{i}/\' /etc/rozofs/storage.conf
/etc/init.d/rozofs-storaged restart
SHELL
end
end
config.vm.define "rozomds" do |rozomds|
rozomds.vm.box = "bento/ubuntu-14.04"
rozomds.vm.hostname = "rozomds"
rozomds.vm.network "private_network", ip: "10.0.42.11"
file_to_disk = "./tmp/large_disk_rozo.vdi"
file_to_disk_export = "./tmp/export_disk.vdi"
rozomds.vm.provider :virtualbox do |vb|
vb.cpus = 2
vb.memory = 2048
unless File.exist?(file_to_disk)
vb.customize ['createhd', '--filename', file_to_disk, '--size', 50 * 1024]
end
vb.customize ['storageattach', :id, '--storagectl', 'SATA Controller', '--port', 1, '--device', 0, '--type', 'hdd', '--medium', file_to_disk]
unless File.exist?(file_to_disk_export)
vb.customize ['createhd', '--filename', file_to_disk_export, '--size', 50 * 1024]
end
vb.customize ['storageattach', :id, '--storagectl', 'SATA Controller', '--port', 2, '--device', 0, '--type', 'hdd', '--medium', file_to_disk_export]
end
rozomds.vm.provision "shell", inline: <<-SHELL
wget -O - http://dl.rozofs.org/deb/devel@rozofs.com.gpg.key | apt-key add -
echo deb http://dl.rozofs.org/deb/master $(lsb_release -sc) main | tee /etc/apt/sources.list.d/rozofs.list
apt-get update
apt-get install -y rozofs-storaged
apt-get install -y rozofs-exportd
apt-get install -y rozofs-rozofsmount
apt-get install -y rozofs-manager-lib
apt-get install -y rozofs-manager-cli
apt-get install -y rozofs-manager-agent
apt-get install -y rozofs-rozodiag
apt-get install -y nagios-plugins-rozofs
apt-get install -y vim
mkdir -p /srv/rozofs/storages/storage_1_1
mkfs.ext4 /dev/sdb -F
mount /dev/sdb /srv/rozofs/storages/storage_1_1
/etc/init.d/rozofs-storaged restart
cp /vagrant/storage.conf /etc/rozofs/storage.conf
sed -i \'s/storage_1_1/storage_1_1/\' /etc/rozofs/storage.conf
sed -i \'s/sid = 1/sid = 1/\' /etc/rozofs/storage.conf
/etc/init.d/rozofs-storaged restart
rozo agent restart
rozo volume expand 10.0.42.11 10.0.42.12 10.0.42.13 10.0.42.14 -E 10.0.42.11
sleep 20
rozo export create 1 -E 10.0.42.11
sleep 20
rozo mount create -E 10.0.42.11
SHELL
end
config.vm.post_up_message = "
vagrant ssh rozomds
vagrant ssh rozo02
vagrant ssh rozo03
vagrant ssh rozo04"
end
If you want more details on a specific command, you may check Vagrant documentation, you will surely find what you want : https://www.vagrantup.com/docs/
Now, from where the Vagrantfile is located, run the following command:
vagrant up
It will execute all the instructions of the Vagrantfile and once it finishes you will get a fresh environment with RozoFS setup in it.
So now, let’s run some RozoFS commands!
Test the environment with RozoFS commands
Like many cluster architectures, there is a master server. It will be from the master server that the RozoFS commands will be run.
So to connect to this master, named rozomds (according to our Vagrantfile), run the vagrant command:
vagrant ssh rozomds
Now we are inside our Vagrant box. From there, we are going to create an export and with one command we’ll mount it on our cluster.
First, let’s see if we already have an export. Run the following command:
root@rozomds:~# rozo node status -E 10.0.42.11 10.0.42.12: - STORAGED: running - ROZOFSMOUNT: - /mnt/rozofs@10.0.42.11/export_1: mounted 10.0.42.13: - STORAGED: running - ROZOFSMOUNT: - /mnt/rozofs@10.0.42.11/export_1: mounted 10.0.42.11: - EXPORTD: running - STORAGED: running - ROZOFSMOUNT: - /mnt/rozofs@10.0.42.11/export_1: mounted 10.0.42.14: - STORAGED: running - ROZOFSMOUNT: - /mnt/rozofs@10.0.42.11/export_1: mounted
As you can see, we already have an export named export_1 and it’s already mounted on all our vagrant hosts.
The other command to see all your exports with more details is:
root@rozomds:~# rozo export get -E 10.0.42.11 EXPORTS: - EXPORT 1: - vid: 1 - root: /srv/rozofs/exports/export_1 - md5: '' - squota: '' - hquota: ''
Note that the rozo command has a very detailed manual and it could be very useful! man rozo
Now we want to create a new export. It’s as simple as following:
root@rozomds:~# rozo export create -n export_2 1 -E 10.0.42.11
Run the command again to get our exports:
root@rozomds:~# rozo export get -E 10.0.42.11 EXPORTS: - EXPORT 1: - vid: 1 - root: /srv/rozofs/exports/export_1 - md5: '' - squota: '' - hquota: '' - EXPORT 2: - vid: 1 - root: /srv/rozofs/exports/export_2 - md5: '' - squota: '' - hquota: ''
Now we mount our export:
root@rozomds:~# rozo mount create -e export_2 -E 10.0.42.11
10.0.42.12:
- export export_2 (eid=2) on /mnt/rozofs@10.0.42.11/export_2:
configuration: added
status: mounted
10.0.42.13:
- export export_2 (eid=2) on /mnt/rozofs@10.0.42.11/export_2:
configuration: added
status: mounted
10.0.42.11:
- export export_2 (eid=2) on /mnt/rozofs@10.0.42.11/export_2:
configuration: added
status: mounted
10.0.42.14:
- export export_2 (eid=2) on /mnt/rozofs@10.0.42.11/export_2:
configuration: added
status: mounted
Our export has been successfully mounted on all our hosts.
If you want to go further with RozoFS file system, this documentation will be very helpful: https://media.readthedocs.org/pdf/rozofs/latest/rozofs.pdf
Conclusion
In this article, we have seen how to build a functional distributed file system running anywhere with the help of Vagrant. Now you can provide to all your developers this Vagrantfile and they will be able to run the embedded solution from their laptop or their development machine.