Showing posts with label debootstrap. Show all posts

Tuesday 10 September 2013

Linux Containers on Virtualbox - Disposal Boxes by Michal Migurski's

Hey look, a month went by and I stopped blogging because I have a new job. Great.
One of my responsibilities is keeping an eye on our sprawling Github account, currently at 326 repositories and 151 members. The current fellows are working on a huge number of projects and I frequently need to be able to quickly install, test and run projects with a weirdly-large variety of backend and server technologies. So, it’s become incredibly important to me to be able to rapidly spin up disposable Linux web servers to test with. Seth clued me in to Linux Containers (LXC) for this:

LXC provides operating system-level virtualization not via a full blown virtual machine, but rather provides a virtual environment that has its own process and network space. LXC relies on the Linux kernel cgroups functionality that became available in version 2.6.24, developed as part of LXC. … It is used by Heroku to provide separation between their “dynos.”

I use a Mac, so I’m running these under Virtualbox. I move around between a number of different networks, so each server container had to have a no-hassle network connection. I’m also impatient, so I really needed to be able to clone these in seconds and have them ready to use.
This is a guide for creating an Ubuntu Linux virtual machine under Virtualbox to host individual containers with simple two-way network connectivity. You’ll be able to clone a container with a single command, and connect to it using a simple <container>.local host name.

The Linux Host

First, download an Ubuntu ISO. I try to stick to the long-term support releases, so I’m using Ubuntu 12.04 here. Get a copy of Virtualbox, also free.
Create a new Virtualbox virtual machine to boot from the Ubuntu installation ISO. For a root volume, I selected the VDI format with a size of 32GB. The disk image will expand as it’s allocated, so it won’t take up all that space right away. I manually created three partitions on the volume:

4.0 GB ext4 primary.
512 MB swap, matching RAM size. Could use more.
All remaining space btrfs, mounted at /var/lib/lxc.

Btrfs (B-tree file system, pronounced “Butter F S”, “Butterfuss”, “Better F S”, or “B-tree F S") is a GPL-licensed experimental copy-on-write file system. It will allow our cloned containers to occupy only as much disk space as is changed, which will decrease the overall file size of the virtual machine.
During the OS installation process, you’ll need to select a host name. I used “ubuntu-demo” for this demonstration.

Host Linux Networking

Boot into Linux. I started by installing some basics, for me: git, vim, tcsh, screen, htop, and etckeeper.
Set up /etc/network/interfaces with two bridges for eth0 and eth1, both DHCP. Note that eth0 and eth1 must be commented-out, as in this sample part of my /etc/network/interfaces:

## The primary network interface
#auto eth0
#iface eth0 inet dhcp

auto br0
iface br0 inet dhcp
        dns-nameservers 8.8.8.8
        bridge_ports eth0
        bridge_fd 0
        bridge_maxwait 0

auto br1
iface br1 inet dhcp
        bridge_ports eth1
        bridge_fd 0
        bridge_maxwait 0

Back in Virtualbox preferencese, create a new network adapter and call it “vboxnet0”. My settings are 10.1.0.1, 255.255.255.0, with DHCP turned on.

Shut down the Linux host, and add the secondary interface in Virtual box. Choose host-only networking, the vboxnet0 adapter, and “Allow All” promiscuous mode so that the containers can see inbound network traffic.

The primary interface will be NAT by default, which will carry normal out-bound internet traffic.

Adapter 1: NAT (default)
Adapter 2: Host-Only vboxnet0

Start up the Linux host again, and you should now be able to ping the outside world.

% ping 8.8.8.8

PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_req=1 ttl=63 time=340 ms
…

Use ifconfig to find your Linux IP address (mine is 10.1.0.2), and try ssh’ing to that address from your Mac command line with the username you chose during initial Ubuntu installation.

% ifconfig br1

br1       Link encap:Ethernet  HWaddr 08:00:27:94:df:ed  
          inet addr:10.1.0.2  Bcast:10.1.0.255  Mask:255.255.255.0
          inet6 addr: …

Next, we’ll set up Avahi to broadcast host names so we don’t need to remember DHCP-assigned IP addresses. On the Linux host, install avahi-daemon:

% apt-get install avahi-daemon

In the configuration file /etc/avahi/avahi-daemon.conf, change these lines to clarify that our host names need only work on the second, host-only network adapter:

allow-interfaces=br1,eth1
deny-interfaces=br0,eth0,lxcbr0

Then restart Avahi.

% sudo service avahi-daemon restart

Now, you should be able to ping and ssh to ubuntu-demo.local from within the virtual machine and your Mac command line.

No Guest Containers

So far, we have a Linux virtual machine with a reliable two-way network connection that’s resilient to external network failures, available via a meaningful host name, and with a slightly funny disk setup. You could stop here, skipping the LXC steps and use Virtualbox’s built-in cloning functionality or something like Vagrant to set up fresh development environments. I’m going to keep going and set up LXC.

Linux Guest Containers

Install LXC.

% sudo apt-get lxc

Initial LXC setup uses templates, and on Ubuntu there are several useful ones that come with the package. You can find them under /usr/lib/lxc/templates; I have templates for ubuntu, fedora, debian, opensuse, and other popular Linux distributions. To create a new container called “base” use lxc-create with a chosen template.

% sudo lxc-create -n base -t ubuntu

This takes a few minutes, because it needs retrieve a bunch of packages for a minimal Ubuntu system. You’ll see this message at some point:

##
# The default user is 'ubuntu' with password 'ubuntu'!
# Use the 'sudo' command to run tasks as root in the container.
##

Without starting the container, modify its network adapters to match the two we set up earlier. Edit the top of /var/lib/lxc/base/config to look something like this:

lxc.network.type=veth
lxc.network.link=br0
lxc.network.flags=up
lxc.network.hwaddr = 00:16:3e:c2:9d:71

lxc.network.type=veth
lxc.network.link=br1
lxc.network.flags=up
lxc.network.hwaddr = 00:16:3e:c2:9d:72

An initial MAC address will be randomly generated for you under lxc.network.hwaddr, just make sure that the second one is different.
Modify the container’s network interfaces by editing /var/lib/lxc/base/rootfs/etc/network/interfaces (/var/lib/lxc/base/rootfs is the root filesystem of the new container) to look like this:

auto eth0
iface eth0 inet dhcp
        dns-nameservers 8.8.8.8

auto eth1
iface eth1 inet dhcp

Now your container knows about two network adapters, and they have been bridged to the Linux host OS virtual machine NAT and host-only adapters. Start your new container:

% sudo lxc-start -n base

You’ll see a normal Linux login screen at first, use the default username and password “ubuntu” and “ubuntu” from above. The system starts out with minimal packages. Install a few so you can get around, and include language-pack-en so you don’t get a bunch of annoying character set warnings:

% sudo apt-get install language-pack-en
% sudo apt-get install git vim tcsh screen htop etckeeper
% sudo apt-get install avahi-daemon

Make a similar change to the /etc/avahi/avahi-daemon.conf as above:

allow-interfaces=eth1
deny-interfaces=eth0

Shut down to return to the Linux host OS.

% sudo shutdown -h now

Now, restart the container with all the above modifications, in daemon mode.

% sudo lxc-start -d -n base

After it’s started up, you should be able to ping and ssh to base.local from your Linux host OS and your Mac.

% ssh ubuntu@base.local

Cloning a Container

Finally, we will clone the base container. If you’re curious about the effects of Btrfs, check the overall disk usage of the /var/lib/lxc volume where the containers are stored:

% df -h /var/lib/lxc

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda3        28G  572M   26G   3% /var/lib/lxc

Clone the base container to a new one, called “clone”.

% sudo lxc-clone  -o base -n clone

Look at the disk usage again, and you will see that it’s not grown by much.

% df -h /var/lib/lxc

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda3        28G  573M   26G   3% /var/lib/lxc

If you actually look at the disk usage of the individual container directories, you’ll see that Btrfs is allowing 1.1GB of files to live in just 573MB of space, representing the repeating base files between the two containers.

% sudo du -sch /var/lib/lxc/*

560M /var/lib/lxc/base
560M /var/lib/lxc/clone
1.1G total

You can now start the new clone container, connect to it and begin making changes.

% sudo lxc-start -d -n clone
% ssh ubuntu@clone.local

Conclusion

I have been using this setup for the past few weeks, currently with a half-dozen containers that I use for a variety of jobs: testing TileStache, installing Rails applications with RVM, serving Postgres data, and checking out new packages. One drawback that I have encountered is that as the disk image grows, my nightly time machine backups grow considerably. The Mac host OS can only see the Linux disk image as a single file.
On the other hand, having ready access to a variety of local Linux environments has been a boon to my ability to quickly try out ideas. Special thanks again to Seth for helping me work through some of the networking ugliness.

Saturday 23 March 2013

How to Setup Chroot Directory Structure ...

DebootstrapChroot

This article demonstrates a quick and easy way to create a chroot environment on an Ubuntu computer, which is like having a virtual system without the overhead of actual virtualization.

A chroot can be used for things like:

Running a 32-bit Firefox browser or a 32-bit Wine bottle on a 64-bit system.
Trying an older or newer Ubuntu release without reinstalling the operating system.
Trying a Debian release or other distribution derived from Debian.
Cross compiling and building packages for a different platform like Launchpad or Soyuz does it.

Example Configuration

In this example, we use a current Ubuntu 9.04 Jaunty system (the "host") to create a chroot for the older Ubuntu 8.04 Hardy release (the "target"). We are arbitrarily naming the new chroot environment hardy_i386 and putting it in the /srv/chroot directory on the host system.

Step 1: Install packages on the host computer.

First, install debootstrap, which is a utility that downloads and unpacks a basic Ubuntu system:

 $ sudo apt-get install debootstrap

Second, install schroot, which is a utility that wraps the regular chroot program and automatically manages chroot environments:

 $ sudo apt-get install schroot

Note: The debootstrap utility is usually backwards compatible with older releases, but it may be incompatible with newer releases. For example, the debootstrap that is bundled with Jaunty can prepare a Hardy chroot like we are doing here, but the debootstrap that is bundled with Hardy cannot prepare a Jaunty chroot.

If you have any difficultly with a debootstrap version mismatch, then visit http://packages.ubuntu.com/ to manually download and install the debootstrap package on the host system from the repository for the target release.

Step 2: Create a configuration file for schroot.

Choose a short name for the chroot, we use hardy_i386 in this example, and create a configuration file for it like this:

  sudo editor /etc/schroot/chroot.d/hardy_i386.conf

Note: In lucid the filename must not contain '.' , it should be lucid_i386_conf.

Put this in the new file:

  [hardy_i386]
  description=Ubuntu 8.04 Hardy for i386
  location=/srv/chroot/hardy_i386
  #personality=linux32
  root-users=bob
  run-setup-scripts=true
  run-exec-scripts=true
  type=directory
  users=alice,bob,charlie

Note: if you copy this example to your clipboard, be careful to start each line in column 1 before you save the new file! If you forget, the command schroot -l will fail with an error, e.g.

E:/etc/schroot/chroot.d/hardy_i386.conf: line 0: Invalid line: “ [hardy_i386]”.

Note: for lucid use directory instead of location, e.g. directory=/srv/chroot/hardy_i386 .

Change these things in the example configuration file to fit your system:

location: This should be a directory that is outside of the /home tree. The latest schroot documentation recommends /srv/chroot.
personality: Enable this line if the host system is 64-bit running on an amd64/x64 computer and the chroot is 32-bit for i386. Otherwise, leave it disabled.
users: These are users on the host system that can invoke the schroot program and get access to the chroot system. Your username on the host system should be here.
root-users: These are users on the host system that can invoke the schroot program and get direct access to the chroot system as the root user.

Note: Do not put whitespace around the '=' character, and do not quote strings after the '=' character.

Step 3: Run debootstrap.

This will download and unpack a basic Ubuntu system to the chroot directory, similar to what the host system already has at the real root directory ("/").

 $ sudo mkdir -p /srv/chroot/hardy_i386
 $ sudo debootstrap --variant=buildd --arch=i386 hardy /srv/chroot/hardy_i386 http://archive.ubuntu.com/ubuntu/

This command should work for any distribution that is derived from Debian. Substitute the architecture "i386", the release name "hardy", and the repository address "http://archive.ubuntu.com/ubuntu/" appropriately. For example, do this to get the 64-bit build of Hardy instead of the 32-bit build:

  $ sudo debootstrap --arch=amd64 hardy /srv/chroot/hardy_amd64/ http://archive.ubuntu.com/ubuntu/

Note: Remember to change all instances of hardy_i386 to hardy_amd64 in the configuration file and on the command line if you actually do this.

Do something like this to get an upstream Debian release:

  $ sudo debootstrap --arch=amd64 sid /srv/chroot/sid_amd64/ http://ftp.debian.org/debian/

If trouble arises, debootsrap accepts a --verbose flag that may provide further insight.

Step 4: Check the chroot

This command lists configured chroots:tro

  $ schroot -l

If hardy_i386 appears in the list, then run:

  $ schroot -c hardy_i386 -u root

Note: This should work without using sudo to invoke the schroot program, and it should result in a root prompt in the chroot environment. Check that the root prompt is in a different system:

  # lsb_release -a

For the Hardy system that we just built, the lsb_release command should print:

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 8.04
Release:        8.04
Codename:       hardy

We're done!

WARNING

For convenience, the default schroot configuration rebinds the /home directory on the host system so that it appears in the chroot system. This could be unexpected if you are familiar with the older dchroot program or the regular chroot program because it means that you can accidentally delete or otherwise damage things in /home on the host system.

To change this behavior run:

  $ sudo editor /etc/schroot/mount-defaults

And disable the /home line so that the file reads:

  # mount.defaults: static file system information for chroots.
  # Note that the mount point will be prefixed by the chroot path
  # (CHROOT_PATH)
  #
  # <file system> <mount point>   <type>  <options>       <dump>  <pass>
  proc            /proc           proc    defaults        0       0
  /dev/pts        /dev/pts        none    rw,bind         0       0
  tmpfs           /dev/shm        tmpfs   defaults        0       0
  #/home           /home           none    rw,bind         0       0
  /tmp            /tmp            none    rw,bind         0       0

The mount.defaults file is the /etc/fstab for chroot environments.

Hints

Install the ubuntu-minimal package in a new chroot after you create it:

  $ schroot -c hardy_i386 -u root
  # apt-get install ubuntu-minimal

If you get locale warnings in the chroot like "Locale not supported by C library." or "perl: warning: Setting locale failed." , then try one or more of these commands:

  $ sudo dpkg-reconfigure locales

  $ sudo apt-get install language-pack-en

  $ locale-gen en_US.UTF-8

If your preferred language is not English, then change "-en" and "en_US" appropriately.

As of Lucid, schroot has changed in these ways:

The file should be named: /etc/schroot/chroot.d/hardy-i386
The keywords in the file have changed and some have been deprecated. Additionally, keywords have to start at the beginning of the line. The file should read:

  [hardy-i386]
  description=Ubuntu 8.04 Hardy for i386
  directory=/srv/chroot/hardy-i386
  #personality=linux32
  root-users=bob
  type=directory
  users=alice,bob,charlie

As of Maverick schroot has further changed in these ways:

The configuration file should be stored in /etc/schroot/

TLDR

There's a much simplier way to get a basic chroot environment from an ISO image; if the text above seems TLDR, try this.

First of all, install Ubuntu Customization Kit:

  $ sudo apt-get install uck

Then set the directory in which you want to create the chroot environment:

  $ export BASEDIR=/path/to/chroot/directory/

Unpack the ISO image (this may take quite some time):

  $ sudo uck-remaster-unpack-iso /path/to/your/image.iso "$BASEDIR" && sudo uck-remaster-unpack-rootfs "$BASEDIR" && sudo uck-remaster-unpack-initrd "$BASEDIR"

You're done! Now, to enter the chroot environment, just execute

  $ sudo uck-remaster-chroot-rootfs /path/to/chroot/directory/

every time you wish to enter the chroot console. To leave it, type "exit".

To be able to run X applications, e.g. gedit, run

  # HOME=/root

in the chroot environment.

{ Beat The Code & Hack Around }