Local Conspiracy (less scary than the global kind!)

Create an OpenStack machine image inside an OpenStack environment

Tue March 19 2013 by Christopher Aedo

(Alternate title: make booting from an ISO image in OpenStack useful)

If you have access to a server with plenty of disk space and CPU that supports the same hypervisor your cloud uses, it’s relatively easy to create an OpenStack machine image.  You can also pretty easily do this on a desktop/laptop machine using VirtualBox (though it can be a little more complicated).  In either case, you have to make sure you’ve got the right drivers for your target environment, and sometimes you’ll find you made a slight mistake early on only after you’ve gone all the way down the path of converting and importing an image of anywhere from a few hundred megs up to several gigs.

An alternative approach is to create the machine image within the environment itself.  OpenStack allows you to import and boot from an ISO, though the work is not yet complete.  The ephemeral disk that is attached to the instance can’t be snapshotted, so even after the OS install there’s no way to save or use the disk you installed to.  This will probably be addressed soon (likely fixed in Grizzly, as long as this bug gets resolved).  When fixed, it should allow you to boot from ISO with a local writable disk attached.  Then you could install the OS on that disk, snapshot it, and you’re done!

While you’re waiting for that to be resolved, here’s something you can do right now if you have access to an OpenStack environment with block storage available.  This relatively easy approach allows you to boot from ISO, install an OS on the attached volume, and then create an image based on that volume.  You’ll also need to launch an “image prep” VM in that environment so you have a place to install the Nova, Glance and the Qemu utils.  After you install the OS on the block storage volume, you’ll attach that volume to this image prep VM where you’ll convert it to qcow2 format and import into the environment.  The whole process from start to finish is less than an hour, and most of that time is either waiting for the OS install or the zero-fill.

  1. Import the ISO into your OpenStack environment
  2. Launch a VM using that ISO
  3. Provision and attach a volume to that VM
    4. - Be sure to make the volume large enough to support your OS install
  4. Launch the VNC console for the VM
  5. Install OS on iSCSI-attached volume
  6. Detach that volume from the ISO-booted VM
  7. **[OPTIONAL]** Launch a VM from that volume if you want to do any prep here
    8. - To boot from volume you’ll need to find the volume ID (nova volume-list), and the ID of a valid image - Nova won’t boot that image, but there’s a bug that requires you to specify an image even when booting from volume.  The command looks like: nova boot --image \ --flavor 2 \\ --block\_device\_mapping vda=\:::0 vm-from-volume - Add cloud-init or something similar if you want it to grab hostname, SSH keys, etc. from the metadata server - Remove any files you don’t want to see in every image (any temp files you might have created during prep, etc.) - Zero-fill the free space on the disk so the qcow image will be as small as possible (something like “dd if=/dev/zero of=/junk; rm -f /junk”) - Shutdown the VM and detach the volume when done
  8. Launch a VM for image preparation (I prefer an [Ubuntu UEC VM image](http://cloud-images.ubuntu.com/) for this) and install the necessary tools: apt-get install qemu-utils glance-client python-novaclient
  9. Attach the volume that has your new OS to the image prep VM (note which device it was attached to, in this example it was /dev/vdc)
  10. Convert the image: qemu-img convert -c -f raw -O qcow2 /dev/vdc newmachine-qcow.img
  11. Import your image: glance add name=”New Image” container\_format=bare \\ disk\_format=qcow2 is\_public=1 \< newmachine-qcow.img

All done!  You should see that image in glance now, and you’ll be able to launch as many as you need.  In my experience this was quicker than building on a remote machine and importing, and a major advantage was that my prep and testing was all in the same environment so I could be sure all the right drivers were there before importing into glance.

Coming soon, this Cinder patch by Avishay Traeger will automate the process of putting the machine image on an iSCSI volume directly into glance (obviating steps 8-11).  It will also make it easy to copy an image directly from glance onto a volume (combine that with boot-from-volume and you’ve got the analog of an EBS root volume on AWS!)

Let me know if this was helpful, or if you find any ways to improve the process!