Wednesday 6 August 2014

Creating an Amazon Centos HVM AMI with Resizable LVM Root

With the introduction of t2 instance types on aws I needed a base AMI which was of a HVM type but used LVM for its partitions. I wanted the ability to modify the partiton sizes both on the fly by added a new ebs volume and when launching images without the hassle of repartitioning ebs volumes. The following is a step by step guide to creating your AMI. I assume you have a good understanding of using AWS.

Launch any existing CentOS 6 AMI. I run updates to make sure all packages version match not sure this is actually required.

Create a 1G EBS and a 9G EBS drive and attach these to your new Image Builder instance. The 1G will contain the boot partition and swap space. The 9G will contain the lvm volumes.

These should appear as
  • /dev/sdf 1G
  • /dev/sdg 9G
Partion /dev/sdf into a 256Mb Boot and the rest as Swap.

fdisk -l /dev/sdf should return
   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1          34      273073+  83  Linux
/dev/sdf2              35         130      771120   82  Linux swap / Solaris 
Format the partitions on /dev/sdf
mkfs.ext2 /dev/sdf1
mkswap -L swapfs /dev/sdf2
e2label /dev/sdf1 bootfs
Create the Volume Group and Logical Volumes. I use a modified CIS partitioning scheme removing the /home as the systems doesn't have users.
vgcreate vgEBS /dev/sdg
lvcreate -l 1279 -n lvRoot /dev/vgEBS
lvcreate -L 2G -n lvVar /dev/vgEBS
lvcreate -L 1G -n lvLog /dev/vgEBS
lvcreate -L 512 -n lvAudit /dev/vgEBS
lvcreate -L 512 -n lvTmp /dev/vgEBS
Running lvs should return
LV      VG    Attr       LSize   Pool Origin Data%  Move Log Cpy%Sync Convert
lvAudit vgEBS -wi-ao---- 512.00m                                         
lvLog   vgEBS -wi-ao----   1.00g                                         
lvRoot  vgEBS -wi-ao----   5.00g                                         
lvTmp   vgEBS -wi-ao---- 512.00m                                         
lvVar   vgEBS -wi-ao----   2.00g                                             
Format the lvms
mkfs.ext4 /dev/vgEBS/lvRoot 
mkfs.ext4 /dev/vgEBS/lvVar
mkfs.ext4 /dev/vgEBS/lvLog
mkfs.ext4 /dev/vgEBS/lvAudit
mkfs.ext4 /dev/vgEBS/lvTmp
Modify the ext4 disks to remove reserved space, Setting the root to use 1% reserved space.
tune2fs -m 0 /dev/vgEBS/lvVar 
tune2fs -m 0 /dev/vgEBS/lvLog 
tune2fs -m 0 /dev/vgEBS/lvAudit
tune2fs -m 0 /dev/vgEBS/lvTmp
tune2fs -m 1 /dev/vgEBS/lvRoot
Mount the folders and make sub folders
mount /dev/vgEBS/lvRoot /mnt/
mkdir -p /mnt/proc /mnt/etc /mnt/dev /mnt/var /mnt/sys /mnt/tmp /mnt/boot
mount /dev/sdf1 /mnt/boot
mount /dev/vgEBS/lvVar /mnt/var
mount /dev/vgEBS/lvTmp /mnt/tmp
mkdir -p /mnt/var/log /mnt/var/cache /mnt/var/lock/rpm
mount /dev/vgEBS/lvLog /mnt/var/log/
mkdir -p /mnt/var/log/audit
mount /dev/vgEBS/lvAudit /mnt/var/log/audit/
Create the fstab
cat <<EOT >/mnt/etc/fstab
/dev/vgEBS/lvRoot  /                   ext4            defaults            1 1
LABEL=bootfs       /boot               ext3            defaults            1 2
/dev/vgEBS/lvTmp   /tmp                ext4            nodev,nosuid        0 0
/dev/vgEBS/lvVar   /var                ext4            defaults            0 0
/dev/vgEBS/lvLog   /var/log            ext4            defaults            0 0
/dev/vgEBS/lvAudit /var/log/audit      ext4            defaults            0 0
LABEL=swapfs       swap                swap            defaults            0 0
/tmp               /var/tmp            none            bind                0 0
tmpfs              /dev/shm            tmpfs           noexec,nodev,nosuid 0 0
devpts             /dev/pts            devpts          gid=5,mode=620      0 0
sysfs              /sys                sysfs           defaults            0 0
proc               /proc               proc            defaults            0 0
EOT
 Install the Base system into the /mnt folder
yum --releasever=6 --installroot=/mnt/ -y groupinstall Base 
Configure the network settings
cat <<EOT >/mnt/etc/sysconfig/network
NETWORKING=yes
HOSTNAME=localhost.localdomain
NETWORKING_IPV6=no
IPV6INIT=no
EOT
cat <<EOT >/mnt/etc/sysconfig/network-scripts/ifcfg-eth0 
DEVICE="eth0"
NM_CONTROLLED="no"
ONBOOT="yes"
BOOTPROTO="dhcp"
EOT
Turn off ipv6 the correct way
echo "sysctl.conf:net.ipv6.conf.all.disable_ipv6 = 1" >> /mnt/etc/sysctl.conf
Setup Grub Configuration file. Get the version of the kernel installed into your chroot environment.
CHROOT_KERNEL=`yum --installroot=/mnt/ install kernel | awk '/kernel/ {print $2}' | cut -c 8-`
 Setup the grub conf file
cat <<EOT > /mnt/boot/grub/grub.conf
default=0
timeout=0
hiddenmenu

title CentOS ($CHROOT_KERNEL)
    root (hd0,0)
    kernel /vmlinuz-$CHROOT_KERNEL ro root=/dev/vgEBS/lvRoot rd_LVM_LV=vgEBS/lvRoot rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us console=ttyS0
    initrd /initramfs-$CHROOT_KERNEL.img
EOT
ln -s /boot/grub/grub.conf /mnt/boot/grub/menu.lst
Setup the chroot environment.
MAKEDEV -d /mnt/dev -x console
MAKEDEV -d /mnt/dev -x null
MAKEDEV -d /mnt/dev -x zero
 
mount -o bind /dev /mnt/dev
mount -o bind /dev/pts /mnt/dev/pts
mount -o bind /dev/shm /mnt/dev/shm
mount -o bind /proc /mnt/proc
mount -o bind /sys /mnt/sys

cp /etc/resolv.conf /mnt/etc/resolv.conf
Enter the chrooted environment
chroot /mnt
Remove extra things we don't need
yum remove cpuspeed abrt* at hal* iptables-ipv6 irqbalance kexec-tools psacct quota sendmail smartmontools rng-tools mdadm
Add things in we will need.
yum install openssh-server yum-plugin-fastestmirror e2fsprogs dhclient yum-presto vi audit postfix cronie  
Setup a ec2-user and set a secure password
adduser ec2-user
passwd ec2-user
Setup aws scripts to import ssh keys (Use this as a base for other on boot tasks)
cat <<EOT > /usr/local/bin/awsUpdateKeys
#!/bin/bash

if [ ! -d /home/ec2-user/.ssh ]; then
  mkdir -m 0700 -p /home/ec2-user/.ssh
  restorecon /home/ec2-user/.ssh
fi

# Get the home/ec2-user ssh key setup
ReTry=0
Status=1
while [ \$Status -gt 0 ] && [ \$ReTry -lt 5 ]; do
    if (( \$ReTry > 0 ))
    then
        sleep 2
    fi 
    curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /home/ec2-user/.ssh/authorized_keys 2> /dev/null
    Status=\$?
    ReTry=\$[Retry+1]
done

chown -R ec2-user:ec2-user /home/ec2-user/.ssh/
chmod 600 /home/ec2-user/.ssh/authorized_keys && restorecon /home/ec2-user/.ssh/authorized_keys

if (( \$ReTry >= 5 ))
then
    exit 1
fi
EOT
chmod +x /usr/local/bin/awsUpdateKeys
cat <<EOT > /etc/init.d/aws
#!/bin/bash
#
# chkconfig: 2345 11 01
# description: AWS Configuration on Boot

source /etc/rc.d/init.d/functions
case "\$1" in
    start)
        echo -n "Update SSH Keys: "
        /usr/local/bin/awsUpdateKeys && echo_success || echo_failure
        echo    

        # Add your own initialisation scripts here.

        RETVAL=0
    ;;
    stop)
        RETVAL=0
        ;;
    status)
        RETVAL=0
        ;;
    *)
        echo $"Usage: $0 {start|stop|status}"
        RETVAL=2
esac
exit $RETVAL
EOT
chmod +x /etc/init.d/aws

chkconfig --add aws
Install Grub
grub-install /dev/sdf
Probing devices to guess BIOS drives. This may take a long time.
Unknown partition table signature
/dev/xvdf does not have any corresponding BIOS drive.
Don't worry about the output this has populated the /boot/grub folder as required for us. Now manually install grub.
grub
 grub> device (hd0) /dev/sdf
 grub> root (hd0,0)
 grub> setup (hd0)
The setup set should output the following
setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd0)"...  27 sectors are embedded.
succeeded
 Running "install /grub/stage1 (hd0) (hd0)1+27 p (hd0,0)/grub/stage2 /grub/grub.conf"... succeeded
Add the ec2-user into the sudoers files.
echo "%ec2-user     ALL=(ALL)       NOPASSWD: ALL" >> /etc/sudoers  
Set the system to relabel the whole file-system on boot for selinux (Seriously, stop disabling SELinux)
touch /.autorelabel
Now on the AWS management console take a snapshot from both the 1G and 9G drives. Create a HVM image from these snapshot using the 1G as your root drive. And the 9G as the second drive. This process won't work with a para-virtual style image.

You can now launch the AMI as an instance and set the second drive to be as big as you like resizing the partitions using standard lvm tools.

24 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. Hello,

    Did the above procedure work for any one.

    Regards
    Ali

    ReplyDelete
  3. Does anybody have a script for packer building a LVM enabled RedHat/CentOS AMI ?

    ReplyDelete
  4. I have read your blog its very attractive and impressive. I like your blog AWS Online Course

    ReplyDelete
  5. I found your blog while searching for the updates, I am happy to be here. Very useful content and also easily understandable providing.. Believe me I did wrote an post about tutorials for beginners with reference of your blog. 
    best rpa training in bangalore
    rpa training in bangalore
    rpa course in bangalore
    RPA training in bangalore
    rpa training in chennai
    rpa online training

    ReplyDelete
  6. I found your blog while searching for the updates, I am happy to be here. Very useful content and also easily understandable providing.. Believe me I did wrote an post about tutorials for beginners with reference of your blog. 
    best rpa training in bangalore
    rpa training in bangalore
    rpa course in bangalore
    RPA training in bangalore
    rpa training in chennai
    rpa online training

    ReplyDelete
  7. A very nice guide. I will definitely follow these tips. Thank you for sharing such detailed article. I am learning a lot from you.
    AWS Training in Bangalore

    AWS Training in pune

    ReplyDelete
  8. Thanks For Sharing The InFormation The Information Shared Is Very Valuable Please Keeep updating Us Time Just Went On reading the article Python Online Course Data Science Online Course Data Science Online Course Hadoop Online Course Awsw Online Course



    ReplyDelete
  9. Nice Post
    For aws training in Bangalore, Visit:
    AWS training in Bangalore

    ReplyDelete
  10. Nice Post
    Visit for the best AI training in Bangalore:-
    Artificial Intelligence training in Bangalore

    ReplyDelete
  11. The content of the Blogs are attracted to Read all the Articles, people are getting Benefit from these kind of blogs.Big Thanks for this effective works.Keep going
    python training in chennai | python training in annanagar | python training in omr | python training in porur | python training in tambaram | python training in velachery

    ReplyDelete
  12. This is an awesome post.Really very informative and creative contents. These concept is a good way to enhance the knowledge.I like it and help me to development very well.Thank youLovely blog ..! I really enjoyed reading this article. Thank you for sharing such a wonderful postJava training in Chennai

    Java Online training in Chennai

    Java Course in Chennai

    Best JAVA Training Institutes in Chennai

    Java training in Bangalore

    Java training in Hyderabad

    Java Training in Coimbatore

    Java Training

    Java Online Training

    ReplyDelete
  13. Great post i must say and thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more.thanks lot!!

    android training in chennai

    android online training in chennai

    android training in bangalore

    android training in hyderabad

    android Training in coimbatore

    android training

    android online training

    ReplyDelete
  14. Nice post. Thanks for sharing! I want people to know just how good this information is in your article. It’s interesting content and Great work.Very useful and informative content has been shared out here, Thanks for sharing it

    Azure Training in Chennai

    Azure Training in Bangalore

    Azure Training in Hyderabad

    Azure Training in Pune

    Azure Training | microsoft azure certification | Azure Online Training Course

    Azure Online Training

    ReplyDelete
  15. Thanks For Sharing The Information The Information Shared Is Very Valuable.Nice blog Post ! This post contains very informative and knowledgeable. Thanks for sharing the most valuable information.
    DevOps Training in Chennai

    DevOps Online Training in Chennai

    DevOps Training in Bangalore

    DevOps Training in Hyderabad

    DevOps Training in Coimbatore

    DevOps Training

    DevOps Online Training

    ReplyDelete
  16. I found your blog while searching for the updates, I am happy to be here. Very useful content and also easily understandable providing.. Believe me I did wrote an post about tutorials for beginners with reference of your blog.



    AWS Course in Bangalore

    AWS Course in Hyderabad

    AWS Course in Coimbatore

    AWS Course

    AWS Certification Course

    AWS Certification Training

    AWS Online Training

    AWS Training

    ReplyDelete