Posts by: "Chad Stewart"

FreeBSD – root ZFS – automated install script

By On · Add Comment

I finally found a working method of installing FreeBSD 9.1-RELEASE using all ZFS, no UFS.  It took me a bit of tinkering to come up with this script.  I tried playing with the zpool cache file, etc., but only what I have here in this script ended up working for me.

The script shown below is specific to this particular server, it will need to be adjusted to suit the hardware. I put the script on a web server, then I boot the to be installed system from CD, select shell or Live CD.  Then I download the script and execute.  Tip: you can do that in one step

 

ftp -o - http://server/script.sh | sh

That will download the script and immediately execute it.

 

In this example the server has 6 disks.  I’ve configured the hardware raid controller such that it is basically a JBOD.  I want ZFS handling this, not hardware. While I would prefer ZFS have the entire disk, I don’t think that is possible today.  I opted to make each disk identical from a layout perspective. My thinking being should a disk fail, any other disk should be bootable, etc..  I also opted to only create a raidz pool with 5 disks, the 6th disk is a spare.  These are not the newest servers or drives, I know they will fail at some point and thus I’m willing to trade space for data protection.

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104

#!/bin/sh
 
MP=/mnt
 
zpool export zroot
for DSK in 0 1 2 3 4 5; do
gpart destroy -F da$DSK
gpart create -s gpt da$DSK
gpart add -s 122 -t freebsd-boot da$DSK
gpart add -s 512M -t freebsd-swap -l swap$DSK da$DSK
gpart add -t freebsd-zfs -l disk$DSK da$DSK
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da$DSK
done
 
# Now create the pool
zpool create -f -o altroot=$MP -O canmount=off zroot raidz /dev/gpt/disk0 /dev/gpt/disk1 /dev/gpt/disk2 /dev/gpt/disk3 /dev/gpt/disk4 spare /dev/gpt/disk5
 
 
 
 
# I personally dislike the root of pools being mounted and available.
zfs set mountpoint=none                                         zroot
 
 
 
# Due to the above, we have to specify the mount points for each FS we create,
# this same rule applies post install when we create a FS, must set the mount point
 
zfs create -o mountpoint=/                                                                      zroot/ROOT
zfs create -o mountpoint=/usr                                                                   zroot/usr
zfs create                              -o mountpoint=/var                                      zroot/var
zfs create -o setuid=off                -o mountpoint=/home                                     zroot/home
zfs create              -o setuid=off                       -o mountpoint=/usr/local            zroot/usr/local
zfs create -o exec=off  -o setuid=off                       -o mountpoint=/usr/locall/etc/      zroot/usr/local/etc
zfs create -o exec=off  -o setuid=off                       -o mountpoint=/usr/src              zroot/usr/src
#zfs create -o setuid=off                                    -o mountpoint=/usr/ports            zroot/usr/ports
#zfs create -o exec=off  -o setuid=off                       -o mountpoint=/usr/ports/distfiles  zroot/usr/ports/distfiles
#zfs create -o exec=off  -o setuid=off                       -o mountpoint=/usr/ports/packages   zroot/usr/ports/packages
zfs create -o exec=off  -o setuid=off   -o compression=on   -o mountpoint=/var/crash            zroot/var/crash
zfs create -o exec=off  -o setuid=off                       -o mountpoint=/var/db               zroot/var/db
zfs create -o exec=on   -o setuid=off  -o compression=on    -o mountpoint=/var/db/pkg           zroot/var/db/pkg
zfs create -o exec=off  -o setuid=off                       -o mountpoint=/var/empty            zroot/var/empty
zfs create -o exec=off  -o setuid=off  -o compression=on    -o mountpoint=/var/log              zroot/var/log
zfs create -o exec=off  -o setuid=off  -o compression=on    -o mountpoint=/var/mail             zroot/var/mail
zfs create -o exec=off  -o setuid=off                       -o mountpoint=/var/run              zroot/var/run
zfs create -o exec=on   -o setuid=off                       -o mountpoint=/var/tmp              zroot/var/tmp
 
 
 
cd /usr/freebsd-dist
export DESTDIR=$MP/
echo "base..."; cat base.txz | tar --unlink -xpJf - -C ${DESTDIR:-/}
echo "src..."; cat src.txz | tar --unlink -xpJf - -C ${DESTDIR:-/}
echo "kernel..."; cat kernel.txz | tar --unlink -xpJf - -C ${DESTDIR:-/}
echo "doc..."; cat doc.txz | tar --unlink -xpJf - -C ${DESTDIR:-/}
 
#cat ports.txz | tar --unlink -xpJf - -C ${DESTDIR:-/}
 
chmod 1777 $MP/tmp
chmod 1777 $MP/var/tmp
 
zfs set readonly=on zroot/var/empty
zpool set bootfs=zroot/ROOT zroot
 
cat > $MP/boot/loader.conf << __EOF__
zfs_load="YES"
__EOF__
 
cat > $MP/etc/fstab << __EOF__
 # Device                       Mountpoint              FStype  Options         Dump    Pass#
 /dev/gpt/swap0                 none                    swap    sw              0       0
 /dev/gpt/swap1                 none                    swap    sw              0       0
 /dev/gpt/swap2                 none                    swap    sw              0       0
 /dev/gpt/swap3                 none                    swap    sw              0       0
 /dev/gpt/swap4                 none                    swap    sw              0       0
 /dev/gpt/swap5                 none                    swap    sw              0       0
 tmpfs                          /tmp                    tmpfs   rw,mode=777     0       0
__EOF__
 
 
cat > $MP/etc/rc.conf << __EOF__
# General System Config
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
zfs_enable="YES"
clear_tmp_enable="YES"  # Clear /tmp at startup
ifconfig_em0="up"
cloned_interfaces="vlan89"
ifconfig_vlan4="inet 192.168.89.135 netmask 255.255.255.0 vlan 89 vlandev em0"
ifconfig_vlan4_alias0="inet 192.168.89.245 netmask 255.255.255.255"
defaultrouter="192.168.89.1"
hostname="hostname.balius.com"
sshd_enable="YES"
__EOF__
 
 
zfs unmount -af
 
## The next two are "hacks" in my book, without the last line, on reboot
## it gets stuck trying to find zfs:zroot/ROOT, but somehow the -f "fixes" this quirk
#
zpool export zroot
zpool import -f zroot
echo "Now type reboot, remove CD-ROM, etc."

ZFS and netcat – moving files between different operating systems

By On · Add Comment

I decided the other day that using SmartOS for my media server, is not for me.  I need something I am more comfortable with supporting.  I have come up with what I feel is a good baseline image for my servers, using FreeBSD and ZFS, all ZFS, no UFS.  I’ve rebuilt a few of my servers using my new template and things appear to be nice and stable.

I decided that trading a CDROM only based OS, i.e. SmartOS a for system that I’m more comfortable using and is more general purpose is better for my media server.  I give up a bit of disk space, but gain something I am a lot more familiar with and capable of supporting and keeping up-to-date.

I had 129GB of data sitting on /movies/ that I didn’t want to lose.  I had a FreeBSD 9.1 i386 system test system that had about 127GB of space left in the pool.  I started out using rsync+ssh and transfer the files, slow.  Both servers are connected to a 1Gbit network but still the encryption overhead slowed things down I think.  Then I used tar and netcat, better but still not easy enough.  I pruned my media a bit, down to about 106GB.

Then I figured I’d try using ZFS send / recv over netcat.  I’ve done it once before as a bare metal restore and it worked great then.  The unknown was could I do this using SmartOS (from around 10/2012) as the source and FreeBSD 9.1-RELEASE i386 as the destination.  Turns out the answer is yes!

On the destination system I first created a ZFS file system

zfs create -o mountpoint=/movies zroot/movies

Next I got netcat and ZFS ready to receive

nc -w 300 -l 5600 | zfs recv -Fv zroot/movies

 

On the source system I created a snapshot first

zfs snapshot zones/movies@before-move

Next was sending the snapshot over

zfs send  -v zones/movies@before-move | nc -w 30 destination-server-IP 5600

 

It started moving the bits and no complaints from either side.  I started this later at night so I went to bed. When I woke up in the morning I found the process had completed and everything looked good.  I then went about installing FreeBSD 9.1-RELEASE amd64 replacing SmartOS.  That took less than 30 minutes. Then I used the same procedure again to copy the files from the temporary box back to the same hardware with a new OS.

I do enjoy ZFS, yes it is memory hungry, but being able to basically dump a live file system and then send that to another system over the network or other means is awesome! In this case I did it between two different operating systems and architectures. I admit I have not yet tried to stream the content from the rebuilt server.  It could be I’ve completely messed up my data and will have to recreate it from DVDs, in which case I’ll be pissed.  On the other hand had this been critical data I would have tested that it was usable before I rebuilt the existing system. I’m happy it took about an hour to move 106GB of data between the 2 servers, both ways.

 

 

Custom Screensaver images on Kindle Paperwhite

By On · Add Comment

I struggled today with getting custom screensaver images on my new Kindle Paperwhite (KPW).  You should be adept at command line things before attempting to do this.

The high level steps are as follows:

  1. Download and jailbreak using the files at this link http://www.mobileread.com/forums/showthread.php?p=2169819
  2. Download and install the USB network from the same place
  3. From http://wiki.mobileread.com/wiki/Kindle_Touch_Hacking#Screen_Savers you need to ssh into your KPW and  then do the following
mntroot rw
mv /usr/share/blanket/screensaver /mnt/us/screensaver.original
ln -sfn /mnt/us/screensaver /usr/share/blanket/screensaver

Despite what the directions say in the screensaver hack, the images for KPW are 1024 x 758 and the naming conventions are a little different. The names need to follow the pattern of

bg_medium_ss00.png

Then you can simply attach your KPW to your computer and copy the images you want to use to the screensaver folder.  Hopefully you found this useful.

 

If you do not have the images sized correctly, it appears to double the amount of time for the screen saver to load.

I took the images that I had been using on my Kindle 3 Keyboard and I noticed the time for the screensaver to be displayed was a lot longer.  I played with it this morning and it looks like if the size is incorrect, then it takes longer for it to display, nearly 7 seconds.  Pictures sized appropriately load in nearly 3 seconds.  Now if only I could get Aperture to export every picture to this exact size without my having to crop it first.

I have also noticed that after a restart, the first book I try to read reports an error.  When I try a 2nd time it works, along with other books. Annoying and not sure of the cause but thought you might like to know.

 

Gears

Automation tip — adjust a file on a lot of servers

By On · Add Comment

I have a customer that has 40 servers that perform a given function. They are comprised of physical machines and Solaris zones. I needed to adjust a file on each of those machines. I was not about to ssh into each machine and then start up vi and adjust the file by hand.

Here is what I did instead

for host in 1 2 3 4 5; do
  for zone in 1 2 3 4 5 6 7 8; do
    ssh -q $host\-$zone 'perl -p -i -e "s/ReplaceMe/WithMe/g" /path/to/file'
  done
done

I’m confident that I’m not the first person do this but I thought it was creative all the same. Combines a PERL one liner with two nested for loops for nice system automation.

See my post on consistency, this is a great example of why it is necessary.

ESXi – creating new virtual machines (servers) from the command line

By On · Add Comment

I was able to get a server up and running at home again, and given what I want to do, using ESXi is a good solution. When it comes to servers I prefer to do: (a) from the command line and (b) using ssh. First thing I did after getting ESXi installed was to enable their “Tech Support Mode“, and then things got interesting. The command line of ESXi 4.1 is limited, but yet powerful enough to do the job nicely.  After some searching I learned how to create a new server on the command line, power it on/off, register it with ESX and destroy it too.

To create and power on a new server I created the following script

#!/bin/sh

## Most of this taken from http://www.vm-help.com/esx40i/manage_without_VI_client_1.php

mkdir $1

# First make the disk
vmkfstools -c 15G -a lsilogic $1/$1.vmdk

# Now output the vmx file
cat < $1/$1.vmx

config.version = "8"
virtualHW.version = "7"
vmci0.present = "TRUE"
displayName = "$1"
floppy0.present = "FALSE"
numvcpus = "2"
scsi0.present = "TRUE"
scsi0.sharedBus = "none"
scsi0.virtualDev = "lsilogic"
memsize = "256"
scsi0:0.present = "TRUE"
scsi0:0.fileName = "$1.vmdk"
scsi0:0.deviceType = "scsi-hardDisk"
ide1:0.present = "TRUE"
ide1:0.fileName = "/vmfs/volumes/datastore1/ISOs/install48-amd64.iso"
ide1:0.deviceType = "cdrom-image"
ethernet0.present = "TRUE"
ethernet0.virtualDev = "vmxnet"
ethernet0.features = "15"
ethernet0.networkName = "VM Network"
ethernet0.addressType = "generated"
ethernet1.present = "TRUE"
ethernet1.virtualDev = "vmxnet"
ethernet1.features = "15"
ethernet1.networkName = "VM Network 2"
ethernet1.addressType = "generated"
guestOS = "freebsd-64"
EOF

# Now register our new VM
vnum=`vim-cmd solo/registervm /vmfs/volumes/datastore1/$1/$1.vmx`
vim-cmd vmsvc/power.on $vnum


#!/bin/sh
vim-cmd vmsvc/power.off `vim-cmd vmsvc/getallvms |grep $1|awk '{print $1}'`
vim-cmd vmsvc/destroy `vim-cmd vmsvc/getallvms |grep $1|awk '{print $1}'`

Then I went ahead and wrote a one liner to create 10 new machines and then destroy them.  Once I get my completely automated OpenBSD installer finished, then I can adjust the creation of the machines to boot from the network.  Thus all I will have to do is run the script to create the machine and then sit back and wait.  In I’m guessing about 15 minutes I’ll have an up and working OpenBSD system.  Since the install will be automated I can also fully customize the final result.  If I was responsible for a group of say web servers and my company just announced some awesome widget that everyone wants, then I had better be prepared to deploy more servers quickly.  Using the above I could easily accomplish that task.  After all if our customers could not use our website then they will not be happy.