misctechmusings notebook for projects, problems and solutions

Image cloning in VMware ESX shell

VMware image clone via ESX shell and vmkfstools

The free version of the VMware vSphere Hypervisor has a reduced feature set. One of the missing features is the ability to clone or create templates from existing vms. You can clone an image using vmkfstools within the ESX shell.

vmkfstools

OPTIONS FOR FILE SYSTEMS:

vmkfstools -C --createfs [vmfs3|vmfs5]
               -b --blocksize #[mMkK]
               -S --setfsname fsName
           -Z --spanfs span-partition
           -G --growfs grown-partition
   deviceName

           -P --queryfs -h --humanreadable
           -T --upgradevmfs
   vmfsPath
           -y --reclaimBlocks vmfsPath [--reclaimBlocksUnit #blocks]

OPTIONS FOR VIRTUAL DISKS:

vmkfstools -c --createvirtualdisk #[gGmMkK]
               -d --diskformat [zeroedthick
                               |thin
                               |eagerzeroedthick
                               ]
               -a --adaptertype [buslogic|lsilogic|ide
                                |lsisas|pvscsi]
               -W --objecttype [file|vsan]
               --policyFile <fileName>
           -w --writezeros
           -j --inflatedisk
           -k --eagerzero
           -K --punchzero
           -U --deletevirtualdisk
           -E --renamevirtualdisk srcDisk
           -i --clonevirtualdisk srcDisk
               -d --diskformat [zeroedthick
                               |thin
                               |eagerzeroedthick
                               |rdm:<device>|rdmp:<device>
                               |2gbsparse]
               -W --object [file|vsan]
               --policyFile <fileName>
               -N --avoidnativeclone
           -X --extendvirtualdisk #[gGmMkK]
               [-d --diskformat eagerzeroedthick]
           -M --migratevirtualdisk
           -r --createrdm /vmfs/devices/disks/...
           -q --queryrdm
           -z --createrdmpassthru /vmfs/devices/disks/...
           -v --verbose #
           -g --geometry
           -x --fix [check|repair]
           -e --chainConsistent
           -Q --objecttype name/value pair
           --uniqueblocks childDisk
   vmfsPath

OPTIONS FOR DEVICES:

           -L --lock [reserve|release|lunreset|targetreset|busreset|readkeys|readresv
                     ] /vmfs/devices/disks/...
           -B --breaklock /vmfs/devices/disks/...

vmkfstools -H --help

Identify source image and destination location

Log into the ESX sever and identify the location of the source image. In this example I will be cloning image centos-0 on datastore-1

~ # cd vmfs/volumes/datastore-1/
/vmfs/volumes/datastore-1 # ls
centos-0

I am going to create 3 additional CentOS vm images. Create the destination directory/s prior to running vmkfstools.

/vmfs/volumes/datastore-1 # mkdir centos-1 centos-2 centos-3
/vmfs/volumes/datastore-1 # ls
centos-0  centos-1  centos-2  centos-3

Clone the image via vmkfstools -i source destination. If not specified the new image will be thick provisioned. The first will use the vmkfstools command without additional options. The 2nd and 3rd will thin provision.

Thin - These virtual disks do not reserve space on the VMFS filesystem, nor do they reserve space on the back-end storage. They only consume blocks when data is written to disk from within the VM/Guest OS. The amount of actual space consumed by the VMDK starts out small, but grows in size as the Guest OS commits more I/O to disk, up to a maximum size set at VMDK creation time. The Guest OS believes that it has the maximum disk size available to it as storage space from the start.

Thick (aka LazyZeroedThick) – These disks reserve space on the VMFS filesystem but there is an interesting caveat. Although they are called thick disks, they behave similar to thinly provisioned disks. Disk blocks are only used on the back-end (array) when they get written to inside in the VM/Guest OS. Again, the Guest OS inside this VM thinks it has this maximum size from the start.

EagerZeroedThick – These virtual disks reserve space on the VMFS filesystem and zero out the disk blocks at creation time. This disk type may take a little longer to create as it zeroes out the blocks, but its performance should be optimal from deployment time (no overhead in zeroing out disk blocks on-demand, meaning no latency incurred from the zeroing operation). However, if the array supports the VAAI Zero primitive which offloads the zero operation to the array, then the additional time to create the zeroed out VMDK should be minimal.

/vmfs/volumes/datastore-1 # vmkfstools -i centos-0/centos-0.vmdk centos-1/centos-1.vmdk
Destination disk format: VMFS zeroedthick
Cloning disk 'centos-0/centos-0.vmdk'...
Clone: 100% done.

/vmfs/volumes/datastore-1 # ls -la centos-1/
total 16778248
drwxr-xr-x    1 root     root           560 Feb  3 18:27 .
drwxr-xr-t    1 root     root          1820 Feb  3 18:17 ..
-rw-------    1 root     root     17179869184 Feb  3 18:25 centos-1-flat.vmdk
-rw-------    1 root     root           524 Feb  3 18:27 centos-1.vmdk

Clone a 2nd copy to the centos-2 directory. Specify -d thin to thin provision this image.

/vmfs/volumes/datastore-1 # vmkfstools -i centos-0/centos-0.vmdk centos-2/centos-2.vmdk -d thin
Destination disk format: VMFS thin-provisioned
Cloning disk 'centos-0/centos-0.vmdk'...
Clone: 100% done.

/vmfs/volumes/datastore-1 # ls -la centos-2/
total 1471496
drwxr-xr-x    1 root     root           560 Feb  3 18:36 .
drwxr-xr-t    1 root     root          1820 Feb  3 18:17   ..
-rw-------    1 root     root     17179869184 Feb  3 18:34 centos-2-flat.vmdk
-rw-------    1 root     root           550 Feb  3 18:36 centos-2.vmdk

Clone a 3rd copy to the centos-3 directory

/vmfs/volumes/datastore-1 # vmkfstools -i centos-0/centos-0.vmdk centos-3/centos-3.vmdk -d thin
Destination disk format: VMFS thin-provisioned
Cloning disk 'centos-0/centos-0.vmdk'...
Clone: 100% done.

/vmfs/volumes/datastore-1 # ls -la centos-3/
total 1471496
drwxr-xr-x    1 root     root           560 Feb  3 18:42 .
drwxr-xr-t    1 root     root          1820 Feb  3 18:17 ..
-rw-------    1 root     root     17179869184 Feb  3 18:40 centos-3-flat.vmdk
-rw-------    1 root     root           550 Feb  3 18:42 centos-3.vmdk

3 new copies of the CentOS vmdk files have been created. The vmkfstools command does not copy any of the additional files from within the source directory. I will add some details on this at a later time. For this example the cloned images will be added via the vSphere Client.

Add the newly cloned images to the vSphere inventory.

Log into the vSphere Client and add a New Virtual Machine (Ctrl-N).

Step through the New Virtual Machine Wizard taking note of Configuration and Select a Disk pages. Select Custom from the Configuration step of the Configuration wizard. vsphere-config-custom

Select Use an existing virtual disk. from the Select a Disk step of the Configuration wizrd. vsphere-config-existing

Persistent static route via Windows registry

For whatever reason I was unable to set a static route via the route command. I assume the system was pending a reboot. However I did not have the time to schedule the outage.

So I added the route directly to the registry. Problem solved, next.

Persistent static routed are stored in the following registry key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip \Parameters\PersistentRoutes

Add a new String Value with the name destination,mask,gateway,metric

Example:

registry-route

Identify CPU architecture on Linux

Using lscpu is a quick and easy way to determine your CPU architecture.

64bit AMD processor example:

$ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                6
On-line CPU(s) list:   0-5
Thread(s) per core:    1
Core(s) per socket:    6
Socket(s):             1
NUMA node(s):          1
Vendor ID:             AuthenticAMD
CPU family:            16
Model:                 10
Model name:            AMD Phenom(tm) II X6 1055T Processor
Stepping:              0
CPU MHz:               800.000
CPU max MHz:           2800.0000
CPU min MHz:           800.0000
BogoMIPS:              5601.83
Virtualization:        AMD-V
L1d cache:             64K
L1i cache:             64K
L2 cache:              512K
L3 cache:              6144K
NUMA node0 CPU(s):     0-5

32bit Intel Pentium 4 example:

$ lscpu
Architecture:          i686
CPU op-mode(s):        32-bit
Byte Order:            Little Endian
CPU(s):                2
On-line CPU(s) list:   0,1
Thread(s) per core:    2
Core(s) per socket:    1
Socket(s):             1
Vendor ID:             GenuineIntel
CPU family:            15
Model:                 3
Model name:            Intel(R) Pentium(R) 4 CPU 3.40GHz
Stepping:              4
CPU MHz:               3391.668
BogoMIPS:              6786.97
L1d cache:             16K
L2 cache:              1024K

Archlinux Kernel Repair

So I managed to uninstall my kernel.

In an effort to get VMware Workstation running I needed to downgrade my kernel. I installed the new kernel and got everything setup. Sometime later the system reverted to the newer version.

I already had the package exclusion IgnorePjg = linux, linux-headers in pacman.conf. So I decided to purge the newer kernel 3.16 from the system. After doing so, I reinstalled the downlevel kernel 3.13. However I was not paying attention when it failed. Little did I know the /boot/ directory was empty when I rebooted. Ouch.

Thankfully everything I needed was on the system. Just need to get into it. So I boot into the archlinux.iso and setup a chroot into the existing system.

System Repair

My partion layout is as follows:

  • sda1 = /boot
  • sda2 = swap
  • sda3 = /root

Boot into the live CD and create a new directory for the mount point. Mount the /root then /boot file systems onto the new mount point.

# mkdir /mnt/arch
# mount /dev/sda3 /mnt/arch
# mount /dev/sda1 /mnt/arch/boot

Change directory into /mnt/arch and chroot the mounted filesystem.

# cd /mnt/arch
# mount -t proc proc proc/
# mount -t sysfs sys sys/
# mount -o bind /dev dev/
# chroot . /bin/bash
[[email protected] /]#

Inside the chroot you have access to your broken system. I have the kernel and headers downloaded already. Otherwise the packages could be found in the pacman cache /var/cache/pacman/pkg/.

Reinstall the kernel..

pacman -U --force linux-3.13.7-1-x86_64.pkg.tar.xz linux-headers-3.13.7-1-x86_64.pkg.tar.xz

Verify

This time I double checked /boot.

# ls /boot
total 23568
drwxr-xr-x 6 root root     1024 Sep 15 07:54 grub
-rw-r--r-- 1 root root 16859876 Sep 15 08:22 initramfs-linux-fallback.img
-rw-r--r-- 1 root root  3280187 Sep 15 08:22 initramfs-linux.img
drwx------ 2 root root    12288 Jan 29  2014 lost+found
-rw-r--r-- 1 root root  3979248 Mar 24 15:07 vmlinuz-linux

Powerline on Archlinux

Powerline on Archlinux

I recently switched from GNU Screen to Tmux. While reviewing random tmux.confs on Github I stumbled onto the screenshot below of tmux & powerline.

Powerline is a statusline plugin for vim, and provides statuslines and prompts for several other applications, including zsh, bash, tmux, IPython, Awesome and Qtile.

https://github.com/gpakosz/.tmux

powerline-tmux

An AUR package available for Powerline. However for this document I am going to detail the standard installation.

Powerline Installation:

Install python-pip (and if you don’t aleady have it, git):

$ sudo pacman -Sy python-pip git

Install Powerline:

$ su -c 'pip install git+git://github.com/Lokaltog/powerline'

Powerline is installed into: /usr/lib/python3.4/site-packages/powerline

Powerline scripts for bash, tmux, fish etc are located in: /usr/lib/python3.4/site-packages/powerline/bindings

Install the Powerline patched fonts:

$ wget https://github.com/Lokaltog/powerline/raw/develop/font/PowerlineSymbols.otf https://github.com/Lokaltog/powerline/raw/develop/font/10-powerline-symbols.conf
$ sudo mv PowerlineSymbols.otf /usr/share/fonts/
$ sudo fc-cache -vf
$ sudo mv 10-powerline-symbols.conf /etc/fonts/conf.d/

Dotfile Configuration:

Edit your various dotfiles to run their powershell configuration scripts.

~/.tmux.conf

source /usr/lib/python3.4/site-packages/powerline/bindings/tmux/powerline.conf

~/.zshrc

if [[ -r /usr/lib/python3.4/site-packages/powerline/bindings/zsh/powerline.zsh ]]; then
    source /usr/lib/python3.4/site-packages/powerline/bindings/zsh/powerline.zsh
fi

~/.bashrc

if [ -f /usr/lib/python3.4/site-packages/powerline/bindings/bash/powerline.sh ]; then
    source /usr/lib/python3.4/site-packages/powerline/bindings/bash/powerline.sh
fi

Patched fonts are required on remote systems used to connect to your powerline/tmux box. Download and install a patched font for your system. https://github.com/Lokaltog/powerline-fonts

powerline

Failure / Nothing works!

Should nothing above work as expected and/or you give up.

Uninstall Powerline:

$ su -c 'pip uninstall powerline'

Warning failed to load theme

While recklessly installing/testing some new themes I broke my GUI. Nothing seemed to be working right. After rebooting the box I was sitting at a blank screen. A quick check of ps -aux showed Gnome and Xorg running. What gives?

Fired up journalctl and saw a bunch of these listed. Yep that looks like a place I might have screwed up.

gnome-session[7460]: Window manager warning: Failed to load theme "Adwaita": Failed to find a valid file for theme Adwaita
gnome-session[7460]: Window manager warning: Failed to load theme "Simple": Failed to find a valid file for theme Simple
gnome-session[7460]: Window manager warning: Failed to load theme "ThinIce": Failed to find a valid file for theme ThinIce
gnome-session[7460]: Window manager warning: Failed to load theme "Zukitre": Failed to find a valid file for theme Zukitre
gnome-session[7460]: Window manager warning: Failed to load theme "Default": Failed to find a valid file for theme Default
gnome-session[7460]: Window manager warning: Failed to load theme "Emacs": Failed to find a valid file for theme Emacs
gnome-session[7460]: Window manager warning: Failed to load theme "Redmond": Failed to find a valid file for theme Redmond
gnome-session[7460]: Window manager warning: Failed to load theme "Moka": Failed to find a valid file for theme Moka
gnome-session[7460]: Window manager warning: Failed to load theme "Numix": Failed to find a valid file for theme Numix
gnome-session[7460]: Window manager warning: Failed to load theme "Zukitwo": Failed to find a valid file for theme Zukitwo

I quick Google search found a recommendation to reinstall the gnome-themes-standard package.

$ sudo pacman -Sy gnome-themes-standard

The package spit out a couple warnings..

pacman[7583]: warning: directory permissions differ on /usr/share/themes/Adwaita/backgrounds/
		  filesystem: 766  package: 755
pacman[7583]: warning: directory permissions differ on /usr/share/themes/Adwaita/metacity-1/
		  filesystem: 766  package: 755
pacman[7583]: warning: directory permissions differ on /usr/share/themes/Adwaita/gtk-3.0/
		  filesystem: 766  package: 755
pacman[7583]: warning: directory permissions differ on /usr/share/themes/Adwaita/gtk-2.0/
		  filesystem: 766  package: 755
pacman[7583]: warning: directory permissions differ on /usr/share/themes/Adwaita/gtk-2.0/Arrows/
		  filesystem: 766  package: 755
pacman[7583]: warning: directory permissions differ on /usr/share/themes/Adwaita/gtk-2.0/Buttons/
		  filesystem: 766  package: 755
pacman[7583]: warning: directory permissions differ on /usr/share/themes/Adwaita/gtk-2.0/Check-Radio/
		  filesystem: 766  package: 755
pacman[7583]: warning: directory permissions differ on /usr/share/themes/Adwaita/gtk-2.0/Entry/
		  filesystem: 766  package: 755

ok.. someone fat fingered the permissions.

$ sudo chmod -R 755 /usr/share/themes
$ sudo systemctl restart display-manager

Everything is back to normal.. The take away is use ~/.themes or be more careful. :)

Active directory Users and Computers on Windows 7

Download the Remote Server Administration Tools for Windows 7.

Note: Windows 7 Professional or better.

Install the software then run the follow commands to enable the AD Users & Computer snapin.

dism /online /enable-feature /featurename:RemoteServerAdministrationTools
dism /online /enable-feature /featurename:RemoteServerAdministrationTools-Roles
dism /online /enable-feature /featurename:RemoteServerAdministrationTools-Roles-AD
dism /online /enable-feature /featurename:RemoteServerAdministrationTools-Roles-AD-DS
dism /online /enable-feature /featurename:RemoteServerAdministrationTools-Roles-AD-DS-SnapIns

I received an error when trying to install the snapin directly.

c:\>dism /online /enable-feature /featurename:RemoteServerAdministrationTools-Roles-AD-DS-SnapIns

Deployment Image Servicing and Management tool
Version: 6.1.7600.16385

Image Version: 6.1.7601.17514

Enabling feature(s)
[==========================100.0%========================= ]

Error: 50

The operation completed but RemoteServerAdministrationTools-Roles-AD-DS-SnapIns feature was not enabled.
Ensure that the following parent feature(s) are enabled first. If they are already enabled, refer to the log file for further diagnostics.
RemoteServerAdministrationTools-Roles-AD-DS, RemoteServerAdministrationTools-Roles-AD, RemoteServerAdministrationTools-Roles, RemoteServerAdministrationTools

Per the error message, install one component at a time.

c:\>dism /online /enable-feature /featurename:RemoteServerAdministrationTools

Deployment Image Servicing and Management tool
Version: 6.1.7600.16385

Image Version: 6.1.7601.17514

Enabling feature(s)
[==========================100.0%========================= ]
The operation completed successfully.


c:\admin\dism /online /enable-feature /featurename:RemoteServerAdministrationTools-Roles

Deployment Image Servicing and Management tool
Version: 6.1.7600.16385

Image Version: 6.1.7601.17514

Enabling feature(s)
[==========================100.0%========================= ]
The operation completed successfully.

c:\>dism /online /enable-feature /featurename:RemoteServerAdministrationTools-Roles-AD

Deployment Image Servicing and Management tool
Version: 6.1.7600.16385

Image Version: 6.1.7601.17514

Enabling feature(s)
[==========================100.0%========================= ]
The operation completed successfully.

c:\>dism /online /enable-feature /featurename:RemoteServerAdministrationTools-Roles-AD-DS

Deployment Image Servicing and Management tool
Version: 6.1.7600.16385

Image Version: 6.1.7601.17514

Enabling feature(s)
[==========================100.0%========================= ]
The operation completed successfully.

c:\>dism /online /enable-feature /featurename:RemoteServerAdministrationTools-Roles-AD-DS-SnapIns

Deployment Image Servicing and Management tool
Version: 6.1.7600.16385

Image Version: 6.1.7601.17514

Enabling feature(s)
[==========================100.0%==========================]
The operation completed successfully.

System Volume Information folder size

Reduce the size of your System Volume Information folder by shrinking the shadow storage.

Use vssadmin list shadowstorage to list the current Shadow Copy Storage size.

C:\>vssadmin list shadowstorage
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Shadow Copy Storage association
For volume: (C:)\\?\Volume{278ce142-19d8-11e1-9176-806e6f6e6963}\
Shadow Copy Storage volume: (C:)\\?\Volume{278ce142-19d8-11e1-9176-806e6f6e69
63}\
Used Shadow Copy Storage space: 15.681 GB (39%)
Allocated Shadow Copy Storage space: 16.152 GB (40%)
Maximum Shadow Copy Storage space: UNBOUNDED (100%)

In the example above there is no limit set on Maxium storage. To set a limit use the maxsize= command. In Windows Server 2008+ this can be set as a percentage of disk space.

C:\>vssadmin resize shadowstorage /on=C: /For=C: /Maxsize=4GB
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Successfully resized the shadow copy storage association

After setting maxsize rerun list shadowstorage to verify/review the updated configuration.

C:\>vssadmin list shadowstorage
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Shadow Copy Storage association
For volume: (C:)\\?\Volume{278ce142-19d8-11e1-9176-806e6f6e6963}\
Shadow Copy Storage volume: (C:)\\?\Volume{278ce142-19d8-11e1-9176-806e6f6e6963}\
Used Shadow Copy Storage space: 2.81 GB (7%)
Allocated Shadow Copy Storage space: 3.279 GB (8%)
Maximum Shadow Copy Storage space: 4 GB (10%)

Remove Ghosted nics

Found a great script for removing ghosted (hidden) NICS on Frank Peter’s blog.

Using the Microsoft devcon.exe tool the script compares the output of find and findall. If the matching device count is not equal the additional device is removed.

  • find - Find devices that match the specific hardware or instance ID.
  • findall - Find devices including those that are not present.

The Device Console Utilitu (DevCon) is available from support.microsoft.com however this version is outdated. An up to date version can be acquired from the Windows Driver Kit.

Detailed instructions on how to extract DevCon can be found on technet.microsoft.com.

  • Download Windows Driver Kit. See “Links” section for dowonload [sic] location of the most recent version of WDK.
  • Extract or mount ISO.
  • To obtain DevCon.exe you need either to:
    • install WDK: * -execute KitSetup.exe and follow on-screen instructions.
    • or perform Administrative Installation of “setuptools_<Platform>fre” Microsoft Software Installer (MSI) package:
    • execute the following command:
    • "%SystemRoot%\System32\msiexec.exe" /a "<path to WDK>\setuptools_<Platform>fre.msi" targetdir="<Desired Path>"
  • for example, assuming that: *you need DevCon.exe for x64 (aka AMD64 aka EM64T) platform,
    • you mounted WDK ISO as drive D:,
    • you want to find DevCon.exe under your Temporary directory;
  • you should execute:
    • "%SystemRoot%\System32\msiexec.exe" /a "D:\WDK\setuptools_x64fre.msi" targetdir="%temp%"

Keep in mind that you should have a working local administrator account and password. Should networking break its the only way back into the server.

@ECHO OFF

devcon.exe findall =net >%TEMP%\%~n0-findall.tmp
devcon.exe find =net >%TEMP%\%~n0-find.tmp

FOR /F %%i IN ('find.exe "matching device" ^<%TEMP%\%~n0-findall.tmp') DO SET findall=%%i
FOR /F %%i IN ('find.exe "matching device" ^<%TEMP%\%~n0-find.tmp') DO SET find=%%i

IF %findall% EQU %find% (
    ECHO Nothing to do.
    GOTO :End
)

FOR /F "TOKENS=1,2 DELIMS=:" %%i IN ('find.exe /V "matching device" ^<%TEMP%\%~n0-findall.tmp') DO (
    find.exe "%%i" <%TEMP%\%~n0-find.tmp >NUL || (
        ECHO Remove "%%j"
        devcon.exe remove "@%%i"
    )
)

:End
DEL %TEMP%\%~n0-*.tmp

Using Winsat to test hard drive performance

The Windows Assessment Tool (WinSAT) is included Microsoft Vista and Windows 7. This tool can perform performance tests including CPU, memory, disk and video.

winsat disk -drive c

Windows System Assessment Tool
> Running: Feature Enumeration ''
> Run Time 00:00:00.00
> Running: Storage Assessment '-drive c -seq -read'
> Run Time 00:00:05.48
> Running: Storage Assessment '-drive c -ran -read'
> Run Time 00:00:06.99
> Running: Storage Assessment '-drive c -scen 2009'
> Run Time 00:01:13.62
> Running: Storage Assessment '-drive c -seq -write'
> Run Time 00:00:36.26
> Running: Storage Assessment '-drive c -flush -seq'
> Run Time 00:00:05.94
> Running: Storage Assessment '-drive c -flush -ran'
> Run Time 00:00:12.18
> Running: Storage Assessment '-drive c -hybrid -ran -read -ransize 4096'
NV Cache not present.
> Run Time 00:00:00.02
> Running: Storage Assessment '-drive c -hybrid -ran -read -ransize 16384'
NV Cache not present.
> Run Time 00:00:00.01
> Disk  Sequential 64.0 Read                   111.23 MB/s          6.7
> Disk  Random 16.0 Read                       2.43 MB/s          4.4
> Responsiveness: Average IO Rate              3.57 ms/IO          6.0
> Responsiveness: Grouped IOs                  9.67 units          7.2
> Responsiveness: Long IOs                     8.72 units          7.5
> Responsiveness: Overall                      84.37 units          6.9
> Responsiveness: PenaltyFactor                0.0
> Disk  Sequential 64.0 Write                  116.12 MB/s          6.8
> Average Read Time with Sequential Writes     6.720 ms          5.4
> Latency: 95th Percentile                     30.732 ms          3.4
> Latency: Maximum                             56.218 ms          7.8
> Average Read Time with Random Writes         15.075 ms          3.4
> Total Run Time 00:02:22.10

Uninstall LibreOffice on Redhat/Fedora

Uninstall any existing installation of LibreOffice.

$ sudo yum remove libreoffice*

Change directory into the new extracted LibreOffice package. Install all of the rpms.

$ sudo rpm -Uvh *.rpm

Change directory to the desktop-integration and install the desktop additions.

$ cd desktop-integration
$ sudo rpm -Uvh *freedesktop*.rpm

CAPI2 error - Missing System Writer on Windows 2008

System Writer is missing from the output of vssadmin list writers. The Application Event log is showing Event ID 513 source CAPI2.

Had a hell of a time resolving this issue. Most of the sources point to a permissions issue. However the security log remains error free. The script below is combines a couple suggested fixes. One of the fixed is from the Microsoft KB article 2009272. Other items were found in various forums.

Either run the script as is or break it down in parts to show the exact issue. Be sure to restart the cryptographic service if not running the script in its entirety.

::  Reset Reg and Dir ACLs

Takeown /f "%windir%\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files" /a

icacls "%windir%\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files" /grant "NT AUTHORITY\SYSTEM:(RX)"
icacls "%windir%\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files"  /grant "NT Service\trustedinstaller:(F)"
icacls "%windir%\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files"   /grant "BUILTIN\Users:(RX)"
icacls "%windir%\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files" /grant "Administratoren:(RX)"

Takeown /f %windir%\System32\DriverStore /a

icacls %windir%\System32\DriverStore /grant "NT AUTHORITY\SYSTEM:(RX)"
icacls %windir%\System32\DriverStore /grant "NT Service\trustedinstaller:(F)"
icacls %windir%\System32\DriverStore /grant "BUILTIN\Users:(RX)"
icacls %windir%\System32\DriverStore /grant "Administratoren:(RX)"

Regini Registry\Machine\System\CurrentControlSet\services [1 5 7 11 14 17 21]
Regini Registry\Machine\Software\Microsoft\WBEM\CIMOM [1 5 7 11 14 17 21]


:: System state backup failed w/ The system writer is not found in the backup
:: kb2009272
:: vssadmin list writers = "System Writer" missing

Takeown /f %windir%\winsxs\temp\PendingRenames /a

icacls %windir%\winsxs\temp\PendingRenames /grant "NT AUTHORITY\SYSTEM:(RX)"
icacls %windir%\winsxs\temp\PendingRenames /grant "NT Service\trustedinstaller:(F)"
icacls %windir%\winsxs\temp\PendingRenames /grant BUILTIN\Users:(RX)

Takeown /f %windir%\winsxs\filemaps\* /a

icacls %windir%\winsxs\filemaps\*.* /grant "NT AUTHORITY\SYSTEM:(RX)"
icacls %windir%\winsxs\filemaps\*.* /grant "NT Service\trustedinstaller:(F)"
icacls %windir%\winsxs\filemaps\*.* /grant BUILTIN\Users:(RX)


:: Restart the Cryptographic Services

net stop cryptsvc
net start cryptsvc

BOINC Linux GPU

As of version 7.0.40 BOINC has the ability to split GPU tasks across multiple work units.

boinc_manager_gpu_tasks

Preliminary Change Log 7.0.39 –> 7.0.40:

client: add “client app configuration” feature; see  This lets users do the following: 1) limit the number of concurrent jobs of a given app (e.g. for WCG apps that are I/O-intensive) 2) Specify the CPU and GPU usage parameters of GPU versions of a given app.

BOINC development versions are found here: http://boinc.berkeley.edu/download_all.php

Create a new file named app_config.xml in the project directory to make use of the new options. Details are found under ClientAppConfig of the BOINC wiki.

On Ubuntu the project directory is found: /var/lib/boinc-client/projects/ProjectName

<app_config>
   <app>
      <name>hcc1</name>
      <max_concurrent>4</max_concurrent>
      <gpu_versions>
          <gpu_usage>.25</gpu_usage>
          <cpu_usage>1</cpu_usage>
      </gpu_versions>
   </app>
</app_config>
  • name: Short name of the project. Project short names are found in the client_state.xml file.
  • max_concurrent: This specifies the maximum number of tasks the app can run at a time. Set this to the number of total HCC1 tasks your computer will be doing at a time.
  • gpu_usage: This specifies the % use of the GPU per task. Set this according to how many tasks you want to run per GPU.

    Tasks per GPU = gpu_usage

  • cpu_usage: This specifies the % use of CPU threads per GPU task. The default value is 1, but if you want to run more GPU tasks than you have CPU cores edit it accordingly.

    Total CPU Threads / Total GPU Task = cpu_usage

Detect .NET version using Internet Explorer

Here is a quick method of getting the .NET versions running on a system.

Past the following text into the Internet Explorer address bar.

javascript:alert(navigator.userAgent)

useragent

Note: Be aware that the userAgent response can be modified.

Curses based full screen BOINC manager

Do you have a couple boxes running headless or otherwise not running Xorg? If you ever wanted to or had the need there is a Curses based BOINC manager. The project name is Boinctui and is available on Google Code.

Boinctui source code along with deb and rpm binaries are available for download.

wget http://code.google.com/p/boinctui/downloads/detail?name=boinctui_1.0.0-0_i386.deb
sudo dpkg -i boinctui_1.0.0-0_i386.deb

boincui

iPad ad blocking using privoxy

I recently acquired an iPad 2 which I use mostly for surfing the web. I had no idea that you could not use browser extensions.

Assuming you have an extra computer running on your network. One method of blocking ads (on your home network) is to use a web proxy like Privoxy. Privoxy has binaries for Windows, OS X, BSD and Linux.

In the example below we are going to install and setup Privoxy on Ubuntu.

sudo apt-get install privoxy

Edit the configuration file and change the listen-address from 127.0.0.1 to your servers IP address. In the example below we will use 192.168.1.1.

sudo vim /etc/privoxy/config
#  Example:
#
#      Suppose you are running Privoxy on a machine which has the
#      address 192.168.0.1 on your local private network (192.168.0.0)
#      and has another outside connection with a different address. You
#      want it to serve requests from inside only:
#
#        listen-address  192.168.0.1:8118
#
#      Suppose you are running Privoxy on an IPv6-capable machine and
#      you want it to listen on the IPv6 address of the loopback device:
#
#        listen-address [::1]:8118
#
listen-address 192.168.1.1:8118
#
#
#  4.2. toggle
#  ============
#

Restart Privoxy after editing the config file. Privoxy will now be listening on port 8118 with the default settings.

AdblockPlus lists for Privoxy.

There is a script that will convert the AdblockPlus lists for Privoxy. http://andrwe.org/scripting/bash/privoxy-blocklist Development version can be found on Github

Download the script:

wget http://andrwe.org/_export/code/scripting/bash/privoxy-blocklist

Make it executable and run:

$ chmod +x privoxy-blocklist.sh
$ sudo ./privoxy-blocklist.sh

Verify Privoxy is running and listening on port 8118:

$ sudo netstat -anp | grep privoxy
tcp        0      0 192.168.1.1:8118    0.0.0.0:*               LISTEN      15152/privoxy

Configure the Ipad proxy settings

  • Tap Settings > General > Network > WiFi.
  • Modify the network settings for the SSID at the location by pressing on the blue icon with the right arrow in it.
  • Select “Manual” under “HTTP Proxy”.
  • For the “server”, type the IP address of the proxy server. For the “Port”, enter your proxy server port (typically 8118 for Privoxy)

Now you up and running AD free.

Improve Google Reader readability with userstyles.org and stylish

As with most Google products Reader is utilitarian.

Install the Stylish Google Chrome extension.

Restyle the web with Stylish, a user styles manager. Stylish lets you easily install themes and skins for Google, Facebook,… User styles are themes for web sites. User styles empower your browsing experience by letting you customize web sites. Take out irrelevant content, change colors, or completely redesign the entire site.

Stylish lets you easily manage user styles. Add, delete, enable, disable, and organize with a few clicks of a mouse, no code to edit, no obscure configuration to find. Stylish’s companion website, userstyles.org, hosts tens of thousands of user styles made by other Stylish users that you can try.

For you technical types out there, think of it this way: Stylish and userstyles.org are to CSS as Greasemonkey and userscripts.org are to JavaScript.

Head on over to Userstyles.org and browse the style collections.

Examples of vanilla vs styled Google Reader:

google-reader

stylish

gpedit.msi could not create snap-in

When opening gpedit.msc you receive the following error:

MMC could not create the snap-in. The snap-in might not have been installed correctly.

gpedit-msc-error

Verify that “%SystemRoot%\System32\Wbem” is in the PATH environment variable.

echo %PATH%

Add the environment variable if it is missing and rerun gpedit.msc.

Set recursive file or directory permission

Command to recursively change either file or directory permissions in Bash.

Example of changing directory permissions to 755

sudo find . -type d -exec chmod 755 {} \;

Example of changing file permissions to 666

sudo find . -type f -exec chmod 666 {} \;

Solarized colors on Gnome terminal

Solarized is a beautiful color palette. The palette was designed for readability and comes with both light and dark options.

Solarized is a sixteen color palette (eight monotones, eight accent colors) designed for use with terminal and gui applications. It has several unique properties. I designed this color scheme with both precise CIELAB lightness relationships and a refined set of hues based on fixed color wheel relationships. It has been tested extensively in real world use on color calibrated displays (as well as uncalibrated/intentionally miscalibrated displays) and in a variety of lighting conditions.

solarized-palette

Install git-core on Ubuntu/Deb. Adjust according to your distribution installer.

$ sudo apt-get install git-core
$ git clone https://github.com/sigurdga/gnome-terminal-colors-solarized.git

Change directory into the downloaded Solarized directory

$ cd gnome-terminal-colors-solarized

Run one of the following scripts for either the dark or light color theme.

$ ./set_dark.sh
$ ./set_light.sh

There are Solarized color palettes available for terminals, editors, IDEs etc. Easy to set up and beautiful to look at. I highly recommend checking it out.

Bash 4 recursive globbing

Bash version 4 introduced a new option called globstar. When the option is set, recursive globbing with ** is enabled.

Lets say you want to move all files from a directory and its subdirectories. Most common method would be to use the find command.

For example, find and move all .mp3 files from the current and all subdirectories to a directory named music.

$ sudo find . -type f -exec mv *.mp3 /music {} \;

The can be simplified by enabling the Bash option globstar.

$ shopt -s globstar
$ shopt |grep globstar

globstar        on

With the BASH globstar setting enabled the command is as simple as:

$ mv **/*.mp3 /music

Citrix - Unable to connect to the CGP tunnel destination

Citrix users are receiving an error on login.

citrix-error

The Citrix server is reporting a number of Event ID: 118 errors.

event-id-118

Open the Terminal Services Manager and check for disconnected listeners. In my case the ICA Listener was down (disconnected). Resetting the listener resolved the issue.

Windows 2008 access denied killing a process

There are multiple times where I have been unable to kill a process on Windows 2008 server. Generally the process is running as a domain account. Killing the process failed using an administrative prompt and/or the account the process was started with.

The process is owned by the System account. You can use the Sysinternals tool psexec with the -s switch to run as the System account. The -i switch allows the process to interact with the desktop.

psexec -i -s pskill someprocess.exe`

You can use psexec to call a CMD prompt, taskkill, etc.

Getting the most from GNU Screen

Via gnu.org >Screen is a full-screen window manager that multiplexes a physical terminal between several processes, typically interactive shells. Each virtual terminal provides the functions of the DEC VT100 terminal and, in addition, several control functions from the ANSI X3.64 (ISO 6429) and ISO 2022 standards (e.g., insert/delete line and support for multiple character sets). There is a scrollback history buffer for each virtual terminal and a copy-and-paste mechanism that allows the user to move text regions between windows. When screen is called, it creates a single window with a shell in it (or the specified command) and then gets out of your way so that you can use the program as you normally would. Then, at any time, you can create new (full-screen) windows with other programs in them (including more shells), kill the current window, view a list of the active windows, turn output logging on and off, copy text between windows, view the scrollback history, switch between windows, etc. All windows run their programs completely independent of each other. Programs continue to run when their window is currently not visible and even when the whole screen session is detached from the users terminal.

Screen will allow you to keep a persistent terminal session. Even when the connection is lost. In the middle of compiling a new kernel when your network connection drops? No problem.. You were in a screen session right?

###Running Screen

Screen is simple enough to use. To start a new session in a terminal

screen

Attach to an existing screen session

screen -r

Force reattach to an existing session

screen -rd

Share a screen session (multi display mode)

screen -x

###Basic Keyboard Commands

Here are some basics commands to help you get around.

The following table shows the default key bindings:

C- represents the ctrl key. The first command could be executed by pressing ctrl a followed by c

C-a c C-a C-c (screen) Create a new window with a shell and switch to that window.

C-a d C-a C-d (detach) Detach screen from this terminal.

C-a D D (pow_detach) Detach and logout.

C-a k C-a C-k (kill) Destroy the current window.

C-a C-a n C-a C-n (next) Switch to the next window.

C-a p C-a C-p C-a C-h C-a (prev) Switch to the previous window (opposite of C-a n).

C-a x C-a C-x (lockscreen) Lock your terminal.

C-a z C-a C-z (suspend) Suspend screen.

C-a ? (help) Show key bindings.

C-a C-\ (quit) Kill all windows and terminate screen.

The commands above are just the basics. They will get the job done on their own. There are many more options available. Default-Key-Bindings

###Example .screenrc for a webserver:

# basics
startup_message off
vbell           on
defscrollback   10000
autodetach      on

# status
hardstatus alwayslastline "%{= kw}%{g}[ %{R}%H %{g}] %{Y} %{g}[%=%{ =kw}%{w}%-w%{Y}[%{W}%n-%t%{Y}]%{w}%+w%=%{g}][ %{w}%m-%d %{Y}%c %{g}]"

# add caption
caption splitonly "%{= kw}%?%-Lw%?%{kw}%n*%t%f %?(%u)%?%{= kw}%?%+Lw%?"

screen -t shell 0 bash
screen -t vim 1 bash
screen -t htop 2 htop
screen -t apache 3 goaccess
screen -t log 4 tail -f -n 50 /var/log/apache2/error.log
screen -t log 5 tail -f -n 50 /var/log/messages
select 0

Additional Information on GNU Screen:

http://www.gnu.org/software/screen http://www.gnu.org/software/screen/manual http://en.wikipedia.org/wiki/GNU_Screen