Category Archives: Guides

A collection of tutorial, how to guides and tips on system administration or other technology related posts.

Install proxmox on a partition instead of a full-disk

By default, installing Proxmox with ZFS during the installation process will force you to use the entire disk for the root zpool. For most installs this is good enough. However, I like to do things differently sometimes.

I have a pair of Samsung 840 Pro 256GB SSDs that I wanted to use for my new homelab that I am currently building (moving from vmware to proxmox). You may be wondering why I want to install the operating system on a partition instead of an entire disk. Several reasons:
Continue reading

Troubleshooting networking issues after fresh install of proxmox VE 4.4

Writing a quick troubleshooting guide and informative post to address an issue I came across when installing Proxmox VE 4.4 on two of my machines.

On servers with more than two network interfaces Debian/Proxmox renames all interfaces and does not properly detect eth0 as the on-board ethernet as many other linux flavors. This may cause a mild headache if you just installed Proxmox with static IP addresses using the installer and upon reboot you can’t access any network resources. Continue reading

How to flash Seagate firmware onto HP enterprise MB2000EAMZF drives

In my previous post I talked about how HP locks down their hard drives (made by Seagate) with custom firmware that is exclusive to HP and with the huge caveat that if you do not have an HP storage controller the software update package provided by HP won’t work for you at all.

I spent the past 48 hours researching and trying different things. I unpacked the official HP firmware and tried to find the binary/firmware dump from within the .scexe file provided by HP on their website to no avail.  I even contacted HP for help and they would not be willing to help if the drives were not connected to HP hardware.

After some research I discovered that the HP MB2000EAMZF 2TB drive I had was almost exactly the same (part number) as the official Seagate ST32000644NS  drive after failing to be able to flash HPs firmware using HP’s bootable Firmware Update 8.3 (injecting the .scexe files inside this live CD) – I decided I really had nothing to lose but try the method I will explain below.
Continue reading

Useful software development tools

This post is meant to be updated with useful tools for software development. I will only classify the tool and post a link to it, this post will be helpful for someone just starting out in devOps

API

  • Postman: allows to share, test, document & monitor APIs.

IDE

  • Visual Studio: Windows software development IDE (C#/WPF)
  • Jetbrains Webstorm: Javascript IDE that becomes powerful with plugins (I use it for writing Go code)

Text editors

Must have tools

  • Git (or Github Desktop or Git for Windows)
  • Linux terminal/bash (available on any Mac/Linux machine, for Windows download Git for Windows linked above)

cPanel DNSonly bind recursion

In case you may be running into issues with named/bind domain service on cPanel DNSonly not responding to DNS queries recursively, I have a fix for you.

In older versions of named/bind9 used by cPanel if you wanted to allow anyone (or if you wanted security you could have setup an ACL, let’s assume you want to allow all) you could simply edit the configuration file and change “recursion no;” to “recursion yes;” to allow anyone to make queries to your DNS server for those domains or records that are not kept in your local server (ie: resolve yahoo.com)
Newer versions of BIND9/named changed their past behavior and now require a more specific configuration, just add these under “options” section

options {
...
allow-recursion { any; };
allow-query { any; };
allow-query-cache { any; };
...
};

I found a detailed explanation on why bind/named dns server changed its behavior, on the next page  a copy of the support bulletin from July 2007:

Overriding Google Compute Engine hostname from getting reset

On my last post, I explained how I am experimenting with Google Compute Engine (GCE) to host my backup DNS service and my post about configuring mailjet as an exim mail relay.

I have run into a few issues on Google Compute Engine, it looks like every time the instance gets a DHCP offer the hostname of the machine will get reset to the internal hostname, overriding whatever setting you may have set… this is bad news for a server that needs to respond to DNS settings.

After a few hours of digging and testing out different approaches, including trying to setup a DHCP hook to run the “hostname” linux command I came across documentation from Google talking about “custom metadata”.

How to setup Mailjet on exim for cPanel dnsonly on Google Cloud

I have been experimenting with Google Compute Engine (GCE) for the past few days. I wanted to migrate a slave DNS server to Google as an experiment and this blog post will talk about the shortcomings, my workarounds, configuration and tips to achieve the goal of having cPanel DNSonly installed and setup using Google’s infrastructure.

What is Google Compute Engine?

Google Compute Engine (or “GCE”) is Google’s answer to Amazon’s AWS, it basically offers you the ability to deploy Virtual Machines (VMs) at Google’s datacenters. In the most simplistics of setups you would use a single VM or “compute instance” and if you wanted to have a more complex setup you could deploy multiple instances in different geographic locations spread across different datacenters in the United States to achieve a more redundant setup or to provide you with better capacity for large traffic workflows.

It all depends on what your goals are, myself I am just starting out with a simple requirement. I would like to use a single compute instance running CentOS linux that runs cPanel DNSonly (proprietary control panel/dns clustering solution for cPanel servers).

The goal is not to host websites or any critical services on this instance, but rather only keep a copy of my DNS zones. If my experiment fails or my instance goes down or gets destroyed there won’t be any losses or damages for this experiment as my primary server is still running on Proxmox PVE.

The limitations of GCE… What I learned so far

GCE instances run behind Google’s firewalls and depending on the datacenter you decide to host your instance your VM will have an internal IP within that datacenters network scope/range (for example: us-central1 datacenter zone has a 10.128.0.0/20 range for all VMs).

This means that assigning a IPv4 public static IP, which is needed to allow anyone to query the DNS server we’ll be hosting inside this instance will actually have to forward packets to the instances internal network. Not really a big deal unless your application can’t be behind a NAT firewall like it is the case here.

For our intensive use and purposes it is fine. One important issue that I found is that doing any kind of e-mail server related activities on GCE is prohibited (source). You won’t be able to use SMTP (Simple Mail Transfer Protocol) port 25, 465 or 587 to send or receive email.

Now for a DNS server that will not be a mailserver this is not a blocker, but it is really a pain in the ass. Why? because monitoring scripts running on the server need to email the administrator to notify me of issues or errors so they can be investigated.

Continued on the next page.

SecureCRT mapping “Page Up” and “Page Down” for Nano/SSH

If you use a windows ssh terminal client, or even macosx’s terminal.app, the functionality of your page up and page down key may not behave like you want it to. For example, in SecureCRT using pageup will actually page up your scroll buffer, and pagedown will scroll down your page buffer.

You’ll need to remap your keys to send the correct signal to your terminal. Change it from the default system function to “send string” and the following strings:

PageUp = 33[5~
PageDown = 33[6~

33 is a shortcut for the ESCAPE key (esc) since in some programs you can’t type in the actualy escape key.

Here are some other codes as well in case you need to remap keys.

Pause = 32
Macro = 03 # Break (Shift-Pause)
Home = 33[1~
Insert = 33[2~
Remove = 33[3~ # Delete
End = 33[4~

In SecureCRT the area to remap your keys is Tools -> Keymap Editor.

This is especially handy if you use command line IRC clients such as Irssi or BitchX.

Checking for Hard drive READ and WRITE Cache (onboard) on Solaris

To check for read and write cache for your hard drives do the following:

Giovanni@server:~# format -e
Searching for disks…done
AVAILABLE DISK SELECTIONS:
0. c8t0d0 <DEFAULT cyl 60797 alt 2 hd 255 sec 252>
/pci@0,0/pci15d9,d380@1f,2/disk@0,0
1. c8t1d0 <ATA-Hitachi HDS72202-A3EA-1.82TB>
/pci@0,0/pci15d9,d380@1f,2/disk@1,0
2. c8t2d0 <ATA-Hitachi HDS72202-A28A-1.82TB>
/pci@0,0/pci15d9,d380@1f,2/disk@2,0
3. c8t3d0 <ATA-Hitachi HDS72202-A3EA-1.82TB>
/pci@0,0/pci15d9,d380@1f,2/disk@3,0
4. c8t4d0 <ATA-Hitachi HDS72202-A3EA-1.82TB>
/pci@0,0/pci15d9,d380@1f,2/disk@4,0
5. c8t5d0 <ATA-Hitachi HDS72202-A3EA-1.82TB>
/pci@0,0/pci15d9,d380@1f,2/disk@5,0
Specify disk (enter its number):

Select a drive, lets pick 5 from the list.

Specify disk (enter its number): 5
selecting c8t5d0
[disk formatted]
/dev/dsk/c8t5d0s0 is part of active ZFS pool gpool. Please see zpool(1M).
FORMAT MENU:
disk       – select a disk
type       – select (define) a disk type
partition  – select (define) a partition table
current    – describe the current disk
format     – format and analyze the disk
fdisk      – run the fdisk program
repair     – repair a defective sector
label      – write label to the disk
analyze    – surface analysis
defect     – defect list management
backup     – search for backup labels
verify     – read and display labels
inquiry    – show vendor, product and revision
scsi       – independent SCSI mode selects
cache      – enable, disable or query SCSI disk cache
volname    – set 8-character volume name
!<cmd>     – execute <cmd>, then return
quit
format>

Now let’s do the checking

Enter “cache” to enter cache menu.

CACHE MENU:
write_cache – display or modify write cache settings
read_cache  – display or modify read cache settings
!<cmd>      – execute <cmd>, then return
quit
cache>

Type: “write_cache” or “read_cache” depending on what you would like to see, lets use write:

cache> write_cache
WRITE_CACHE MENU:
display     – display current setting of write cache
enable      – enable write cache
disable     – disable write cache
!<cmd>      – execute <cmd>, then return
quit
write_cache> display
Write Cache is enabled
write_cache>

Use the same for read_cache and to disable and enable.

Setup Filebench on Solaris for benchmarking

Like any other newbie on Solaris, I didn’t know how to install the packages, I am used to yum or apt-get install but anyway on Solaris I did:

Giovanni@server:~/Downloads/filebench-1.4.8# pkg install SUNWfilebench
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                    1/1       60/60     0.32/0.32

PHASE                                        ACTIONS
Install Phase                                  82/82
Giovanni@server:~/Downloads/filebench-1.4.8#

and it was installed 🙂 Use pkg search to search for packages.

Giovanni@server:/usr/benchmarks/filebench# bin/go_filebench
FileBench Version 1.4.4
filebench> load varmail
742: 3.707: Varmail Version 2.1 personality successfully loaded
742: 3.707: Usage: set $dir=<dir>
742: 3.707:        set $filesize=<size>    defaults to 16384
742: 3.707:        set $nfiles=<value>     defaults to 1000
742: 3.707:        set $nthreads=<value>   defaults to 16
742: 3.707:        set $meaniosize=<value> defaults to 16384
742: 3.707:        set $readiosize=<size>  defaults to 1048576
742: 3.707:        set $meandirwidth=<size> defaults to 1000000
742: 3.707: (sets mean dir width and dir depth is calculated as log (width, nfiles)
742: 3.707:  dirdepth therefore defaults to dir depth of 1 as in postmark
742: 3.707:  set $meandir lower to increase depth beyond 1 if desired)
742: 3.707:
742: 3.707:        run runtime (e.g. run 60)
filebench> set $dir=/gpool
filebench> run 60
742: 27.078: Creating/pre-allocating files and filesets
742: 27.081: Fileset bigfileset: 1000 files, 0 leafdirs avg dir = 1000000, avg depth = 0.5, mbytes=15
742: 27.096: Removed any existing fileset bigfileset in 1 seconds
742: 27.096: making tree for filset /gpool/bigfileset
742: 27.096: Creating fileset bigfileset…
742: 35.092: Preallocated 812 of 1000 of fileset bigfileset in 8 seconds
742: 35.092: waiting for fileset pre-allocation to finish
742: 35.092: Starting 1 filereader instances
744: 36.102: Starting 16 filereaderthread threads
742: 39.112: Running…
742: 99.712: Run took 60 seconds…
742: 99.713: Per-Operation Breakdown
closefile4                449ops/s   0.0mb/s      0.0ms/op        3us/op-cpu
readfile4                 449ops/s   7.0mb/s      0.0ms/op       19us/op-cpu
openfile4                 449ops/s   0.0mb/s      0.0ms/op       18us/op-cpu
closefile3                449ops/s   0.0mb/s      0.0ms/op        3us/op-cpu
fsyncfile3                449ops/s   0.0mb/s     17.4ms/op       20us/op-cpu
appendfilerand3           449ops/s   3.5mb/s      0.0ms/op       27us/op-cpu
readfile3                 449ops/s   7.0mb/s      0.0ms/op       18us/op-cpu
openfile3                 449ops/s   0.0mb/s      0.0ms/op       18us/op-cpu
closefile2                449ops/s   0.0mb/s      0.0ms/op        3us/op-cpu
fsyncfile2                449ops/s   0.0mb/s     17.9ms/op       17us/op-cpu
appendfilerand2           449ops/s   3.5mb/s      0.0ms/op       23us/op-cpu
createfile2               449ops/s   0.0mb/s      0.1ms/op       52us/op-cpu
deletefile1               449ops/s   0.0mb/s      0.0ms/op       33us/op-cpu

742: 99.713:
IO Summary:      353667 ops, 5836.1 ops/s, (898/898 r/w)  21.0mb/s,     78us cpu/op,   8.9ms latency
742: 99.713: Shutting down processes
filebench>
742: 110.144: Aborting…

Going back to normal