Category Archives: Tech

ntp.yaojun.sg – Stratum-1 NTP server on a Raspberry Pi

Posted on by .

I have been running ntp.yaojun.sg for NTP service on a Raspberry PI for a while. This is service is part of pool.ntp.org and responds to sg.pool.ntp.org and asia.pool.ntp.org. However, my SD card decided to give up recently when I wanted to explore SNMP tools for monitoring purposes.

The GPS module I am using is from HAB. I gotten this particular module late last year built specifically for the older Raspberry Pi Model B. My NTP service was made public around March 2014 using a ready made image. However, just looking at how 2014 turn out, it is important to keep both base system and NTP patched. In addition, it is not advisable to just run a “BlackBox” because you never know what’s going on inside it. So here I am restoring the service from a clean Raspbian image.

Do note that the module requires a GPS 3D lock for PPS to work, so remember to connect your antenna. I was scratching my head on why my PPS was not working and had to contact the friendly vendor to ask if I have a dead card.

Installing Raspbian into Raspberry Pi is a straight forward process. After the basic configuration (resizing the File System, locale, timezone and etc), remember to configure Rasbian to disable the serial shell under the advanced settings. Also update and patch the system.

apt-get update
apt-get dist-upgrade
rpi-update

Recent development in Raspbian allows the use of PPS devices without recompiling the kernel. We need to configure the system to use the PPS on the GPIO port.

Edit /boot/config.txt add the following line,

dtoverlay=pps-gpio,gpiopin=18

Add the following line to /etc/modules,

pps-gpio

Install the GPS and PPS tools

apt-get install pps-tools libcap-dev gpsd gpsd-clients python-gps

Configure gpsd to use the GPS sensor at /dev/ttyAMA0 and enable -n as a option using dpkg-reconfigure to allow ntpd to use the GPS clock and remember to enable the gpsd daemon

systemctl enable gpsd.socket
dpkg-reconfigure gpsd

and you will be able to see your GPS data using

cgps -s

Check that your PPS clock is working

ppstest /dev/pps0

Remove default NTP client (which does not work with the GPS and PPS)

apt-get remove ntp

Install NTP dependencies to ensure successful build of NTP from source

apt-get install libbsd-dev libssl-dev

Install checkinstall to compile source to binaries safely

apt-get install checkinstall

Get the latest NTP source code from www.ntp.org, untar and compile. This is not a fancy high speed CPU so compiling will take some time (estimate 30 minutes).

./configure --enable-linuxcaps --with-NMEA --with-ATOM
make 
checkinstall

Follow through checkinstall wizard.

Symbolically link compiled binaries to well defined paths,

ln -s /usr/local/bin/ntp* /usr/bin/
ln -s /usr/local/sbin/ntp* /usr/sbin/

Edit /etc/init.d/ntp to use the compiled binaries,

#PATH=/sbin:/bin:/usr/sbin:/usr/bin
PATH=/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
#DAEMON=/usr/sbin/ntpd
DAEMON=/usr/local/sbin/ntpd

Edit /etc/ntp.conf to use PPS and GPS as Stratum-0 and Stratum-1. This will make PPS disciplined by GPS and verified by other NTP servers.

# pps-gpio on /dev/pps0
server 127.127.22.0 minpoll 4 maxpoll 4
fudge 127.127.22.0 refid PPS
fudge 127.127.22.0 flag3 1  # enable kernel PLL/FLL clock discipline
# gpsd shared memory clock
server 127.127.28.0 minpoll 4 maxpoll 4 prefer  # PPS requires at least one preferred peer
fudge 127.127.28.0 refid GPSD
fudge 127.127.28.0 time1 +0.150 stratum 1 # coarse processing delay offset

Lock the ntp binaries to prevent apt from overwriting from packages

apt-make hold ntp

Nagios (Part 1) – Installing Nagios 4 on FreeBSD 10.1

Posted on by .

Nagios is one of the popular network and systems monitoring tool available. There are two main flavours of Nagios, Nagios XI the enterprise software and Nagios-Core the “Open-Source” version. This post serves as a tutorial on setting it up on FreeBSD 10.1-Release that runs inside a vSphere environment.

Update FreeBSD base system

freebsd-update fetch
freebsd-update install

Get FreeBSD Ports Tree

portsnap fetch extract update

One of the strength of FreeBSD is the FreeBSD ports tree which is a huge repository of make files for installation in FreeBSD. FreeBSD ports tree are maintained by the FreeBSD ports team and they are well maintained and very much up to date. Software can be compiled from source with the use of additional flags made possible by the make system in FreeBSD. If compiling is not your cup of tea, FreeBSD 10 also has the pkg system where binary packages are download from a repository tree much like yum in RHEL/Centos and apt-get in Debian/Ubuntu.

Setting Up OpenSSL

With a handful of SSL related vulnerabilities made public this year, it is advisable to use the Ports version of OpenSSL as it is much more current as compared with the one that came with the base OS. Many applications have a dependency on OpenSSL for SSL connections, so it is advisable to install this first in a new system.

cd /usr/ports/security/openssl
make config-recursive
make install clean

In order to use the ports version of openssl we need to edit /etc/make.conf,

echo "WITH_OPENSSL_PORT=YES" >> /etc/make.conf

Configuring OpenSSL

mv /usr/bin/openssl /usr/bin/openssl.default
ln -s /usr/local/bin/openssl /usr/bin/
cp /usr/local/openssl/openssl.cnf.example /usr/local/openssl/openssl.cnf

Setting Up PortMaster

PortMaster is a tool to manage the ports that are installed on the FreeBSD system which includes easy upgrading of ports.

cd /usr/ports/ports-mgmt/portmaster
make config-recursive
make install clean

Open-VM-Tools

Although VMware freebsd tools works, VMware is always slow in the release of the tools for FreeBSD.

cd /usr/ports/emulators/open-vm-tools-nox11/
make config-recursive
make install clean

Make sure that the Open-VM-Tools works on reboot by editing the /etc/rc.conf

vmware_guest_vmblock_enable="YES"
vmware_guest_vmhgfs_enable="YES"
vmware_guest_vmmemctl_enable="YES"
vmware_guest_vmxnet_enable="YES"
vmware_guestd_enable="YES"

Vim-Lite

We all need our favourite text editor to configure Nagios and other configuration files.

cd /usr/ports/editors/vim-lite
make config-recursive
make install clean

edit ~/.vimrc

syntax on
set background=dark
set shiftwidth=2
set tabstop=2
set nocompatible
set expandtab
set autoindent
set ruler
if has("autocmd")
 filetype plugin indent on
endif
set showcmd " Show (partial) command in status line.
set showmatch " Show matching brackets.
set ignorecase " Do case insensitive matching
set smartcase " Do smart case matching
set incsearch " Incremental search
set hidden " Hide buffers when they are abandoned
set backspace=indent,eol,start
set mouse=

SSMTP

Refer to this article to setup an authenticated system email with Gmail.

Web Server

Nagios has a web based dashboard to provide an overview of the state of the monitored machines.

Install Apache24
cd /usr/ports/www/apache24
make config-recursive
make install clean
vim /etc/rc.conf
apache24_enable=yes

Install PHP 5.6

cd /usr/ports/lang/php56
make config-recursive
make install clean

Install PHP56-extensions

cd /usr/ports/lang/php56-extensions
make config-recursive
make install clean

Install PHP56 Apache module

cd /usr/ports/www/mod_php56
make config-recursive
make install clean

Edit httpd.conf to insert the following to the configuration file

<FilesMatch ".php$">
    SetHandler application/x-httpd-php
</FilesMatch>
<FilesMatch ".phps$">
    SetHandler application/x-httpd-php-source
</FilesMatch>
cp /usr/local/etc/php.ini-production /usr/local/etc/php.ini
cd /usr/ports/net-mgmt/nagios4
make config-recursive
make install clean

Ensure that mod_cgi is enabled in /usr/local/etc/apache24/httpd.conf by uncommenting the following line

LoadModule cgi_module libexec/apache24/mod_cgi.so

Restart the apache service to check if nagios is accessible.

More about the installation and configuration of Nagios to follow. 🙂

My Server Died

Posted on by .

My trusty home server decided to finally give itself up after losing it’s operating system drive, the root mount about three month ago. Yes. My server was running off RAM for the past three months where I was really really busy with my school work – my Honour Year Project – to be exact. As such, I have lost my last blog post on the labeling of ZFS volumes in FreeNAS. But I’m very grateful, it managed to stay alive until yesterday, one day after I gave my last undergraduate academic presentation for my Honour Year Project.

This time round, I’ve decided to run this wordpress app on my edge hypervisor box with a guest OS of FreeBSD. Setting up a FreeBSD server takes time but it’s quite a rewarding experience, well, this might be due to seeing all the codes compile for the necessary services.

The Hard Drive that died was a 2004 Seagate 80GB SATA-I hard drive that was moved across multiple computers. I’ve already knew about the loss of the mount about 3 months ago, but Linux being Linux, it’s quite awesome where everything is still kept in RAM and able to run my wordpress and torrent box like nothing had went wrong. But it seems like time is up. I’ve received the death alert by my remote monitoring system yesterday afternoon.

The death was timely too, I have a upgrade plan to move my storage server to a complete FreeNAS system, using ZFS and hard disks in mirror mode with ZIL and L2ARC with SLC and MLC SSDs. This change in storage system will give me some storage performance boost that is capable of providing iSCSI for my hypervisors, and CIFS/NFS for my storage client machines over my home network.

Just yesterday, I was caught in a very interesting discussion of scalable storage/database reliability between two experienced techies of SOC. They were discussing about the move of storage towards object storage system such as Ceph, and leveraging on various techniques like load-balancing and distributed-hash-tables (DHT) to really grow and scale in an incredible way. A way that never crossed my mind for my Honour Year Project. Maybe after graduation and moving around my computing resources, I could experiment on that technique too.

In the meantime, I still have to tie some loose ends of my HYP and then my last two papers of my Undergraduate life. Yes. I’m finally graduating. 🙂

Using Regular Expressions to Reduce Exposure

Posted on by .

The Internet is filled with bots (zombie machines working to the master’s bid), spiders and etc. Recently, I saw a large number of attempts gaining access to my WordPress Login page (and only the wp-login.php, obviously attempts from some scripts). Okay, I believe I have good password and username hygiene which did deterred the bots but then why not prevent non-public addresses from accessing it?

Such configuration I believe can be easily done by .htaccess in Apache, but I’m using nginx and it’s slightly different so here’s my method. nginx allows the user to put rules into config so look into the your nginx configuration.

One way to do this is to do a regular expression matching for the remote IP address ($remote_addr) or if you are like me who puts the Web Server behind a HAProxy, look at the IP address forwarded for ($http_x_forwarded_for). You can generate your IP range using Google IP Address Range Rules (IP regular expression generator). And here is my additional configuration, this is to be inserted below the basic web root config.

location ~ /wp-login.php {
 if ($remote_addr !~ ^(192.168.([0-7]).([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5])))$ ) { 
  //Only allow my private addresses to have access
  return 403;
 } 

 if ($http_x_forwarded_for != "" ) { 
  //Any proxy-ed access will be denied too
  return 403;
 }

 root html;
 fastcgi_pass unix:/tmp/php-fpm.sock;
 fastcgi_index index.php;
 fastcgi_param SCRIPT_FILENAME /var/www/html/wordpress$fastcgi_script_name;
 include fastcgi_params;
}

Setting up a FreeBSD9.1 Server

Posted on by .

It’s been a while since I last posted something here.

Recently got interested in the “beastie” aka BSD one flavor of UNIX, not say I’m no longer loving the penguin but then it’s good to know more systems out there and FreeBSD is one of the most well know systems for stability and uptime of like 5 years without restarting. So here is how to get it up with Nginx, PHP and MySQL, running inside a VMware environment.


The first thing to do is to get portsnap to do a update from the port tree.

portsnap fetch extract update

Most of the popular applications will be in the ports tree so most of these installation can be automated without much issues.

Installing VMware Tools

Installing Perl5.16

/usr/ports/lang/perl5.16/
make config-recursive
make install clean

Install compat6x

/usr/ports/misc/compat6x/
make config-recursive
make install clean

Manual installation of 2 VMware modules

“Insert” the vmware tools disc, and mount using type cd9660 as such

mount -t cd9600 /dev/cd0 /mnt

then access the mounted disc and copy the tool file to somewhere locally for extraction.

<DIR>vmware-tools-distrib/lib/modules/source/

Untar the following, make and make install them before installing the main perl script

vmmemctl.tar
vmblock.tar

Installing Bash-completion

Firstly will be the installation of  “bash-completion” as I find “csh” not that friendly, plus tab-ing is the way to go… 🙂

cd /usr/ports/shells/bash-completion
make config-recursive

No additional selections is necessary (defaults are alright)

make install clean

Then you have to change the default shell of the user by using

chpass

and change the Shell to

/usr/local/bin/bash

Installing VIM

And how can we go about a Unix/Linux system without VIM? vi is just quite painful to use. And since I’m using this machine as a server, I don’t need gvim so vim-lite is suffice.

cd /usr/ports/editors/vim-lite
make config-recursive
make install clean

Do remember to configure your own ~/.vimrc

syntax on
set background=dark
set shiftwidth=2
set tabstop=2
set nocompatible
set expandtab
set autoindent
set ruler
if has("autocmd")
 filetype plugin indent on
endif
set showcmd " Show (partial) command in status line.
set showmatch " Show matching brackets.
set ignorecase " Do case insensitive matching
set smartcase " Do smart case matching
set incsearch " Incremental search
set hidden " Hide buffers when they are abandoned
set backspace=indent,eol,start
set mouse=

Installing wget

cd /usr/ports/ftp/wget
make config-recursive
make install clean

Installing PHP

cd /usr/ports/lang/php5
make config-recursive
make install clean

You will also have to install php extenstions for things like session, mbstring, mycrypt, mysql, mysqli and etc.

cd /usr/ports/lang/php5-extensions
make config-recursive
make install clean

Configuring PHP-FPM

vim /usr/local/etc/php-fpm.conf
events.mechanism = kqueue
listen = /var/run/php-fpm.sock

listen.owner = www
listen.group = www
listen.mode = 0666

You will also have to configure the php.ini for your needs, I need to set my local timezone

cp /usr/local/etc/php.ini-production /usr/local/etc/php.ini
vim /usr/local/etc/php.ini
date.timezone = Asia/Singapore

You will have to add the following line into the rc.conf.

vim /etc/rc.conf
php_fpm_enable="YES"

Installing nginx

cd /usr/ports/www/nginx
make config-recursive
make install clean

Configuring nginx

vim /usr/local/etc/nginx/nginx.conf

Some things that needs to be included are the use of kqueue which is used in BSD.

events {
 worker_connections 1024;
 use kqueue;
}
location ~ .php$ {
 #root html;
 #fastcgi_pass 127.0.0.1:9000;
 fastcgi_pass unix:/var/run/php-fpm.sock;
 #fastcgi_index index.php;
 fastcgi_param SCRIPT_FILENAME /usr/local/www$fastcgi_script_name;
 fastcgi_param PATH_INFO $fastcgi_script_name;
 include fastcgi_params;
}

You will have to add the following line into the rc.conf.

vim /etc/rc.conf
nginx_enable="YES"

Installing MySQL

Depending on requirements, the choice of MySQL configuration will be different, pick the most suitable one and copy it.

cp /usr/local/share/mysql/my-small.cnf /usr/local/etc/my.cnf

Manually start MySQL server to install and configure it.

/usr/local/etc/rc.d/mysql-server start

Follow the guide from the secure installation script.

mysql_secure_installation

You will have to add the following line into the rc.conf.

vim /etc/rc.conf
mysql_enable="YES"

Installing Web-apps

After completing the above installation, it should be a breeze to install the rest of the web-apps like phpMyAdmin and WordPress, just place them at the correct place /usr/local/www 🙂

Secure Emails using Pretty Good Privacy (PGP)

Posted on by .

Did you people know that most emails are actually processed in the clear? No no, I don’t mean you can really intercept an email halfway and read it easily, especially if you are using SSL/TLS or STARTTLS to talk to your email server, but it more of the mail servers can see your messages in the clear. This means that the ISP’s or e-Mail service providers can actually read all your messages upon receiving it from another SMTP server.

There are 3 points where people considers a need for data protection. Storage, Transmit and Process. Each of them requires different methods of protection. So in the case of eMails, we actually need to protect all 3 of them.

So how do we go about it? Encryption. I don’t want to bore readers with much technical details, but the encryption is just merely messing up the message so that it could not be easily read or understood without the proper way to decrypting it. What’s interesting is that there are actually 2 different encryption methods and both are equally good.

The 1st one is S/MIME, this one requires 2 certificates signed from a CA each, and an authorized CA will work Trusted out of the box.While a self-signed one will usually have some trust warning issues. This one is rather easy. Get your certificate, install them and configure your favorite eMail client to use it for signing (ensuring integrity)  or encryption (ensuring integrity and confidentiality). Do remember though, to use encryption, you need keys from both communicating parties (your own private key (usually the one u use to generate your own certificate) and the receiving party’s private key (usually his certificate itself)).

The other one is PGP or Pretty Good Privacy which I recently got myself interested in, as I haven’t really studied PGP nor worked on it before. Anyway, PGP doesn’t work in Outlook 2010. So I had to use my alternate eMail client Thunderbird. After the successful installation of Thunderbird, I added on enigmail which is the PGP add-on.

Following which depending whether you are on Linux or Windows, you need to generate your PGP keys. For windows you can download gpg4win as the PGP software, while most Linux OS have GnuPG built in. In windows, gpg4win has Kleopatra, a key manager. You could use Kleopatra to generate your PGP key , but I’m not sure why it’s limited to a 3K bit key.

As for linux, you can run

gpg --gen-key

and follow it’s instructions accordingly. 1 problem I faced is the problem with generating from terminal is the issue with entrophy. There is this issue with needing additional bytes, so you will really have to keep moving your mouse or run another process in the background to generate the key. This method allows you to generate a 4K bit key. For more information you can refer to the gnupg manual.

After generating the key, make sure Thunderbird can use the key that you had generated. After the successful installation of the addon – enigmail you should be able to see on the menu bar “OpenPGP”. If you are on windows, you have to open up OpenPGP’s preferences and pinpoint where is the gpg file, which in today’s case is actually gpg2.exe that is from the gpg4win that was installed earlier.

Then accordingly, you can select a relevant key to do the signing and encrypting of your messages so as to satisfy Confidentiality, Integrity, Authenticity and Non-repudiation.

Anyways, My PGP key 0xBF5CF327 is as follow, if you need to contact me privately. Alternatively you can get my key from keyserver.pgp.com, just search for my email [email protected].

Yap Yao Jun <[email protected]>
Key ID: 0xBF5CF327
Key fingerprint: 548C 6DC2 E09D CF39 F2EE E56C 83C8 B967 BF5C F327

Installing Centos 6 – LNMP

Posted on by .

I’ve decided to re-start blogging again! That is after 1 year of silence…

Well, I guess it’s time to start a Tech blog on what I do as a techie and to document what’s necessary, as a reference for myself and others if they chance into the problem, let me start first by getting LNMP to work on CentOS6. LNMP is otherwise know as Linux Nginx MySQL and PHP.

Installing CentOS6 is pretty straight forward. Pop the disc/USB in, follow the instructions and voilla, you have a working machine. There are some variations also like installing using the network method, but I won’t cover that here.

Nginx. Nginx is an awesome piece of web server. It’s lightweight and fast. Much lighter than the world famous Apache. Installing it in today’s context is rather easy. With the package already compiled for you.

Firstly, you have to add the Nginx repo into your repo list. And CentOS uses yum. So, you have to create a “nginx.repo” in “/etc/yum.repo./” with this following code

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/6/$basearch/
gpgcheck=0
enabled=1

After that, it’s just run

yum install nginx

And nginx will be installed. Next, Centos usually has iptables enabled, iptables is a firewall, so you’ll have to add in a rule to allow 80 (http) or 443 (https) into the Allow rules.

vim /etc/sysconfig/iptables
-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
service nginx start

Then you restart iptables and http(s) should work.

Then for nginx to load everytime the machine boots

chkconfig --level 345 nginx on

MySQL, the lovely database magic box. Installing a updated copy is not that straight forward as it seems. I have to add additional repos, in this case “remi”‘s repo, set of repo maintained by this french guy. But you also have to get “epel” repo also. So here are the commands:

rpm -Uvh http://mirror.nus.edu.sg/fedora/epel/6/i386/epel-release-6-5.noarch.rpm
rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-6.rpm

Then you install MySQL as such

yum --enablerepo=remi install mysql mysql-server

After installing the binaries, you have to install the database

mysql_install_db --user=mysql

After which start MySQL

mysqld_safe &

Change the default password using

mysql_secure_installation

verify that the change was correct by properly shutting down MySQL

mysqladmin shutdown -p

Similarly for load on reboot

chkconfig --levels 235 mysqld on

Note: unless necessary, remember to add additional firewall rule if you want your MySQL to be remotely accessible. There are also some configuration changes to me made to make that work which I will not cover here.


Now PHP. Pre-Hypertext Processor and my favourite language. We will still be using “remi” repo. So,

yum --enablerepo=remi install php php-fpm
yum --enablerepo=remi install php-gd php-mysql php-mbstring php-xml php-mcrypt php-pecl-apc php-pecl-memcache php-xmlrpc

After installation

service php-fpm start

To check whether PHP is working, you have to configure some things.

Firstly, edit nginx config file.

vim /etc/nginx/conf.d/default.conf
location / {
 root /usr/share/nginx/html;
 index index.html index.htm index.php;
 }

Uncomment the php-pfm codes and input the root of the web file directory.

location ~ .php$ {
 root html;
 fastcgi_pass 127.0.0.1:9000;
 fastcgi_index index.php;
 fastcgi_param SCRIPT_FILENAME /usr/share/nginx/html$fastcgi_script_name;
 include fastcgi_params;
 }

Reload nginx

service nginx reload

And create a test file at the document root

vim /usr/share/nginx/html/info.php

Append the following

<?php phpinfo(); ?>

Go to the website

http://serverNameOrIPAddressHere/info.php

And you should see information related to PHP.

A recommendation by me will be to move PHP-FPM from the TCP socket to the Unix socket and it can be done in such a manner.

Firstly edit

vim /etc/php-fpm.d/www.conf

comment/delete and change the following lines to

;listen = 127.0.0.1:9000
listen = /tmp/php-fpm.sock
;listen.allowed_clients = 127.0.0.1

Change your user and group to

user = nginx
group = nginx

You can verify the changes using

netstat -l -n

Now if you reload your info.php() it will not work, you have to go and adjust the nginx server configuration file. Comment away the old TCP socket and replace with the Unix socket.

vim /etc/nginx/conf.d/default.conf
location ~ .php$ {
 root html;
 #fastcgi_pass 127.0.0.1:9000;
 fastcgi_pass unix:/tmp/php-fpm.sock;
 fastcgi_index index.php;
 fastcgi_param SCRIPT_FILENAME /usr/share/nginx/html$fastcgi_script_name;
 include fastcgi_params;
 }

Restart nginx

service nginx restart

And reload your info.php test page to make sure that it’s running.

To make sure php-fpm runs every time the machine boots

chkconfig --levels 235 php-fpm on

An there as you have it, a Linux Nginx MySQL PHP machine is ready for you to use, be it for WordPress, OwnCloud, phpMyAdmin or other PHP based software.

Hope you had fun configuring it. 🙂