Thursday, 24 March 2016

Backporting Gearmand, libgearman on Centos 6 ( the latest version )

Lets get to work straight away:

wget http://dl.fedoraproject.org/pub/epel/7/SRPMS/g/gearmand-1.1.12-2.el7.src.rpm

yum install rpm-build yum-utils -y

yum groupinstall "Development Tools" -y

rpm --nomd5 -ivh gearmand-1.1.12-2.el7.src.rpm

cd ~/rpmbuild/SPECS

yum-builddep gearmand.spec

rpmbuild -bp gearmand.spec

rpmbuild -ba gearmand.spec

cd ~/rpmbuild/RPMS/x86_64

rpm -ivh *.rpm

Preparing...                ########################################### [100%]
    package libgearman-1.1.12-2.el6.x86_64 is installed
    package gearmand-1.1.12-2.el6.x86_64 is is installed
    package libgearman-devel-1.1.12-2.el6.x86_64 is installed
    package gearmand-debuginfo-1.1.12-2.el6.x86_64 is installed

Verify:

rpm -qa | grep gearman

libgearman-1.1.12-2.el6.x86_64
gearmand-1.1.12-2.el6.x86_64
gearmand-debuginfo-1.1.12-2.el6.x86_64
libgearman-devel-1.1.12-2.el6.x86_64

PECL Extension:

yum -y install php-pear php-devel

pecl channel-update pecl.php.net

pecl install gearman

echo extension=gearman.so >> /etc/php.ini

Verify:

php -i | grep gearman

gearman
gearman support => enabled
libgearman version => 1.1.12

pecl list

Installed packages, channel pecl.php.net:
=========================================
Package Version State
gearman 1.1.2   stable

Porting to another machine :

To install the four packages,first copy them to another machine like this:

scp /root/rpmbuild/RPMS/x86_64/* root@ip-address:/home

ssh root@ip-address

cd /home

yum localinstall *.rpm -y

yum groupinstall "Development Tools" -y

Same steps to be followed for PECL extension.

Sunday, 21 February 2016

Making the WebServer IPv6 Ready!

Gathering initial info:

This can be done by following the following steps:
As we are using Linode, the Linode server is by default IPv6 enabled, to check and confirm that punch in the command as follows:

# ip -6 addr show

It will come up with this result:

3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
      inet6 2001:DB8:2000:aff0::/32 scope global
        valid_lft forever preferred_lft forever
      inet6 ff32:20:2001:db8::/96 scope link
        valid_lft forever preferred_lft forever

The line highlighted in yellow confirms that our server is IPv6 enabled. (The scope global is our IPv6 address which is accessible from any other machine which is also IPv6 enabled and ready)

On the fly testing:

If we don’t get to see the IPv6 scope global line, we can always go to the Linode Remote Panel, note the IPv6 and IPv6GW and add them on the fly like this:

# ip -6 addr add public_ipv6_address/64 dev eth0

# ip -6 route add default via public_ipv6_gateway dev eth0

Making the IPv6 Configuration Persistent:

To make it IPv6 ready we need to just change two files:

1. Modify the /etc/sysconfig/network to look like this:

NETWORKING=yes
NETWORKING_IPV6=yes

2. Modify the /etc/syscofig/network-scripts/ifcfg-eth0 to look exactly like this: ( use your ip that you have noted by using the above statement )

DEVICE='eth0'
TYPE=Ethernet
BOOTPROTO=none
ONBOOT='yes'
HWADDR=04:01:ab:c4:1e:01
IPADDR=82.196.8.192
NETMASK=255.255.255.0
GATEWAY=82.196.8.1
NM_CONTROLLED='no'
IPV6INIT=yes
IPV6ADDR=2A03:B0C0:0000:1010:0000:0000:00A7:E001/64
IPV6_DEFAULTGW=2A03:B0C0:0000:1010:0000:0000:0000:0001
IPV6_AUTOCONF=no
DNS1=2001:4860:4860::8844
DNS2=2001:4860:4860::8888

Once this is all done, we need to disable the network manager, which can interfere in our network settings; this can be done as follows:

# sudo systemctl stop NetworkManager 

# sudo systemctl disable NetworkManager

Now enable the normal network service:

# sudo service network restart

Add Additional IP’s:

To add additional IP’s in linode one can request additional IPv6 addresses at any time by opening a support ticket.

To add it just modify the ifcfg-eth0 something like this:

IPV6INIT=yes
IPV6ADDR=primary_ipv6_address/64
IPV6_DEFAULTGW=ipv6_gateway
IPV6ADDR_SECONDARIES="second_ipv6_address/64 third_ipv6_address/64”
IPV6_AUTOCONF=no

To verify if our Server is now IPv6 ready and working, just issue the following command from the terminal:

# ping6 ipv6.google.com

If the reply is something like this:

PING ipv6.google.com(li-in-x8a.1e100.net) 56 data bytes
64 bytes from li-in-x8a.1e100.net: icmp_seq=1 ttl=42 time=150 ms
64 bytes from li-in-x8a.1e100.net: icmp_seq=2 ttl=42 time=150 ms
64 bytes from li-in-x8a.1e100.net: icmp_seq=3 ttl=42 time=150 ms
64 bytes from li-in-x8a.1e100.net: icmp_seq=4 ttl=42 time=150 ms

That means our Server is IPv6 enabled and ready and functional!

This is only the half of work done! Let’s go onward!

Making the Web Server IPv6 Ready:

To make the Web Server IPv6 ready i.e. serve our web content over IPv6, we need to follow the following steps:

Make a DNS entry:

We need to have the DNS record for IPv6 to resolve the IP. To add the record we put the IP into the AAAA record of the DNS entry.

Disabling the Centos 7 Firewall:

For our ease, we need to disable the default firewall, it is done as follows:

# systemctl mask firewalld
# systemctl stop firewalld

Enabling the Iptables service:

As we need to be serving our content on IPv6 as well as IPv4, therefore we will enable both the iptables service on our server, the steps are listed as follows:

# yum -y install iptables-services

# systemctl enable iptables
# systemctl enable ip6tables

Now, Finally let’s start the iptables services.

# systemctl start iptables
# systemctl start ip6tables

Open ports on Web Server:

In order to serve the content, we need to make our server start listening on port 80 & port 443 for Non-SSl and SSL connections respectively.

i. For IPv4:

# iptables -A INPUT -p tcp --dport 80 -j ACCEPT

# iptables -A INPUT -p tcp --dport 443 -j ACCEPT

ii. For IPv6

# ip6tables -A INPUT -p tcp --dport 80 -j ACCEPT

# ip6tables -A INPUT -p tcp --dport 443 -j ACCEPT

Verify the tables by: (make sure the correct rules are added)

# cat /etc/sysconfig/iptables (IPv4)

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [24:2624]
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT
COMMIT

# cat /etc/sysconfig/ip6tables (IPv6)

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT
COMMIT

Now save the rules:

# service iptables save

# service ip6tables save

Verify that the server is listening:

i. For IPv4:

# nmap 82.196.0.141

Starting Nmap 6.40 ( http://nmap.org ) at 2016-02-20 21:19 EST
Nmap scan report for 82.196.0.141
Host is up (0.000011s latency).
Not shown: 997 closed ports
PORT    STATE SERVICE
80/tcp  open  http
443/tcp open  https

Nmap done: 1 IP address (1 host up) scanned in 0.10 seconds

ii. For IPv6:

# nmap -6 2a03:b0c0:0:1010::102:4001

Starting Nmap 6.40 ( http://nmap.org ) at 2016-02-20 21:19 EST
Nmap scan report for centos-512mb-ams2-01 (2a03:b0c0:0:1010::102:4001)
Host is up (0.000026s latency).
Not shown: 997 closed ports
PORT    STATE SERVICE
80/tcp  open  http
443/tcp open  https

Nmap done: 1 IP address (1 host up) scanned in 0.12 seconds

Keep going!

Making Apache Ready:

There are only slight changes to make on Apache to make apache serve the content over IPv6 and IPv4 simultaneously, they are listed as follows:

Go to httpd.conf and find the line like this:

Listen 80

And modify it to something like this:

Listen *:80

Then move onto file ssl.conf, and find the line something like this:

Listen 443 https

And change it to:

Listen *:443 https

Then go the virtual host file and make sure the vhost header is like this for 80 and 443 respectively:

<VirtualHost *:80>

<VirtualHost *:443>

Now give the server a restart:

# systemctl restart httpd

That’s It! Job Done! Now onwards for verification.

Verifying our IPv6 service:

Although there are multiple ways of verifying that our content is being served correctly over IPv6 or not, I’m listing few of them:

The ping method:

From any other IPv6 enabled server to this:

# yum -y install bind-utils

# host -t AAAA www.nasheikh.com

The response would be something like this:

www.nasheikh.com is an alias for nasheikh.com.
nasheikh.com has IPv6 address 2a03:b0c0:0:1010::102:4001

Now ping it:

# ping6 nasheikh.com

The response:

PING nasheikh.com(centos-512mb-ams2-01) 56 data bytes
64 bytes from centos-512mb-ams2-01: icmp_seq=1 ttl=64 time=0.080 ms
64 bytes from centos-512mb-ams2-01: icmp_seq=2 ttl=64 time=0.130 ms
64 bytes from centos-512mb-ams2-01: icmp_seq=3 ttl=64 time=0.090 ms

Now verified that domain is properly getting resolved, there is one more method of verifying that whether the content is getting properly served or not. This listed method is more comprehensive way of IPv6 testing.

Open the link:

http://ipv6-test.com/validate.php

Enter the domain name and click on validate and it will take you the next page, and show you the result of IPv6 support. If all done correctly you will be shown the result like ipv6 ready!

Congratulation! Your website is now IPv6!! Yay!...... This marks the end of tutorial as well!

Wednesday, 20 January 2016

Pooling JSON data into MySql Table sequentially!

We have JSON data :

[{
"timeMillis": 1452502968583,
"latitude": -33.9041,
"longitude": 170.7297,
"current": 15.7,
"direction": "GROUND"
}, {
"timeMillis": 1452506410228,
"latitude": -33.6113,
"longitude": 172.1532,
"current": 8.1,
"direction": "GROUND"
}, {
"timeMillis": 1452513792306,
"latitude": -48.9861,
"longitude": 169.7887,
"current": -38.5,
"direction": "GROUND"
}]

save it as file.json

Now use this :

<?php
    $con = mysqli_connect("localhost","root","","lightning");

    // Check connection
    if (mysqli_connect_errno())
      {
      echo "Failed to connect to MySQL: " . mysqli_connect_error();
      }

    $jdata = file_get_contents("/home/file.json");
    
    $data = json_decode($jdata, true);
    foreach($data as $mydata)
    {
        $sql = "INSERT INTO data(timeMillis, latitude, longitude, current) VALUES (".$mydata['timeMillis'].",".$mydata['latitude'].",".$mydata['longitude'].",".$mydata['current'].")";   
        if(!mysqli_query($con, $sql))
        {
            die('Error : ' . mysql_error());
        }
    }

    mysqli_close($con);
    //echo "Le bhai, ho gaya tera data store!"         
?>

Job Done!

KickStarting and Linux WSUS Techinque

Make a Webserver:

# yum -y install httpd

Create a folder 

# mkdir  -p /var/www/html/install

Then mount the cd

# mount /dev/cdrom /media

Copy the data

# cp -R /media/* /var/www/html/install

Verify it 

# diff -q -r /media /var/www/html/install

There are may ways to create a kickstart file for example use a gui tool :

# yum install system-config-kickstart

Or use one like this:

#Kickstart file automatically generated by anaconda.

#version=DEVEL
install                                 # instead of "upgrade"
text            # you're not going to be standing there watching it, are you?
url --url http://10.10.10.1/pub/        # use this instead of CD-ROM Option
bootloader --location=mbr --driveorder=sda --append="crashkernel=auto rhgb quiet"
zerombr yes
clearpart --all --initlabel
part / --fstype ext4 --size 4096 --grow
part swap --recommended
selinux --disabled
lang en_US.UTF-8
keyboard us
network --onboot yes --device eth0 --bootproto dhcp --noipv6
rootpw  --iscrypted $6$Y3aFxc5JNbESX3hb$
firewall --service=ssh
authconfig --enableshadow --passalgo=sha512
timezone --utc Pacific/Auckland
repo --name="CentOS"  --baseurl=http://10.10.10.1/pub/ --cost=100    #replace the base url from #cdrom or any other to suit yours
reboot
%packages --nobase
@core
%post 
cd /tmp 
wget http://10.10.10.1/post-install/myrepo.sh 
chmod +x local-repo.sh 
./repo.sh
%end

Save this file as ks.cfg into /var/www/html/install

Now we create our own repo:

# mkdir -p /var/www/html/reposit

# yum install createrepo

# createrepo   /var/www/html/reposit

# rsync -avz rsync://centos.ar.host-engine.com/6.6/os/x86_64/ /var/www/html/reposit

# createrepo --update /var/www/html/reposit

Once this is all done, we make a script 

# mkdir -p /var/www/html/post-install

# cd /var/www/html/post-install

# nano myrepo.sh

cd /etc/yum.repos.d
rm *.repo  
wget http://10.10.10.1/post-install/myrepo.repo
yum clean all
yum -y update

Save it and create a local repo, also in the same folder like this:

# nano myrepo.repo

[myrepo]
name=myrepo
baseurl=http://10.10.10.1/reposit/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-6

Save it.

Now boot up a client and when the installer screen comes in press tab and enter this all in one line:

vmlinuz initrd=initrd.img ks=http://10.10.10.1/install/ks.cfg append ip=10.10.10.100 netmask=255.255.255.0 ksdevice=eth0

Job Done!

Monday, 26 October 2015

Getting a DESKTOP on centos!

Do a minimal install of centos 6.5 and start doing this:

# yum -y groupinstall "Desktop" "Desktop Platform" "X Window System" "Fonts" "General Purpose Desktop"

# init 5

To have CentOS boot into runlevel 5 “X11” instead of runlevel 3 “Full multiuser mode”, modify the /etc/inittab file to change start up level like this :

# nano /etc/inittab

id:3:initdefault:

to

id:5:initdefault:

Job Done!

Friday, 2 October 2015

AWS Auto Scaling - Complete CLI Solution

#! /bin/bash

function ASG {
read -e -p "Enter Min no. of instances(Integer Value)...: " min
echo ""
read -e -p "Enter Max no. of instances(Integer Value)...: " max
echo ""
read -e -p "Enter Desired no. of instances(Integer Value)...: " desired
echo ""
aws autoscaling create-auto-scaling-group --auto-scaling-group-name $Cluster-$COMPONENT-ASG --launch-configuration-name RTB-$COMPONENT-ASG --min-size $min --max-size $max --desired-capacity $desired --availability-zones us-east-1e --health-check-type ELB --health-check-grace-period 240 --default-cooldown 300 --load-balancer-names "RTB" --vpc-zone-identifier subnet-fea1f0d5 ; check;
}

function config {
echo ""
aws autoscaling create-launch-configuration --launch-configuration-name $Cluster-$COMPONENT-ASG --image-id ami-b7f68bd2 --instance-type $COMPONENT --spot-price "0.27" --no-associate-public-ip-address  --security-groups sg-665df302 --user-data $File --key-name rtbservers --instance-monitoring Enabled=false --no-ebs-optimized --block-device-mappings "[{\"DeviceName\":\"/dev/xvda\",\"Ebs\":{\"SnapshotId\":\"snap-68dc2c09\",\"VolumeSize\":10,\"VolumeType\": \"gp2\",\"DeleteOnTermination\": true}},{\"DeviceName\": \"/dev/sdb\",\"VirtualName\":\"ephemeral0\"}]" ; check && ASG;
}

function check {
if [ $? -eq 0 ]
then
echo "Command Executed Successfully"
echo ""
else
echo "Error, Please try again!"
echo ""
exit 1
fi
}

function type {
echo ""
echo "Please select the desired Instance type"
echo ""

select COMPONENT in c3.2xlarge c3.4xlarge m3.2xlarge r3.2xlarge r3.4xlarge g2.2xlarge
do
case $COMPONENT in

c3.2xlarge) config;
            break;;
c3.4xlarge) config;
            break;;
m3.2xlarge) config;
            break;;
r3.2xlarge) config;
            break;;
r3.4xlarge) config;
            break;;
g2.2xlarge) config;
            break;;
esac
done
}

echo ""
echo "Please select the desired Cluster type"
echo ""

select Cluster in RTB RTB3 RTB4
do
case $Cluster in
RTB) File="file:///operations/users/naveed/bootscriptrtb.txt"
     type;
     break;;
RTB3) File="file:///operations/users/naveed/bootscriptrtb3.txt"
      type;
      break;;
RTB4) File="file:///operations/users/naveed/bootscriptrtb4.txt"
      type;
      break;;
esac
done

echo "Please configure AS Policy"
echo ""
read -e -p "Enter No. of Instance to be affected (Integer Value)...: " aff
echo ""

aws autoscaling put-scaling-policy --policy-name increase-policy --auto-scaling-group-name $Cluster-$COMPONENT-ASG --scaling-adjustment $aff --adjustment-type ChangeInCapacity --output text > increase.txt
inc=$(cat increase.txt)
aws autoscaling put-scaling-policy --policy-name decrease-policy --auto-scaling-group-name $Cluster-$COMPONENT-ASG --scaling-adjustment -$aff --adjustment-type ChangeInCapacity --output text > decrease.txt
dec=$(cat decrease.txt)

echo "Please configure AS Alarm"
echo ""
read -e -p "Enter High CPU threshold (Integer Value)...: " high
echo ""
read -e -p "Enter Lower CPU threshold (Integer Value)...: " low
echo ""

aws cloudwatch put-metric-alarm --alarm-name $Cluster-$COMPONENT-AddCapacity --metric-name $Cluster-$COMPONENT-CPU-HIGH --namespace AWS/EC2 --statistic Average --period 300 --threshold $high --comparison-operator GreaterThanOrEqualToThreshold --dimensions "Name=AutoScalingGroupName,Value=$Cluster-$COMPONENT-ASG" --evaluation-periods 2 --alarm-actions $inc
aws cloudwatch put-metric-alarm --alarm-name $Cluster-$COMPONENT-RemoveCapacity --metric-name $Cluster-$COMPONENT-CPU-LOW --namespace AWS/EC2 --statistic Average --period 300 --threshold $low --comparison-operator LessThanOrEqualToThreshold --dimensions "Name=AutoScalingGroupName,Value=$Cluster-$COMPONENT-ASG" --evaluation-periods 2 --alarm-actions $dec

rm -fr increase.txt decrease.txt

echo "Job Finished!"
echo ""

Friday, 25 September 2015

Fixing the GPG missing keys error

Sometimes we face the missing gpg key issue when we add diffrent sources to sources.list files, well this can be fixed easily:
Im gonna show an example where im using ubuntu 14 and try to first add sources of debian jessie 8.

first go to http://debgen.simplylinux.ch/

get your source list, for eg:

deb http://ftp.us.debian.org/debian testing main contrib non-free
deb-src http://ftp.us.debian.org/debian testing main contrib non-free

deb http://security.debian.org/ jessie/updates main contrib non-free
deb-src http://security.debian.org/ jessie/updates main contrib non-free


paste this in

nano /etc/apt/sources.list

then run this script:

#!/bin/bash

apt-get update 2> /tmp/keymissing; for key in $(grep "NO_PUBKEY" /tmp/keymissing |sed "s/.*NO_PUBKEY //"); do echo -e "\nProcessing key: $key"; gpg --keyserver pgpkeys.mit.edu --recv $key && gpg --export --armor $key | apt-key add -; done
apt-get update 2> /tmp/keymissing; for key in $(grep "NO_PUBKEY" /tmp/keymissing |sed "s/.*NO_PUBKEY //"); do echo -e "\nProcessing key: $key"; gpg --keyserver subkeys.pgp.net --recv $key && gpg --export --armor $key | apt-key add -; done

# it works if the command finished and says things like:
#  gpg: requesting key 46925553 from hkp server pgpkeys.mit.edu
#  gpg: key 46925553: "Debian Archive Automatic Signing Key (7.0/wheezy)
#  <ftpmaster@debian.org>" not changed
#  gpg: Total number processed: 1
#  gpg:              unchanged: 1

# If doesnt work and stuck on
# gpg: requesting key 46925553 from hkp server pgpkeys.mit.edu

# Then copy key from this line
# "Processing key: 8B48AD6246925553"
# and google search "8B48AD6246925553" this will return keyserver, like below, put those in for --keyserver line

# And look for keyserver

# Works for debian: pgpkeys.mit.edu
# Works for ubuntu: subkeys.pgp.net


This will work for both, ubuntu and debian - vice versa.

Special thanks to :  http://ram.kossboss.com/fix-missing-gpg-key-apt-get/