Different Perspective!: 2016

Monday 23 May 2016

HHVM or PHP-FPM on Apache 2.4 ( Centos 7 )

Now i have been going through a lot of blogs lately for this configuration but only found conf. with nginx, none with apache, so i decided to come up with this tutorial!

Lot of architecture are still using apache due to historic reasons one of them being .htaccess file!

Dont want to touch your apache, or whatever reason you want to stick with it, with this you can update your server performance using this tutorial!

On with installations then:

get lamp stack up and running:

yum install httpd
systemctl start httpd.service
systemctl enable httpd.service

yum install mariadb-server mariadb
systemctl start mariadb
mysql_secure_installation

systemctl enable mariadb.service

yum install php php-mysql php-mbstring php-pear

nano /var/www/html/info.php

<?php phpinfo(); ?>

nano /etc/php.ini

# line 878: uncomment and add your timezone

date.timezone = "Asia/Tokyo"

nano /var/www/html/index.php

<html>

<body>

<div style="width: 100%; font-size: 40px; font-weight: bold; text-align: center;">

<?php

print Date("Y/m/d");

</div>

</body>

</html>

systemctl restart httpd.service

Open Firewall if using firewalld or iptables adjust accordingly:

sudo firewall-cmd --permanent --zone=public --add-service=http

sudo firewall-cmd --permanent --zone=public --add-service=https

sudo firewall-cmd --reload

Now we fix out PHP-FPM:

yum -y install php-fpm

Normally the mod_proxy_fcgi module comes by default installed and enabled on apache 2.4 which can be verified by the following commands:

ls -l /usr/lib64/httpd/modules/mod_proxy*

cat /etc/httpd/conf.modules.d/00-proxy.conf

httpd -M | grep -i proxy

Now this as verified we need to do this to the vhost conf file: ( just insert the red line )

<VirtualHost *:80>

ServerName localhost

ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:9000/var/www/html/$1

DocumentRoot /var/www/html

<Directory /var/www/html>

AllowOverride All

Require all granted

</Directory>

ErrorLog /var/log/httpd/localhost_error.log

CustomLog /var/log/httpd/localhost_access.log combined

</VirtualHost>

systemctl start php-fpm

systemctl enable php-fpm

systemctl restart httpd

This will make the php-fpm on! not getting into fine tuning php-fpm at the moment!

verify if php-fpm is listening on port 9000 by :

netstat -tulpn | grep 9000

Now on with HHVM :

yum -y install epel-release

yum install cpp gcc-c++ cmake git psmisc {binutils,boost,jemalloc,numactl}-devel \

{ImageMagick,sqlite,tbb,bzip2,openldap,readline,elfutils-libelf,gmp,lz4,pcre}-devel \

lib{xslt,event,yaml,vpx,png,zip,icu,mcrypt,memcached,cap,dwarf}-devel \

{unixODBC,expat,mariadb}-devel lib{edit,curl,xml2,xslt}-devel \

glog-devel oniguruma-devel ocaml gperf enca libjpeg-turbo-devel openssl-devel \

mariadb mariadb-server make -y

rpm -Uvh http://mirrors.linuxeye.com/hhvm-repo/7/x86_64/hhvm-3.12.0-1.el7.centos.x86_64.rpm

Check using this is successfully installed:

hhvm --version

Now do this stuff:

nano /usr/lib/systemd/system/hhvm.service

[Unit]

Description=HHVM HipHop Virtual Machine (FCGI)

[Service]

ExecStart=/usr/local/bin/hhvm

--config /etc/hhvm/server.ini \

--config /etc/hhvm/php.ini \

--config /etc/hhvm/config.hdf

--user apache

--mode daemon

[Install]

WantedBy=multi-user.target

mkdir /var/run/hhvm

chown apache:apache /var/run/hhvm

mkdir /var/log/hhvm

chown apache:apache /var/log/hhvm

Create config.hdf in /etc/hhvm:

ResourceLimit {

CoreFileSize = 0 # in bytes

MaxSocket = 10000 # must be not 0, otherwise HHVM will not start

SocketDefaultTimeout = 5 # in seconds

MaxRSS = 0

MaxRSSPollingCycle = 0 # in seconds, how often to check max memory

DropCacheCycle = 0 # in seconds, how often to drop disk cache

}

Log {

Level = Info

AlwaysLogUnhandledExceptions = true

RuntimeErrorReportingLevel = 8191

UseLogFile = true

UseSyslog = false

File = /var/log/hhvm/error.log

Access {

* {

File = /var/log/hhvm/access.log

Format = %h %l %u % t \"%r\" %>s %b

}

MySQL {

ReadOnly = false

ConnectTimeout = 1000 # in ms

ReadTimeout = 1000 # in ms

SlowQueryThreshold = 1000 # in ms, log slow queries as errors

KillOnTimeout = false

}

Mail {

SendmailPath = /usr/sbin/sendmail -t -i

ForceExtraParameters =

}

Create server.ini in /etc/hhvm:

; php options

pid = /var/run/hhvm/pid

; hhvm specific

hhvm.server.port = 8000

;hhvm.server.file_socket = /var/run/hhvm/sock

hhvm.server.type = fastcgi

hhvm.server.default_document = index.php

hhvm.log.use_log_file = true

hhvm.log.file = /var/log/hhvm/error.log

hhvm.repo.central.path = /var/run/hhvm/hhvm.hhbc

Create php.ini in /etc/hhvm:

hhvm.mysql.socket = /tmp/mysql.sock

hhvm.server.expose_hphp = true

memory_limit = 400M

post_max_size = 50M

systemctl enable hhvm

systemctl start hhvm

systemctl status hhvm

Verify if the HHVM is listening on port 8000 by :

netstat -tulpn | grep 8000

Once done we need to edit the vhost file like this:

ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:8000/var/www/html/$1

systemctl restart httpd

Thats it!

You can verify this by opening info.php page, if HHVM has served it, it will show you!

Which ever compiler you go with is completely your choice!

This marks the end of tutorial!

Comments are welcome!

Tuesday 12 April 2016

Checking email Remotely thru python ( Complete Solution )

This is done in three scripts one in bash rest other 2 in python:

Bash script for cron job and calling the entire procedure: (where the credits.txt is user1 pass1 sequence, all arranged like this)

user1 pass1
user2 pass2
.......

Bash Script:

#!/bin/bash

> /home/admin/mailstatus.txt

count=0

while IFS=' ' read -r f1 f2
do
a=($f1 $f2)
for ((i=0; i<${#a[@]}; i+=2)); do
./working1.py ${a[i]} ${a[i+1]}
done
let count=$count+1
done < /home/admin/credits.txt

minimumsize=$count

actualsize=$(wc -c < /home/admin/mailstatus.txt)
if [ $actualsize -gt $minimumsize ]; then
value=$(cat /home/admin/credits.txt | awk 'NR==3{print $2}')
./sendnew.py $value
else
echo "This script ran correctly at -- > $(date)" >> /home/admin/logfile
fi

Check the box:

#!/usr/bin/python

import sys
import poplib, re

if len(sys.argv) < 3:
sys.exit(-1)

class Logger(object):
def __init__(self):
self.terminal = sys.stdout
self.log = open("/home/admin/mailstatus.txt", "a")
def write(self, message):
self.log.write(message)

sys.stdout = Logger()

# Change this to your needs
POPHOST = "pop.gmail.com"
POPUSER = sys.argv[1]
POPPASS = sys.argv[2]

# How many lines of message body to retrieve
MAXLINES = 10

# Headers we're actually interrested in
rx_headers = re.compile(r"^(From|Subject)")

try:

# Connect to the POPer and identify user
pop = poplib.POP3_SSL(POPHOST)
pop.user(POPUSER)

# Authenticate user
pop.pass_(POPPASS)

# Get some general informations (msg_count, box_size)
stat = pop.stat()

output = stat[0]
check = stat[1]/1000000

if output > 0 and check >15:

#print "" * 30
print "" * 30
print "-" * 30

# Print some useless information
print "Checked %s for oversized emails" % (POPUSER)
print "-" * 30
print ("There are total: %d message(s), which is consuming %d MB's" % (stat[0], check))
print "-" * 30

for n in range(stat[0]):

msgnum = n+1

# Retrieve headers
response, lines, bytes = pop.top(msgnum, MAXLINES)

# Print message info and headers we're interrested in
print " || ".join(filter(rx_headers.match, lines))
print "-" * 30

# Commit operations and disconnect from server
pop.quit()
print "" * 30

Getting the report:

#!/usr/bin/python

import smtplib, os, sys
from email.MIMEMultipart import MIMEMultipart
from email.MIMEText import MIMEText

if len(sys.argv) < 2:
sys.exit(-1)

fromaddr = 'admin@nasheikh.com'
toaddrs = ['naveed@nasheikh.com', 'tamy@nasheikh.com']

# Credentials (if needed)
username = 'admin@nasheikh.com'
password = sys.argv[1]

raport_file = open('/home/admin/mailstatus.txt','rb')
alert_msg = raport_file.read()
raport_file.close()

msg = MIMEMultipart()
msg['From'] = fromaddr
msg['To'] = ", ".join(toaddrs)
msg['Subject'] = "Boxes Mail Status"

body = alert_msg
msg.attach(MIMEText(body, 'plain'))

text = msg.as_string()

# The actual mail send
server = smtplib.SMTP_SSL('smtp.gmail.com:465')
server.login(username,password)
server.sendmail(fromaddr, toaddrs, text)

server.quit()

This marks the end of tutorial !

Thursday 24 March 2016

Backporting Gearmand, libgearman on Centos 6 ( the latest version )

Lets get to work straight away:

wget http://dl.fedoraproject.org/pub/epel/7/SRPMS/g/gearmand-1.1.12-2.el7.src.rpm

yum install rpm-build yum-utils -y

yum groupinstall "Development Tools" -y

rpm --nomd5 -ivh gearmand-1.1.12-2.el7.src.rpm

cd ~/rpmbuild/SPECS

yum-builddep gearmand.spec

rpmbuild -bp gearmand.spec

rpmbuild -ba gearmand.spec

cd ~/rpmbuild/RPMS/x86_64

rpm -ivh *.rpm

Preparing...                ########################################### [100%]
    package libgearman-1.1.12-2.el6.x86_64 is installed
    package gearmand-1.1.12-2.el6.x86_64 is is installed
    package libgearman-devel-1.1.12-2.el6.x86_64 is installed
    package gearmand-debuginfo-1.1.12-2.el6.x86_64 is installed

Verify:

rpm -qa | grep gearman

libgearman-1.1.12-2.el6.x86_64
gearmand-1.1.12-2.el6.x86_64
gearmand-debuginfo-1.1.12-2.el6.x86_64
libgearman-devel-1.1.12-2.el6.x86_64

PECL Extension:

yum -y install php-pear php-devel

pecl channel-update pecl.php.net

pecl install gearman

echo extension=gearman.so >> /etc/php.ini

Verify:

php -i | grep gearman

gearman
gearman support => enabled
libgearman version => 1.1.12

pecl list

Installed packages, channel pecl.php.net:
=========================================
Package Version State
gearman 1.1.2   stable

Porting to another machine :

To install the four packages,first copy them to another machine like this:

scp /root/rpmbuild/RPMS/x86_64/* root@ip-address:/home

ssh root@ip-address

cd /home

yum localinstall *.rpm -y

yum groupinstall "Development Tools" -y

Same steps to be followed for PECL extension.

Sunday 21 February 2016

Making the WebServer IPv6 Ready!

 Gathering initial info:

This can be done by following the following steps:

As we are using Linode, the Linode server is by default IPv6 enabled, to check and confirm that punch in the command as follows:

# ip -6 addr show

It will come up with this result:

3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000

inet6 2001:DB8:2000:aff0::/32 scope global

valid_lft forever preferred_lft forever

inet6 ff32:20:2001:db8::/96 scope link

valid_lft forever preferred_lft forever

The line highlighted in yellow confirms that our server is IPv6 enabled. (The scope global is our IPv6 address which is accessible from any other machine which is also IPv6 enabled and ready)

 On the fly testing:

If we don’t get to see the IPv6 scope global line, we can always go to the Linode Remote Panel, note the IPv6 and IPv6GW and add them on the fly like this:

# ip -6 addr add public_ipv6_address/64 dev eth0

# ip -6 route add default via public_ipv6_gateway dev eth0

 Making the IPv6 Configuration Persistent:

To make it IPv6 ready we need to just change two files:

1. Modify the /etc/sysconfig/network to look like this:

NETWORKING=yes

NETWORKING_IPV6=yes

2. Modify the /etc/syscofig/network-scripts/ifcfg-eth0 to look exactly like this: ( use your ip that you have noted by using the above statement )

DEVICE='eth0'

TYPE=Ethernet

BOOTPROTO=none

ONBOOT='yes'

HWADDR=04:01:ab:c4:1e:01

IPADDR=82.196.8.192

NETMASK=255.255.255.0

GATEWAY=82.196.8.1

NM_CONTROLLED='no'

IPV6INIT=yes

IPV6ADDR=2A03:B0C0:0000:1010:0000:0000:00A7:E001/64

IPV6_DEFAULTGW=2A03:B0C0:0000:1010:0000:0000:0000:0001

IPV6_AUTOCONF=no

DNS1=2001:4860:4860::8844

DNS2=2001:4860:4860::8888

Once this is all done, we need to disable the network manager, which can interfere in our network settings; this can be done as follows:

# sudo systemctl stop NetworkManager

# sudo systemctl disable NetworkManager

Now enable the normal network service:

# sudo service network restart

 Add Additional IP’s:

To add additional IP’s in linode one can request additional IPv6 addresses at any time by opening a support ticket.

To add it just modify the ifcfg-eth0 something like this:

IPV6INIT=yes

IPV6ADDR=primary_ipv6_address/64

IPV6_DEFAULTGW=ipv6_gateway

IPV6ADDR_SECONDARIES="second_ipv6_address/64 third_ipv6_address/64”

IPV6_AUTOCONF=no

To verify if our Server is now IPv6 ready and working, just issue the following command from the terminal:

# ping6 ipv6.google.com

If the reply is something like this:

PING ipv6.google.com(li-in-x8a.1e100.net) 56 data bytes

64 bytes from li-in-x8a.1e100.net: icmp_seq=1 ttl=42 time=150 ms

64 bytes from li-in-x8a.1e100.net: icmp_seq=2 ttl=42 time=150 ms

64 bytes from li-in-x8a.1e100.net: icmp_seq=3 ttl=42 time=150 ms

64 bytes from li-in-x8a.1e100.net: icmp_seq=4 ttl=42 time=150 ms

That means our Server is IPv6 enabled and ready and functional!

This is only the half of work done! Let’s go onward!

 Making the Web Server IPv6 Ready:

To make the Web Server IPv6 ready i.e. serve our web content over IPv6, we need to follow the following steps:

 Make a DNS entry:

We need to have the DNS record for IPv6 to resolve the IP. To add the record we put the IP into the AAAA record of the DNS entry.

 Disabling the Centos 7 Firewall:

For our ease, we need to disable the default firewall, it is done as follows:

# systemctl mask firewalld

# systemctl stop firewalld

 Enabling the Iptables service:

As we need to be serving our content on IPv6 as well as IPv4, therefore we will enable both the iptables service on our server, the steps are listed as follows:

# yum -y install iptables-services

# systemctl enable iptables

# systemctl enable ip6tables

Now, Finally let’s start the iptables services.

# systemctl start iptables

# systemctl start ip6tables

 Open ports on Web Server:

In order to serve the content, we need to make our server start listening on port 80 & port 443 for Non-SSl and SSL connections respectively.

i. For IPv4:

# iptables -A INPUT -p tcp --dport 80 -j ACCEPT

# iptables -A INPUT -p tcp --dport 443 -j ACCEPT

ii. For IPv6

# ip6tables -A INPUT -p tcp --dport 80 -j ACCEPT

# ip6tables -A INPUT -p tcp --dport 443 -j ACCEPT

Verify the tables by: (make sure the correct rules are added)

# cat /etc/sysconfig/iptables (IPv4)

*filter

:INPUT ACCEPT [0:0]

:FORWARD ACCEPT [0:0]

:OUTPUT ACCEPT [24:2624]

-A INPUT -p tcp --dport 80 -j ACCEPT

-A INPUT -p tcp --dport 443 -j ACCEPT

COMMIT

# cat /etc/sysconfig/ip6tables (IPv6)

*filter

:INPUT ACCEPT [0:0]

:FORWARD ACCEPT [0:0]

:OUTPUT ACCEPT [0:0]

-A INPUT -p tcp --dport 80 -j ACCEPT

-A INPUT -p tcp --dport 443 -j ACCEPT

COMMIT

Now save the rules:

# service iptables save

# service ip6tables save

 Verify that the server is listening:

i. For IPv4:

# nmap 82.196.0.141

Starting Nmap 6.40 ( http://nmap.org ) at 2016-02-20 21:19 EST

Nmap scan report for 82.196.0.141

Host is up (0.000011s latency).

Not shown: 997 closed ports

PORT STATE SERVICE

80/tcp open http

443/tcp open https

Nmap done: 1 IP address (1 host up) scanned in 0.10 seconds

ii. For IPv6:

# nmap -6 2a03:b0c0:0:1010::102:4001

Starting Nmap 6.40 ( http://nmap.org ) at 2016-02-20 21:19 EST

Nmap scan report for centos-512mb-ams2-01 (2a03:b0c0:0:1010::102:4001)

Host is up (0.000026s latency).

Not shown: 997 closed ports

PORT STATE SERVICE

80/tcp open http

443/tcp open https

Nmap done: 1 IP address (1 host up) scanned in 0.12 seconds

Keep going!

 Making Apache Ready:

There are only slight changes to make on Apache to make apache serve the content over IPv6 and IPv4 simultaneously, they are listed as follows:

Go to httpd.conf and find the line like this:

Listen 80

And modify it to something like this:

Listen *:80

Then move onto file ssl.conf, and find the line something like this:

Listen 443 https

And change it to:

Listen *:443 https

Then go the virtual host file and make sure the vhost header is like this for 80 and 443 respectively:

<VirtualHost *:80>

<VirtualHost *:443>

Now give the server a restart:

# systemctl restart httpd

That’s It! Job Done! Now onwards for verification.

 Verifying our IPv6 service:

Although there are multiple ways of verifying that our content is being served correctly over IPv6 or not, I’m listing few of them:

The ping method:

From any other IPv6 enabled server to this:

# yum -y install bind-utils

# host -t AAAA www.nasheikh.com

The response would be something like this:

www.nasheikh.com is an alias for nasheikh.com.

nasheikh.com has IPv6 address 2a03:b0c0:0:1010::102:4001

Now ping it:

# ping6 nasheikh.com

The response:

PING nasheikh.com(centos-512mb-ams2-01) 56 data bytes

64 bytes from centos-512mb-ams2-01: icmp_seq=1 ttl=64 time=0.080 ms

64 bytes from centos-512mb-ams2-01: icmp_seq=2 ttl=64 time=0.130 ms

64 bytes from centos-512mb-ams2-01: icmp_seq=3 ttl=64 time=0.090 ms

Now verified that domain is properly getting resolved, there is one more method of verifying that whether the content is getting properly served or not. This listed method is more comprehensive way of IPv6 testing.

Open the link:

http://ipv6-test.com/validate.php

Enter the domain name and click on validate and it will take you the next page, and show you the result of IPv6 support. If all done correctly you will be shown the result like ipv6 ready!

Congratulation! Your website is now IPv6!! Yay!...... This marks the end of tutorial as well!

Wednesday 20 January 2016

Pooling JSON data into MySql Table sequentially!

We have JSON data :

[{
"timeMillis": 1452502968583,
"latitude": -33.9041,
"longitude": 170.7297,
"current": 15.7,
"direction": "GROUND"
}, {
"timeMillis": 1452506410228,
"latitude": -33.6113,
"longitude": 172.1532,
"current": 8.1,
"direction": "GROUND"
}, {
"timeMillis": 1452513792306,
"latitude": -48.9861,
"longitude": 169.7887,
"current": -38.5,
"direction": "GROUND"
}]

save it as file.json

Now use this :

<?php
$con = mysqli_connect("localhost","root","","lightning");

// Check connection
if (mysqli_connect_errno())
{
echo "Failed to connect to MySQL: " . mysqli_connect_error();
}

$jdata = file_get_contents("/home/file.json");

$data = json_decode($jdata, true);
foreach($data as $mydata)
{
$sql = "INSERT INTO data(timeMillis, latitude, longitude, current) VALUES (".$mydata['timeMillis'].",".$mydata['latitude'].",".$mydata['longitude'].",".$mydata['current'].")";
if(!mysqli_query($con, $sql))
{
die('Error : ' . mysql_error());
}
}

mysqli_close($con);
//echo "Le bhai, ho gaya tera data store!"
?>

Job Done!

KickStarting and Linux WSUS Techinque

Make a Webserver:

# yum -y install httpd

Create a folder

# mkdir -p /var/www/html/install

Then mount the cd

# mount /dev/cdrom /media

Copy the data

# cp -R /media/* /var/www/html/install

Verify it

# diff -q -r /media /var/www/html/install

There are may ways to create a kickstart file for example use a gui tool :

# yum install system-config-kickstart

Or use one like this:

#Kickstart file automatically generated by anaconda.

#version=DEVEL

install # instead of "upgrade"

text # you're not going to be standing there watching it, are you?

url --url http://10.10.10.1/pub/ # use this instead of CD-ROM Option

bootloader --location=mbr --driveorder=sda --append="crashkernel=auto rhgb quiet"

zerombr yes

clearpart --all --initlabel

part / --fstype ext4 --size 4096 --grow

part swap --recommended

selinux --disabled

lang en_US.UTF-8

keyboard us

network --onboot yes --device eth0 --bootproto dhcp --noipv6

rootpw --iscrypted $6$Y3aFxc5JNbESX3hb$

firewall --service=ssh

authconfig --enableshadow --passalgo=sha512

timezone --utc Pacific/Auckland

repo --name="CentOS" --baseurl=http://10.10.10.1/pub/ --cost=100 #replace the base url from #cdrom or any other to suit yours

reboot

%packages --nobase

@core

%post

cd /tmp

wget http://10.10.10.1/post-install/myrepo.sh

chmod +x local-repo.sh

./repo.sh

%end

Save this file as ks.cfg into /var/www/html/install

Now we create our own repo:

# mkdir -p /var/www/html/reposit

# yum install createrepo

# createrepo /var/www/html/reposit

# rsync -avz rsync://centos.ar.host-engine.com/6.6/os/x86_64/ /var/www/html/reposit

# createrepo --update /var/www/html/reposit

Once this is all done, we make a script

# mkdir -p /var/www/html/post-install

# cd /var/www/html/post-install

# nano myrepo.sh

cd /etc/yum.repos.d

rm *.repo

wget http://10.10.10.1/post-install/myrepo.repo

yum clean all

yum -y update

Save it and create a local repo, also in the same folder like this:

# nano myrepo.repo

[myrepo]

name=myrepo

baseurl=http://10.10.10.1/reposit/

gpgcheck=1

gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-6

Save it.

Now boot up a client and when the installer screen comes in press tab and enter this all in one line:

vmlinuz initrd=initrd.img ks=http://10.10.10.1/install/ks.cfg append ip=10.10.10.100 netmask=255.255.255.0 ksdevice=eth0

Job Done!