Friday 25 September 2015

Fixing the GPG missing keys error

Sometimes we face the missing gpg key issue when we add diffrent sources to sources.list files, well this can be fixed easily:
Im gonna show an example where im using ubuntu 14 and try to first add sources of debian jessie 8.

first go to http://debgen.simplylinux.ch/

get your source list, for eg:

deb http://ftp.us.debian.org/debian testing main contrib non-free
deb-src http://ftp.us.debian.org/debian testing main contrib non-free

deb http://security.debian.org/ jessie/updates main contrib non-free
deb-src http://security.debian.org/ jessie/updates main contrib non-free


paste this in

nano /etc/apt/sources.list

then run this script:

#!/bin/bash

apt-get update 2> /tmp/keymissing; for key in $(grep "NO_PUBKEY" /tmp/keymissing |sed "s/.*NO_PUBKEY //"); do echo -e "\nProcessing key: $key"; gpg --keyserver pgpkeys.mit.edu --recv $key && gpg --export --armor $key | apt-key add -; done
apt-get update 2> /tmp/keymissing; for key in $(grep "NO_PUBKEY" /tmp/keymissing |sed "s/.*NO_PUBKEY //"); do echo -e "\nProcessing key: $key"; gpg --keyserver subkeys.pgp.net --recv $key && gpg --export --armor $key | apt-key add -; done

# it works if the command finished and says things like:
#  gpg: requesting key 46925553 from hkp server pgpkeys.mit.edu
#  gpg: key 46925553: "Debian Archive Automatic Signing Key (7.0/wheezy)
#  <ftpmaster@debian.org>" not changed
#  gpg: Total number processed: 1
#  gpg:              unchanged: 1

# If doesnt work and stuck on
# gpg: requesting key 46925553 from hkp server pgpkeys.mit.edu

# Then copy key from this line
# "Processing key: 8B48AD6246925553"
# and google search "8B48AD6246925553" this will return keyserver, like below, put those in for --keyserver line

# And look for keyserver

# Works for debian: pgpkeys.mit.edu
# Works for ubuntu: subkeys.pgp.net


This will work for both, ubuntu and debian - vice versa.

Special thanks to :  http://ram.kossboss.com/fix-missing-gpg-key-apt-get/

Monday 7 September 2015

Init Script Alternative

Install upstart package from repo, avaiable in both debian and centos.

To create a job to be started automatically when Ubuntu starts. As written example, suppose create the following file /etc/init/testservice.conf with sudo:

# testservice - test service job file

description "my service description"
author "Me <myself@i.com>"

# Stanzas
#
# Stanzas control when and how a process is started and stopped
# See a list of stanzas here: http://upstart.ubuntu.com/wiki/Stanzas

# When to start the service
start on runlevel [2345]

# When to stop the service
stop on runlevel [016]

# Automatically restart process if crashed
respawn

# Essentially lets upstart know the process will detach itself to the background
# This option does not seem to be of great importance, so it does not need to be set.
#expect fork

# Specify working directory
chdir /home/user/testcode

# Specify the process/command to start, e.g.
exec python mycommand.py arg1 arg2

To 'manually' start or stop the process use

sudo start testservice
sudo stop testservice

The above example works great for everything , but however, there is short cut too :

The solution is to use a shutdown-hook for the system. In Ubuntu it is very easy to add a shutdown-hook by using upstart.

The upstart job config file (/etc/init/shutdown-hook.conf) should look something like this:

description "run at shutdown"

start on starting rc
task
exec /bin/bash /etc/my_service/upload_log_to_s3.sh log

Finally I just needed the script (/etc/my_service/upload_log_to_s3.sh) for uploading a log file to S3:

#!/bin/bash

log_file_ext=$1

gzip -c /var/log/my_service/*.$log_file_ext > /tmp/log.gz

aws s3 cp /tmp/log.gz s3://my_service_bucket/logs/`date +%Y-%m-%dT%H:%M:%SZ`.log.gz

The upload script will just gzip the log file (needed as I’m using delaycompress), rename the log file to the current timestamp, and upload the file using aws-cli. The argument sets the file extension of the log file, which is necessary to be able to upload both the current (.log) as well as the previous log file (.log.1).

Sunday 6 September 2015

Post HTTP API grafana Query



The details are as follows:


URL : http://www.nasheikh.com:3000/api/datasources?=
Option : Post
Headers :  Authorization: Bearer eyJrIjoiT0tTcG1pUlY2RnVKZTFVaDFsNFZXdE9ZWmNrMkZYbk
Payload : 

{
        "name":"test3",
        "type":"influxdb_08",
        "url":"http://localhost:8086",
        "access":"proxy",
        "basicAuth":false,
        "database":"server1",
        "user":"root",
        "password":"root"
}


Do this in one liner in curl :

curl -H "Authorization: Bearer eyJrIjoiT0tTcG1pUlY2RnVKZTFVaDFsNFZXdE9ZWmNrMkZYbk" -H "Accept: application/json" -H "Content-type: application/json" -X POST -d @datainject.json http://www.nasheikh.com:3000/api/datasources?=

Here datainject.json is the file which is pasted above just saved as a json file and passed in curl.

Watch a file for its content, and perform an action accordingly:


Parent Script

#!/bin/bash

while true; do
  change=$(inotifywait -e close_write,moved_to,create .)
  change=${change#./ * }
  if [ "$change" = "changingfile.txt" ]; then ./child.sh; fi
done

Child Script

#!/bin/bash

rm -fr middle*
count=0
eraser=0
while read server
do
let count=$count+1
sed 's/server1/'${server}'/g;s/Tracker/'${server}'/g' center > middle.$count
sed 's/test1/'${server}'/g' datainject.json > newdata.$count.json
curl -H "Authorization: Bearer eyJrIjoiT0tTcG1pUlY2RnVKZTFVaDFsNFZXdE9ZWmNrMkZYbk" -H "Accept: application/json" -H "Content-type: application/json" -X POST -d @newdata.$count.json http://www.nasheikh.com:3000/api/datasources?=
sleep 2
done < network.txt
eraser=$(echo middle.$count)
sed -i '$ s/.$//' $eraser
sleep 2
cat prepend middle* final > merged.json
curl -H "Authorization: Bearer eyJrIjoiNzNjc2s5amxkZW9aUE81UXV3dVVOcmRHNTJhRkVOY1oiLCJuIjoiYWRtaW4iLCJpZCI6MX0=" -H "Accept: application/json" -H "Content-type: application/json" -X POST -d @merged.json http://nasheikh.com:3000/api/dashboards/db
rm -fr middle* merged.json newdata*

========================================================================

Saturday 5 September 2015

Update Server List from Amazon VPC

#!/bin/bash

aws ec2 describe-instances --query 'Reservations[*].Instances[*].[PublicIpAddress]' --output text --profile nash >> mainusa.txt

while true

do

aws ec2 describe-instances --query 'Reservations[*].Instances[*].[PublicIpAddress]' --output text --profile nash >> freshusa.txt

diff -Naur mainusa.txt freshusa.txt > my.patch

check=$(cat my.patch | wc -l)

echo $check

if [ "$check" -gt 0 ]

then

patch mainusa.txt < my.patch

else

echo "no chnage"

fi

rm -fr freshusa.txt my.patch

sleep 30

done