Redhat satellite or Spacewalk – real time push to clients.

By default, a client waits for a set of interval (minutes) configured in /etc/sysconfig/rhn/rhnsd to pull scheduled tasks from satellite server. For instance, if a remote command is set to be executed on client or a patch is waiting to be applied, rhn_check has to wait at least for 60 minutes to pick up the task.

For real time command execution or patch or configuration deployment, the following steps have to be performed on server and client –

1. Server : Install osa-dispatcher

root:homevm:~:# rpm -q osa-dispatcher
osa-dispatcher-5.11.43-1.el6.noarch

root:homevm:~:# service osa-dispatcher status

root:homevm:~:# chkconfig osa-dispatcher on

root:homevm:~:# chkconfig osa-dispatcher --list
osa-dispatcher  0:off   1:off   2:on    3:on    4:on    5:on    6:off

2. Client : Install and enable osad (OSA daemon).

# yum install osad -y
# chkconfig osad on
# /etc/init.d/osad restart

3. Client : Make sure the deploy and run options are enabled –

# rhn-actions-control --enable-run
# rhn-actions-control --enable-deploy

# rhn-actions-control --report
deploy is enabled
diff is enabled
upload is enabled
mtime_upload is enabled
run is enabled

Extra steps in case you encounter SSL certificate issues –
OSA is picky on SSL certificte verification, make sure the right CA cert is deployed on client, and the serverURL on up2date should match with the CN on the server certificate.

1. Copy RHN certificate from satellite server to client, make sure the cert has not expired and the CN matches server name.

root:homevm:~:# openssl x509 -in /var/www/html/pub/RHN-ORG-TRUSTED-SSL-CERT -noout -subject
subject= /C=US/ST=CA/L=SanFrancisco/O=home.net/OU=spacewalk.home.net/CN=homevm.home.net

root:homevm:~:# openssl x509 -in /var/www/html/pub/RHN-ORG-TRUSTED-SSL-CERT -noout -dates
notBefore=Aug  2 06:04:05 2014 GMT
notAfter=Jul 27 06:04:05 2036 GMT

root:homevm:~:# scp /var/www/html/pub/RHN-ORG-TRUSTED-SSL-CERT root@client:/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT

[root@blackhat rpm-gpg]# grep -i serverurl /etc/sysconfig/rhn/up2date 
serverURL=http://homevm.home.net/XMLRPC

2. If you get certificate error, during package deployment, copy the RPM GPG public keys from satellite to the clients
On Server side –

root:homevm:/etc/pki/rpm-gpg:# ls -al RPM-GPG-KEY-*
-rw-r--r-- 1 root root 1706 Nov 30  2013 RPM-GPG-KEY-CentOS-6
-rw-r--r-- 1 root root 1730 Nov 30  2013 RPM-GPG-KEY-CentOS-Debug-6
-rw-r--r-- 1 root root 1730 Nov 30  2013 RPM-GPG-KEY-CentOS-Security-6
-rw-r--r-- 1 root root 1734 Nov 30  2013 RPM-GPG-KEY-CentOS-Testing-6
-rw-r--r-- 1 root root 1649 Nov  4  2012 RPM-GPG-KEY-EPEL-6
-rw-r--r-- 1 root root 1011 Feb  5  2011 RPM-GPG-KEY-oracle

root:homevm:/etc/pki/rpm-gpg:# scp RPM-GPG-KEY-* root@client:/etc/pki/rpm-gpg

On client side -

# rpm --import RPM-GPG-KEY-CentOS-*

References –
https://access.redhat.com/documentation/en-US/Red_Hat_Network_Satellite/5.3/html/Installation_Guide/s1-maintenance-push-clients.html

Reduce or shrink the size of non root LVM mount.

In a system with limited disk size, you might run out of disk space in one LVM mount while having plenty of space in another mount. If both LVMs are in the same volume group (VGs), you can easily take away some of the free space from one LVM and add it to the one with low disk space. Both lvreduce and lvresize commands can be used to shrink the LVM. In this example, we will use lvresize.

Note – the steps below have to be done with care, there is a potential for losing data. If the data in the existing partition is critical, make sure you take a backup.

Shrink LVM by example – we will reduce the LVM for /usr/local file system mount from 2.0G to approximately 1.5G.

1. Unmount partition after confirming no file is in use from the partition.

root:homevm:~:# df -Pvh /usr/local
/dev/mapper/vg00-lvol04  2.0G   68M  1.9G   4% /usr/local

root:homevm:~:# lsof /dev/mapper/vg00-lvol04 

root:homevm:~:# umount /usr/local/

2. Do a file system consistency check –

root:homevm:~:# e2fsck -f /dev/mapper/vg00-lvol04 
e2fsck 1.41.12 (17-May-2010)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mapper/vg00-lvol04: 46/131072 files (0.0% non-contiguous), 25423/524288 blocks

3. Reduce the file system first, so that the logical volume is always at least as large as the file system expects it to be.

root:homevm:~:# resize2fs /dev/mapper/vg00-lvol04 1400M
resize2fs 1.41.12 (17-May-2010)
Resizing the filesystem on /dev/mapper/vg00-lvol04 to 358400 (4k) blocks.
The filesystem on /dev/mapper/vg00-lvol04 is now 358400 blocks long.

root:homevm:~:# mount /usr/local/

root:homevm:~:# lvresize -L 1500M /dev/mapper/vg00-lvol04 
  Rounding size to boundary between physical extents: 1.47 GiB
  WARNING: Reducing active and open logical volume to 1.47 GiB
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce lvol04? [y/n]: y
  Reducing logical volume lvol04 to 1.47 GiB
  Logical volume lvol04 successfully resized

root:homevm:~:# resize2fs /dev/mapper/vg00-lvol04 
resize2fs /dev/mapper/vg00-lvol04
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/mapper/vg00-lvol04 is mounted on /usr/local; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/mapper/vg00-lvol04 to 385024 (4k) blocks.
The filesystem on /dev/mapper/vg00-lvol04 is now 385024 blocks long.

root:homevm:~:# df -Pvh /usr/local
/dev/mapper/vg00-lvol04  1.5G   68M  1.4G   5% /usr/local

References –
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/Logical_Volume_Manager_Administration/

When will the SSL certificate for a site expire or in how many days will an SSL certificate expire?

If you are a system administrator, at least once in your career you might have worked with managing SSL certificates as well as making sure that SSL certificates are renewed before they expire. I have seen Linux admins using Nagios to monitor SSL certificates and get notified a few days before expiry and in some cases admins setup a cron job which polls the sites to be monitored and send out an email if any of the certs for a site are going to expire soon.

Googling for information on how to check SSL certificate expiration for a site might return results like this one on openssl s_client.

My favorite tool for getting certificate expiry is the Nagios plugin utility – check_http. The check_http script displays the exact date/time the SSL certificate for a given site expires as well as how many days are left before expiry.

Installation –

apt-get install nagios-plugins
yum install nagios-plugins-all

In my system, the plugins were installed under /usr/lib/nagios/plugins directory –

root@linubuvma:/usr/lib/nagios/plugins# cat /etc/issue
Ubuntu 14.04.5 LTS \n \l

root@linubuvma:/usr/lib/nagios/plugins# pwd
/usr/lib/nagios/plugins

root@linubuvma:/usr/lib/nagios/plugins# ls
check_apt      check_dbi       check_dns       check_host       check_ifoperstatus  check_ldap   check_mrtg         check_nntp      check_ntp_time  check_ping   check_rta_multi  check_spop   check_time   negate
check_breeze   check_dhcp      check_dummy     check_hpjd       check_ifstatus      check_ldaps  check_mrtgtraf     check_nntps     check_nwstat    check_pop    check_sensors    check_ssh    check_udp    urlize
check_by_ssh   check_dig       check_file_age  check_http       check_imap          check_load   check_mysql        check_nt        check_oracle    check_procs  check_simap      check_ssmtp  check_ups    utils.pm
check_clamd    check_disk      check_flexlm    check_icmp       check_ircd          check_log    check_mysql_query  check_ntp       check_overcr    check_real   check_smtp       check_swap   check_users  utils.sh
check_cluster  check_disk_smb  check_ftp       check_ide_smart  check_jabber        check_mailq  check_nagios       check_ntp_peer  check_pgsql     check_rpc    check_snmp       check_tcp    check_wave

How to get the expiry information?

The -C option of check_http is what we are looking for. The help page for check_http explains the -C option as below –

-C, --certificate=INTEGER
Minimum number of days a certificate has to be valid. Port defaults to 443
(when this option is used the URL is not checked.)

Let us test it if any of the sites below have certificates which expire in the coming 30 days –

root@linubuvma:/usr/lib/nagios/plugins# ./check_http -t 60 -H yahoo.com -C 30
OK - Certificate 'www.yahoo.com' will expire on 10/30/2017 23:59.

root@linubuvma:/usr/lib/nagios/plugins# ./check_http -t 60 -H gmail.com -C 30
OK - Certificate 'mail.google.com' will expire on 03/09/2017 13:34.

root@linubuvma:/usr/lib/nagios/plugins# ./check_http -t 60 -H linuxfreelancer.com -C 30
OK - Certificate 'linuxfreelancer.com' will expire on 08/12/2017 03:01.

In order for check_http to show us how many days are left before the SSL certificate expires, we give it a much longer number of days (-C) –

root@linubuvma:/usr/lib/nagios/plugins# ./check_http -t 60 -H yahoo.com -C 1000
WARNING - Certificate 'www.yahoo.com' expires in 298 day(s) (10/30/2017 23:59).

root@linubuvma:/usr/lib/nagios/plugins# ./check_http -t 60 -H gmail.com -C 1000
WARNING - Certificate 'mail.google.com' expires in 63 day(s) (03/09/2017 13:34).

root@linubuvma:/usr/lib/nagios/plugins# ./check_http -t 60 -H linuxfreelancer.com -C 1000
WARNING - Certificate 'linuxfreelancer.com' expires in 219 day(s) (08/12/2017 03:01).

If the output doesn’t show the number of days left or the status is ‘OK’, keep on increasing the number of days. The ‘-t’ option is the connection timeout in seconds. In addition to running it interactively, check_http is very useful for scripting as well as automated monitoring.