Howto check disk drive for errors and badblocks

source : http://linuxpoison.blogspot.com/2008/01/howto-check-disk-drive-for-errors-and.html

badblocks is a Linux utility to check for bad sectors on a disk drive (A bad sector is a sector on a computer's disk drive or flash memory that cannot be used due to permanent damage or an OS inability to successfully access it.). It creates a list of these sectors that can be used with other programs, like mkfs, so that they are not used in the future and thus do not cause corruption of data. It is part of the e2fsprogs project.

It can be a good idea to periodically check for bad blocks. This is done with the badblocks command. It outputs a list of the numbers of all bad blocks it can find. This list can be fed to fsck to be recorded in the filesystem data structures so that the operating system won’t try to use the bad blocks for storing data. The following example will show how this could be done.

From the terminal, type following command:

$ sudo badblocks -v /dev/hda1 > bad-blocks

The above command will generate the file bad-blocks in the current directory from where you are running this command.

Now, you can pass this file to the fsck command to record these bad blocks

$ sudo fsck -t ext3 -l bad-blocks /dev/hda1
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Check reference counts.
Pass 5: Checking group summary information.

/dev/hda1: ***** FILE SYSTEM WAS MODIFIED *****

/dev/hda1: 11/360 files, 63/1440 blocks

If badblocks reports a block that was already used, e2fsck will try to move the block to another place. If the block was really bad, not just marginal, the contents of the file may be corrupted.

Read more: http://linuxpoison.blogspot.com/2008/01/howto-check-disk-drive-for-errors-and.html#ixzz3alzpHpBf

explain .ssh/ contents?

-rw------- 1 root root 2.1K Nov 7 18:15 authorized_keys
> > > These are the public keys from other computer systems. You can revoke access by removing the lines in the file that pertain to the remote system.

-rw------- 1 root root 1.7K Nov 7 06:01 id_rsa
> > > This key is your private key, do not share it with anyone. I'll explain why in the next file.

-rw-r--r-- 1 root root 405 Nov 7 06:01 id_rsa.pub
> > > This key is your public key. Without the private key, you cannot access servers containing your public key. Anyone that has the private key can gain access into systems with the public key setup.

-rw-r--r--. 1 root root 1.2K Nov 6 07:41 known_hosts
> > > These are your trusted connections. If an IP address changes of a remote system (For instance, let's say you've authenticated with domains and not an IP address), you will receive a warning and should proceed with caution if the IP of the domain changes. In some cases, depending on your configuration, your system will deny you connecting to the remote system if there's a mismatch.


source : http://www.linux.org/threads/can-anyone-explain-ssh-contents.7160/

Regarding PORTMAP !!!!

What is port map why is it required ?

Linux uses a combination of kernel-level support and continuously running daemon processes to provide NFS file sharing, however, NFS support must be enabled in the Linux kernel in order to function. NFS uses Remote Procedure Calls (RPC) to route requests between clients and servers, meaning that the portmap service must be enabled and active at the proper runlevels for NFS communication to occur. Working with portmap, the following processes ensure that a given NFS connection is allowed and may proceed without error:

NFS and portmap

NFS relies upon remote procedure calls (RPC) to function. The portmap service is requiredto map RPC requests to the correct services. RPC processes notify portmap when they start, revealing the port number they are monitoring and the RPC program numbers they expect to serve. The client system then contacts portmap on the server with a particular RPC program number. portmap then redirects the client to the proper port number to communicate with its intended service.

Because RPC-based services rely on portmap to make all connections with incoming client requests, portmap must be available before any of these services start. If, for some reason, the portmap service unexpectedly quits, restart portmap and any services running when it was started.

The portmap service can be used with TCP wrappers' hosts access files (/etc/hosts.allowand /etc/hosts.deny) to control which remote systems are permitted to use RPC-based services on the server. See Chapter 15 TCP Wrappers and xinetd for more information. Access control rules for portmap will affect all RPC-based services. Alternatively, it is possible to specify each of the NFS RPC daemons to be affected by a particular access control rule. The man pages for rpc.mountd and rpc.statd contain information regarding the precise syntax for these rules.

The rpcinfo command shows each RPC-based service with its port number, RPC program number, version, and IP protocol type (TCP or UDP).

o make sure the proper NFS RPC-based services are enabled for portmap, use the rpcinfo -p command:

program vers proto port 100000 2 tcp 111 portmapper 100000 2 udp 111 portmapper 100024 1 udp 1024 status 100024 1 tcp 1024 status 100011 1 udp 819 rquotad 100011 2 udp 819 rquotad 100005 1 udp 1027 mountd 100005 1 tcp 1106 mountd 100005 2 udp 1027 mountd 100005 2 tcp 1106 mountd 100005 3 udp 1027 mountd 100005 3 tcp 1106 mountd 100003 2 udp 2049 nfs 100003 3 udp 2049 nfs 100021 1 udp 1028 nlockmgr 100021 3 udp 1028 nlockmgr 100021 4 udp 1028 nlockmgr

The -p option probes the portmapper on the specified host or defaults to localhost if no specific host is listed. Other options are available from the rpcinfo man page.

From this output, it is apparent that various NFS services are running. If one of the NFS services does not start up correctly, portmap will be unable to map RPC requests from clients for that service to the correct port. In many cases, restarting NFS as root (/sbin/service nfs restart) will cause those service to correctly register withportmap and begin working.

Configure an NTP Client And Server in Linux

source : http://www.thegeekstuff.com/2014/06/linux-ntp-server-client/


SERVER SIDE configuration :
======================

1) set /etc/hosts file :


 [root@pbldc-ntpsrv ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.20.163.16   pbldc-ntpsrv.pbl.com    pbldc-ntpsrv


2) install package :

yum install ntp

3) Working Port :

 UDP 123

4) Setup Restrict values in ntp.conf

Modify the /etc/ntp.conf file to make sure it has the following two restrict lines.


# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery


>>The first restrict line allows other clients to query your time server.
This restrict line has the following parameters :

> noquery prevents dumping status data from ntpd.
> notrap prevents control message trap service.
> nomodify prevents all ntpq queries that attempts to modify the server.
> nopeer prevents all packets that attempts to establish a peer association.
> Kod – Kiss-o-death packet is to be sent to reduce unwanted queries

The value -6 in the second line allows forces the DNS resolution to the IPV6 address
resolution. For more information on the access parameters list, Please refer
 to documentation on “man ntp_acc”



5)  Allow Only Specific Clients

To only allow machines on your own network to synchronize with your NTP server, add the following restrict line to your /etc/ntp.conf file:
restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap


If the localhost needs to have the full access to query or modify, add the following line to /etc/ntp.conf
restrict 127.0.0.1


6) Add Local Clock as Backup

Add the local clock to the ntp.conf file so that if the NTP server is disconnected from the internet, NTP server provides time from its local system clock.

server  127.127.1.0 # local clock
fudge   127.127.1.0 stratum 10

In the above line, Stratum is used to synchronize the time with the server based on distance. A stratum-1 time server acts as a primary network time standard. A stratum-2 server is connected to the stratum-1 server over the network. Thus, a stratum-2 server gets its time via NTP packet requests from a stratum-1 server. A stratum-3 server gets its time via NTP packet requests from a stratum-2 server, and so on.
Also stratum 0 devices are always used as reference clock.

7) Setup NTP Log Parameters

Specify the drift file and the log file location in your ntp.conf file
driftfile /var/lib/ntp/ntp.drift
logfile /var/log/ntp.log

driftfile is used to log how far your clock is from what it should be, and slowly ntp should lower this value as time progress.


6. Start the NTP Serrver

    service ntpd start
After setting up appropriate values in the ntp.conf file, start the ntp service:

service ntpd start



CLIENT SIDE configuration :
======================
1) cat /etc/ntp.conf

server pbldc-ad iburst
#restrict pbldc-ad mask 255.255.255.255 nomodify notrap noquery

2) Start ntp service :

/etc/init.d/ntpd start

NOTE :

>> Also Concentrate following configurations :


  1) ntpd service should be up

  2) No need to set any crontab entry .



>> Crontab Script :

[root@pbldc-ntpsrv ~]# cat  ntp.sh
#!/bin/bash
/usr/sbin/ntpdate -s bd.pool.ntp.org

[root@pbldc-ntpsrv ~]# cron
crond    crontab
[root@pbldc-ntpsrv ~]# crontab -l
#Time Syn with NTP
#* * * * *    sh /root/ntp.sh

Few Definitions !!!

http://www.pctechguide.com/glossary/disk-storage

1) Disk Overhead :

 disk overhead - (computer science) the disk space required for information that is not data but is used for location and timing .

Disk drive overhead is not solely applied to RAID system (although this case, you have 50% overhead because of RAID 1 mirror). What about a single IDE drive, does it have hard drive overhead? The answer is "Yes". Disk drive overhead is some spaces in the hard drive has been utilized for system file (FAT, NTFS, FHS....) and RAID array (if you have RAID configured). Hard drive overhead cannot be reduced in single drive. However, in RAID (multiple drives) you can reduce the overhead (in each drive) by adding more drives in your array (the total overhead is unchanged but it spread out to multi drives so each drive has smaller overhead usage).

Command overhead refers to the processing time required by the controller, host adapter, or drive prior to the execution of a command. Lower command overhead yields higher drive performance. Disk overhead refers to the space required for non- data information such as location and timing. Disk overhead often accounts for about ten percent of drive capacity. Lower disk overhead yields greater disk capacity.

RAID concept

User data in RAID can be stored using various data storage techniques:

• striping
• mirroring
• parity
• combination of the above

Striping is a way of writing data to member disks in which the data flow is split onto the blocks of a certain size and then written to the disks in turn.

Mirroring - stores identical data copies on the array member disks.

Parity is a type of data organization where splitting data onto the blocks (striping) is used along with calculation of a certain checksum which then is written to the member disks.

--

There are the following methods to write data to RAID.

• Mirroring: It is required to use no less than two disks. The same data is written on two of the array member disks. The example of such an array is RAID 1 (sometimes called Mirror).
• Striping: It is required to use no less than two disks. The data blocks are written on each of the array member disks in turn. The example of such an array is RAID 0.
• Parity: It is required to use no less than three disks. Along with data blocks, some checksum (called parity) is written to provide array redundancy. The examples are RAID 5 and RAID 6.

RAID levels

Depending on the way the data in RAID are organized, the following RAID types (so calledlevels) can be distinguished.

RAID 0 - sometimes, this RAID level is called stripe set since it is based on a striping technique. This is a non-redundant array and it is created mainly to increase system performance because N-disks array provides N times read and write speed improvement as compared to a single disk. For a controller failure, RAID 0 recovery is relatively simple and straightforward, However, if one of the disks is unreadable, almost nothing can be recovered.

RAID 1 is based on a mirroring technique. It is used where system reliability is critical. RAID 1 is very easy to recover - just attach the disk to a known-good PC.

RAID 0+1, same as RAID 1+0 is based on mirroring and striping techniques. The array inherits RAID 1 fault tolerance and RAID 0 speed efficiency. There is a 100% disk space overhead in RAID 0+1 implementations. Such arrays are able to survive a single disk failure and, in some cases, multiple simultaneous disk failures as well. This is one of the most expensive RAID types, because the capacity overhead increases with a number of disks.

RAID 5 - sometimes called stripe set with parity - utilizes striping and parity techniques. Parity technique provides fault tolerance - in case of a single disk failure the missing data is reconstructed using parity data and data from other member disks. Read speed for the array with one failed disk degrades dramatically. This is relatively inexpensive RAID type because disk space overhead equals the capacity of one member disk and doesn't increase with a number of disks. RAID 5 recovery is more complicated than for RAID 0, but the data can still be recovered if one of the disks had failed. There are many variations of RAID 5 layout: left and right, synchronous and asyncronous, plus an optional delayed parity.

RAID 6 is similar to RAID 5 but utilizes two different parity functions. Usually one of them is the same as in RAID 5 (XOR), but the second is more complex. On the one hand, using the two parity functions enables RAID 6 to survive a dual hard drive failure, on the other hand, the need to store parity data of the two functions results in that the disk space overhead equals the capacity of two member disks. RAID 6 recovery is very complex.

>>youtube : https://www.youtube.com/watch?v=Rn5LOtMBj_8

Configure Centralized Syslog server in Linux

source : http://freelinuxtutorials.com/tutorials/configure-centralized-syslog-server-in-linux-setup-syslog-clients-on-different-platforms/

Linux comes with a built-in syslog package that you can use as a centralized syslog server. Basically, it can be used to send all log files from your network devices or appliances in a remote and centralized location. There are also some popular syslog packages for Linux that you can use  such as syslog-ng,rsyslog & splunk syslog server which all offers more features and flexibility.

On this tutorial, we are going to use the syslogd/sysklogd server which is a multi-platform and proven stable software.

>[Syslog Server]

1.verify if the sysklogd package is installed.

[root@freelinux ~]# rpm -qa | grep sysklogd
sysklogd-1.4.1-46.el5

2. start the syslog daemon

[root@freelinux ~]# service syslog start
Starting system logger: [  OK  ]
Starting kernel logger: [  OK  ]

3. verify if the process is running

[root@freelinux ~]# ps -ef | grep syslog
root      2174     1  0 17:53 ?        00:00:00 syslogd -m 0
root      2180  2110  0 17:54 pts/1    00:00:00 grep syslog
[root@freelinux ~]# ls -la /var/run | grep syslog
-rw——-  1 root  root     5 Oct  9 17:53 syslogd.pid

4. configure the syslog

configuration files:
/etc/sysconfig/syslog
/etc/syslog.conf

Add the “-r” options to enable logging from the remote machines

[root@freelinux ~]# cat /etc/sysconfig/syslog
# Options to syslogd
# -m 0 disables ‘MARK’ messages.
# -r enables logging from remote machines
# -x disables DNS lookups on messages recieved with -r
# See syslogd(8) for more details
SYSLOGD_OPTIONS=”-r -m 0″
# Options to klogd
# -2 prints all kernel oops messages twice; once for klogd to decode, and
#    once for processing with ‘ksymoops’
# -x disables all klogd processing of oops messages entirely
# See klogd(8) for more details
KLOGD_OPTIONS=”-x”
#
SYSLOG_UMASK=077
# set this to a umask value to use for all log files as in umask(1).
# By default, all permissions are removed for “group” and “other”.

5. Restart the syslog service

[root@freelinux ~]# service syslog restart
Shutting down kernel logger: [  OK  ]
Shutting down system logger: [  OK  ]
Starting system logger: [  OK  ]
Starting kernel logger: [  OK  ]

>>[Client devices]

configuration file: /etc/syslog.conf

a. Linux servers

[root@freelinux ~]# cat /etc/syslog.conf
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

*.* @172.16.0.100

# Log anything (except mail) of level info or higher.
# Don’t log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog

# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 *

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log

b. Cisco/Motorola devices

Cisco#configure terminal
Cisco#logging facility local6
Cisco#logging 172.16.0.49

c. Juniper OS
darwin@Juniper> configure
Entering configuration mode

{master}[edit]
darwin@Juniper#
darwin@Juniper#  set system syslog host a.a.a.a facility-override local6 any any
darwin@Juniper#commit synch

darwin@Juniper#show configuration

host 172.16.0.100 {
any any;
facility-override local6;
}

d. Unix

# @(#)B.11.11_LR
#
# syslogd configuration file.
#
# See syslogd(1M) for information about the format of this file.
#
mail.debug              /var/adm/syslog/mail.log
*.info;mail.none;local1.none;local2.none;local5.none;local6.none;local7.none    /var/adm/syslog/syslog.log
*.alert                 /dev/console
#*.alert                        root
*.emerg                 *
local4.info             /var/adm/syslog/fw.log
local6.info             /var/adm/syslog/cisco.log

e. Windows

There’s no way to directly configure the syslog clients or send your event log messages to syslog server. You have to use syslog clients such as Snare & winlogd which I will not cover on this tutorial. Alternative way, is send your event logs as snmp traps by configuring your SNMP service & using the event to trap translator or “evntwin” command.

Additional Tips:

1. To make syslogd, re-read its configuration file, send it a HANGUP  signal:
[root@freelinux ~]# kill -HUP `cat /var/run/syslog.pid`

2. Familiarize with syslog facility and severity levels. A good reference about this is Wiki http://en.wikipedia.org/wiki/Syslog. Sample syslog.conf is on the Config-Scripts Section to show how this will be very useful for system administrators.

3.You can verify messages if it’s being logged in your syslog or if you want to test your /etc/syslog.conf, you can use the “logger” command

e.g.
@/etc/syslog.conf
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

[root@freelinux ~]# logger -p user.info “Test Message”
[root@freelinux ~]# tail /var/log/messages
Oct 11 21:25:39 localhost root: Test Message