NIC card fails to start

When NIC card fails to start try this

#echo "STARTMODE  auto" >> /etc/sysconfig/network/ifcfg-eth0

To manually start it up:
#ifup
#ifdown

How to split the files

#split -b 200m image.iso
#cat xa* > new-image.iso

It will generate three files - xaa, xab, xac, of 200MB each.
Then, use cat to combine the files:

SWAP File

Increase size of swap without reinstalling OS. Add a new partition or create a swap file.


#dd if=/dev/zero of=/swapfile bs=1024 count=(1024MB * 1024)
#mkswap /swapfile
#swapon /swapfile
#vi /etc/fstab
Add following:
/swapfile  swap  swap  defaults  0  0


Check swap size:
#free
#cat /proc/swap

3-SUM implementation - Java code

.
/*
 * 3-SUM problem
 * Are there 3 numbers a, b, c among N numbers whose sum is ZERO?
 * */


public class test{

public static void main(String[] args) {
int[] al = {1, 5, 7, 0, -1, -5, -8, 2, 4};
if(is3sum(al))
System.out.println("Above three numbers in the array have sum ZERO!");
else
System.out.println("There are no three numbers whose sum is ZERO");
}
static boolean is3sum(int[] al){
int j, k = 0;
int iterations = 0;
for (int i = 0; i <= al.length - 2; i++) {
j = i + 1;
k = al.length - 1;
while(k > j){
iterations++;
if(al[i] + al[j] + al[k] == 0){
System.out.println("Iterations: " + iterations);
System.out.print("Numbers: ");
System.out.println(al[i] + ", " + al[j] + ", " + al[k]);
return true;
}else if (al[i] + al[j] + al[k] > 0){
k = k - 1;
}else {
j = j + 1;
}
}
}
return false;
}

}

Lets try Networking

Lets ping


Script to ping the network:

#/bin/bash
for i in 10.112.1.1..255}
do
ping $i > /dev/null
[$? -eq 0] && echo $i is up
trap "exit" SIGINT
done

Run the script.

Configure Network:

ifconfig = for static IP.
dhclient == for dynamic IP using DHCP.

#ifconfig -a
This shows the interface devices available.
Output as:
+ eth0 is ethernet card.
+ lo is loopback device.
+ wlan0 is wireless LAN card.

Assign Static IP:
#ifconfig eth0 10.112.1.15

The see the information on eth0:
#ifconfig eth0

Setting the subnet mask:
#ifconfig eth0 10.112.1.15 netmask 255.255.255.0

Now get your ethernet up (or down)
#ifup eth0
#ifdown eth0

Wireless Networking
Use following commands:
#iwconfig
#iwconfig wlan0 essid <name of wireless network>
#iwconfig wlan0 essid <name of wireless network> key <KEY>

Scan and check availability of wireless networks:
#iwlist
#iwlist wlan0 scan
#ifconfig wlan0 <IP>
#dhclient wlan0

The IP settings assigned above for LAN or WLAN, is not persistent after system reboot. For that:

#vi /etc/networks/interfaces
 Add below lineto configure eth0 as DHCP:
 auto eth0
iface eth0 inet dhcp

For static:
auto eth0
iface eth0 inet static
 address <IP>
netmask <netmask>
gateway <gw ip>

If its a wireless network, add below also:
wireless-essid <network-name>
wireless-key <key>

Now, restart network daemon:
#/etc/init.d/network restart

Spoofing a MAC ID:
#ifconfig eth0 hw ether <new hw address>
Use this command to change the MAC address of your network card.

DNS:
It provides name resolution. Look at below file for name server's IP.
#cat /etc/resolv.conf
nameserver <IP>

Search more at www.opendns.org

SSH
Make remote logins to other machines.
[MachineA]#ssh test@<IP>
[MachineB]$

sftp is extension of SSH to transfer files.
#sftp  test@<IP>
sftp>get abc

To download files, use get command. To upload files, use put command.

sshfs is another extension of SSH to mount directories ona remote machine.
#sshfs test@<IP>:/home/test /mnt/test

Crazy Commands

1. When commands on CLI get longer and you need to edit it try this:
CTRL+E -- move to end
CTRL+A -- move to start
CTRL+W --delete word.

2. Use !$ that points to the last string in the last command.Many times we need last word of command in next command.

3. What ls or date does internally? Know the basic block of a command:
#strace -c /usr/bin/ls

strace is a system call monitor command and provice info about system calls made by an app.

4. Create a chain of dir:
#mkdir -p /tmp/dir1/dir2

5. Combine related commands:
#cd dir1 && ls && cd ..

6. fork-bomb
$ :(){:|:&};:

This is an unnamed function :() and body inside {}
Statement :|: makes a call to the function itself, and pipes the output to another function call.
& put all processes in bg.
; finishes the function.
: initiates a call to this unnamed function.

7. Edit /etc/inittab and modify line id:5:initdefault: to id:6:initdefault:
User mode 6 is for reboot. So the system keeps on rebooting.
As rescue, modify GRUB config and login to single user mode. Revert the changes.

cron - Scheduling jobs

cron makes tasks automatically run in the background at regular intervals. crontab file contains the schedule of cron entries to be run. Use below command to edit it.
#crobtab -e

Syntax:
*  *  *  *  *  command

first = day of week (0 - 6)
second = month (1-12)
third = day of month (1-31)
fourth = hour (0-23)
fifth = minute (0 - 59)

E.g. 15  5  *  *  *  script.sh

nginx - clustering reverse proxy

A Cluster is a group of closely linked computers that appear as single entity.
Types of Clusters:
- HA
- LB
- HPC
- grids

HA is also called failover cluster. It improves service availability rather than performance, by using redundant nodes.
Models of HA cluster config:
+ active-passive
+ active-active
+ N+1
+ N+M
+ N to 1
+ N to N

LB clusters distribute the workload evenly among redundant nodes.

HPC clusters are used for highly CPU-intensive compute jobs.Nodes are tightly couped.

Grid is a special class of compute clusters with heterogeneous nodes that are not tightly coupled.

Heartbeat is a piece of software from "The HA Linux" project.

The architecture: active -passive HA cluster.
Active-Passive: Primary node is active and serves the requests. When it fails, the services are transferred to secondary node.
Active-Active: Both nodes remain active and server their requests. When one fails, the services are transferred to the other.

A service that is served by the HA cluster depends on the IP address. Each interface on the cluster should have an administrative address and can have service addresses. OS controls the administrative address. whereas Heartbeat software controls service address and assigns it to the cluster nodes. Active node has the service address. When failover happens, passive node takes over the service address.

Install and Configure Heartbeat.
#apt-get install heartbeat

ha.cf file
The main config file for Heartbeat is ha.cf, which list the nodes of the cluster, communication topology, and al the features that are enabled.

haresources file
We need to tell Heartbeat about the resources the cluster will be managing.

authkeys file
The authkeys file authenticates the cluster nodes and maintains the cluster security.
File owner root and permissions 600, otherwise heartbeat will not start.
Heartbeat supports three authentication methods: crc, md5, sha1.

nginx

engine x is a HTTP/Web Server / reverse proxy and IMAP/POP3 reverse proxy.
A reverse proxy is a front end to to Web servers. Connections from Internet come via reverse proxy to the web server. So reverse proxy can act as a load balancer.

Architecture: Layer approach.
Layer 1: High Availability nginx reverse proxy for load balance the Web servers located in Layer 2.
Layer 2: nginx that server web sites.We can add more web servers to the cluster as per need (scalability depends on load)
Layer 3: Database servers

Install nginx:
#apt-get install nginx

Configure nginx on Web servers (Layer 2).
Copy original nginx.conf file.And modify nginx.conf.as per needs. Put a Hello World HTML file in root dir.
Then test the nginx configuration and start it.
#nginx -t
#invoke-rc.d nginx start

Configure the reverse proxy (Layer 1).
Open /etc/sites-available/rev-proxy-lb file.
The upstream directive is in the nginx_http_upstream module that balances load. It uses RR load-balance algo.

Bottleneck:
Active-passive clustering between two nginx reverse proxy servers is needed.

Networking Tools

Nmap - Network mapper.
It uses RAW raw IP packets to determine what hosts are available on network, the services those hosts are offering, the OS they are running, the firewalls in use, etc.


RRDtool - Round Robin Database Tool.
Used for data-logging, and graphical system.


Nagios - Network Monitoring Tool


Snort - Network Intrusion Prevention System.
Perform real-time traffic analysis, and packet logging.
It can perform protocol analysis, content searching, and attacks (buffer overflow, stealth port scan), CGI attack, SMB probes, OS fingerprinting etc.


Tor - network of virtual tunnels.
Improves security by sending your data through proxies.


Wireshark - a protocol analyzer.

Cacti - Network Graphing Solution.


OpenVAS - Oen Vulnerability Assessment System.

Disk Space Availability

du - disk usage
#du -ch

df - disk free
#df -h

Other tools:
KDirStat
Baobab on Gnome

Performance and Load Testing

We use software applications and we think of functional testing. But when that application's response time is very slow, then we think of performance testing.

Purpose of Performance Test:
No. 1 reason is to do performance test before deploying the application to production environment.
Most software performs good with limited users. But what happens when large amount of traffic hits a web application (in other words your software?).
Also a multi-user software doesnt make all the functions available to all its users.
It can also find realistic check of configuration between systems.
Lastly, performance tests can detect bottlenecks in the system.
Load and Performance Test is Non-Functional testing.

Topics of concern to Performance Test:
1. Resources.
2. Response time.
3. Multiple Users.

Some challenges:
1. The high scale system applications are distributed in nature.
2. The interaction between components and services is more abstract and complex.

Types of Non-Functional Tests (in this context):
1. Performance Test
2. Load Test
3. Volume Test
4. Stress Test

Such test find faults in following areas:
- Memory problems
- Performance issues
- Concurrency problems
- Excessive use of system resources
- Lack of storage space


 Lifecycle of Performance Testing means a cycle of tuning the system and doing testing.

Test Procedure: 
Define your objectives before taking any action!
1. Define Goals and specify metrics (benchmarks).
 2. Test Data
3. Reporting test results

Define benchmarks before running tests. Benchmarks are basis for selection of test techniques.
Benchmark should fir application requirement.
- Max Users.
- Max logged in users
- Avg user think time per page.
- Avg of max response time per page.
- Throughput ie No. of transactions per time unit.
- Test data based on realistic data.
- Reproducible baseline.
- Realistic test environment.

Performance requirements must be defined during specification phase.
Performance requirements should be checked for consistency and validity. SMART criteia:
- Specific.
- Measurable.
- Attainable.
- Relevant.
- Traceable.

Test data - Performance testing should be done with realistic data volumes.

Reporting test results:
- Response time on client side compared to system load.
- Throughput compared to system load.
- Faults occuring suring test.
- Utilization of processor and disk volume.
- Network utilization and network latency.

Diagnose Network Problems

1. PING.
#ping host

Try to ping the host. Ping shows RTT. RTT means how close a host is to you.
Ping also shows if the connection is unreliable. E.g. 50% packet loss.
ping works on ICMP.

#ping -f -i 1 host


#ping -f -c 500 -s 1400 host (Flood ping (dont use it))

2. TRACEROUTE
#tracerote host

Sometimes, ICMP, or UDP are not open on remote server. Then use below:
#traceroute -T (this uses TCP and not UDP)

3. MTR
Instead of above two, use MTR. It is a network diagnostic tool. And MTR = traceroute + ping.
It also shows you jitter, ie inconsistency in response time. In other words its the congestion in network.
Press j or d during mtr.

#mtr

Alternatively, use following tools. It gives more info than traceroute like firewalls, gateways, routers etc.
#lft
#conky on host
#gkrellm
#netstat
#iftop -nNPB
#iptraf

Cryptography

Cryptography: means to encrypt data transmission.

PKI: Public Key Cryptography is used to encrypt TCP/IP.
PKI use encryption algos for data security.
When a message is encryped using 8-bit key: 256 combinations of key needed to decrypt data.
If it is 32 bit: 65536 combinations needed.
So key length is important.
Also Algo used is important.
Algorithms: SHA1, 3DES, etc
Keys == symmetric & Asymmetric.
Symmetric use 1 key for encryption and decryption.
Asymmetric use two different keys.
Cryptography functionality is designed with objective of data confidentiality, integrity and authentication.


Internet use SSL and TLS for encrypt and decrypt the data sent.
SSL and TLS = asymmetric.
Two things: data is encrypted, and website is authentic.
Web server has SSL certificate, which nable traffic via HTTPS (TCP port 443).
SSL cert is signed by a certification authority (e.g. Verisign, Thawte).
SSL cert is tied to domain name.

Digital certificates using asymmetric PKI have two keys: public key and private key.
Private key is on Web server (website URL is secured by SSL)
Public key is on all browsers that support SSL.
Browsers support multiple CA vendors.
Public keys have expiration date and need to be updated.


Example:
Client hits website (SSL enabled).
Browser challenges Server by sending its own cipher strength.
Server does the same and responds by sending his SSL certificate.
Browser checks the validity and authenticity of certificate using public key.
Browser sends back a digitally signed response to server. In case server certificate cannot be verified for authenticity, browser alerts the user.
SSL introduces overhead. Addition of encryption layer to TCP/IP. Bigger TCP packets.

SQLlite

RDBMS are old. NoSQL or NewSQL is new. But this shift is sloq, coz RDBMS is mature.

SQLite is a Cross-platform, relational database management system contained in a small (~350 KB) C programming library that implements a self-contained, embedded, zero-configuration SQL database engine.

It was started as a embedded db project for US Navy.
It requires no Installation, No Administration.
It is used in mobile or desktop web apps. SQL92 standard compliant.
No configuration to setup.
Data is stored in a flat file on disk.
Db Max size == max file size on disk.
Serverless - requires no process for its running.
Embedded in Apps.
Can also be used as in-memory db.
Supports ACID.
Android, iOS, Firefox, Chrome, Opera use it.

sqlite3 is a command-line tool. Goto to the prompt and use .help for possible commands.

There are few limitations of SQLite - in terms of ALTER, JOIN, triggers, Views.

Alternatives: HSQLDB, Firebird, Ultralite.

Basic Linux Commands

#cat /etc/issue

#uname -a

#fdisk -l

#free -m

#df -h

#uptime

#last sn

#lastlog

#last reboot

#who

#w
(w = who + uptime + ps -a)

#lspci

#dmesg

#cat /proc/version

#cat /proc/cpuinfo

#cat /proc/interrupts

#cat /proc/filesystems

#cat /proc/partitions

#cat /proc/meminfo

#

SMTP

SMTP is used to transfer email across IP network. Here is how the email is transferred.

1.       Email is submitted by a mail client to a mail server using SMTP on TCP port 587 or 25.

2.       MSA delivers the mail to its mail transfer agent.

3.       The boundary MTA has to locate the target host. It uses the DNS to look up the mail exchanger record (MX record) for the recipient's domain.

4.       The MTA next connects to the exchange server as an SMTP client.

5.       MX target accepts the incoming message, and hands it to a mail delivery agent (MDA) for local mail delivery.

6.       MDA deliver messages to storage, or forward them over a network.

7.       On local mail server, the mail is stored for retrieval by mail clients (MUAs).

8.       Mail is retrieved by email clients, using IMAP or POP.

View who is accessing your services on Linux server

On Linux server, you want to see who is accessing your services?
#netstat
#lsof


To see Active Internet connections:
#netstat --inet


To see particular type of Active connections
#lsof -iTCP:https
#lsof -iUDP:rtp

ARP cache and ARP flush

When you contact another machine using IP or FQDN, its MAC address is mapped in your machine using ARP cache.

To I view current ARP cache on linux ubuntu:
#arp -a
#cat /proc/net/arp


To Flush the ARP cache:
#ip neigh flush dev eth0