Programmer should know about memory

Hereby am sharing some of points about modern memory architecture

- Intel Hyper-threading enabled single processor core to be used for two or more concurrent executions with just little extra hardware

how exec system call works

Interesting implementation of parallel file upload and do socket polling 

Scenario  you need to look for the incoming data, and do ftp file upload. 

From C program you can invoke exec system call to do ftp file upload 

For FTP  either you can use CURL library  / linux / some other libraries .  I recommend CURL, i've been working on this quite a time.

Lets say:  I've a process  created thread in it ,doing socket select operations , 

               when i receive some X message i need to do file upload. 

Better approach: 

From your packet handler thread , spawn process  obliviously do "fork" , let child process do ftp file upload, parent wait for child to complete file upload. 

After file upload child process exited, followed by process detached . 

Here there is no interruption with packet handler routine, since both packet handling and ftp file upload runs in different process context .. so kernel would take care of scheduling them.. As application we achieving parallelism by this approach. 

Exec : system call could be used to  execute your binary inherit from process that has invoked, meaning exec system call replace current process context with new one. 

For this scenario,  "packet handling & ftp file upload post receive message" -- exec system call in new process context is best approach to achieve. 

Exec system call could be used in different scenarios w.r.t requirements.

Perforce Commands

Basic Commands:

P4 opened

function to set p4 client

function p4client()

{

        if [ -z $1 ]; then

                curr=`pwd`

                base=`basename $curr`

        else

                base=$1

        fi

        export P4PORT="p4sw1:1666"

        export P4USER="ssankar"

        export P4CLIENT="$base"

        export F10MKVERBOSE=1

}

P4 diff  <file>  or p4 diff – full diff

P4 revert <filename>

P4 submit –c <changelist no>

#save changelist (cl> to p4 server then pathc in other client

P4 shelve –c <cl no>

P4 unshelve –s <changelist no>

#delete p4 shelve

P4 shelve –d –c <changelistno>

 p4 client -d <clientname>

  

If you facing problem in deleting P4 Client ,  check for pending change list associated with P4 Client ,  revert the changes or you can delete forcibly if you have admin permission

p4 client -d  <p4clietn name>

p4 changes -c  <client-name> -s pending

p4 change -d <clno>

 p4 client -d <client-name>

print current previous next line for the context grep in a file

this is the simplest way i believe to get the previous,current, next line for content searched in a file

 #  perl -e 'while($line=<>) {if(!$found){if(grep(/netdump/,$line)){$curr=$line;$found=1;}else{$prev=$line;}}else{$next=$line;print $prev,$curr,$next;;last;}}' < /etc/passwd
haldaemon:x:68:68:HAL daemon:/:/sbin/nologin
netdump:x:34:34:Network Crash Dump user:/var/crash:/bin/bash
nscd:x:28:28:NSCD Daemon:/:/sbin/nologin
 #

How ping works

Here am going to share snapshot of how ping works,

we all know that ping is the utility program to test connectivity between hosts.

I am taking two categories ,

1. Ping host which is in same subnet
2. ping host which is in different network

case -1 :
    - when ping host (which is in same subnet ), first ARP broadcast happens , then icmp request/reply processed
    - broadcast mac ff:ff:ff:ff:ff:ff , destination ip address ARP broadcast will be zero as its unknown
case -2 :
    - icmp request sent to gateway as the destination address , as ping host is in different network so none of the present route matches
    - the ping can flow in any of the interfaces on the local host, depends on the destination ip subnet
     -     ping request/reply processed between localhost to gateway - remote host    
ARP cache can have its neighbor entries along with its gateway.

Difference between define and enum

define
----------
#define - is called symbolic constants
 - this is the one  among processing unit
 - just replaces the value defined , at the first stage of building your code i.e preprocessing stage
 - values can't be looked in debuggers 
-  definition should followed always  upper letters

#define TRUE  1
#define FALSE 0

enum
-----
- collective of constant integers
- it's not a proprocessor,  enumeration happens in compilation stage

-  advantage is auto increment
eg: instead of
#define ONE 1
#define  TWO 2 etc

use enum val {
 ZERO,
ONE,
etc};

- value always starts from zero, until not defined 
- the values can be looked in debuggers
- can use upper / lower characters


Note that both #define and enum are constants, they don't  memory allotted.

Register and Volatile Variable

Let us discuss why do we need register variable and how it helps.

- declaring register variable advices to compiler that this variable is heavily used
- register variable are to be placed in machine registers
- register variables can only be applied to automatic/local scope of variables
- register variables also depends on the hardware, since it directly deal with machine registers
- in case if compiler couldn't able to allot machine register for requesting register variable then the variable will be scoped as "automatic/local" variables.

What is Volatile ?  why do we need them ?

- volatile variable declaration advices compiler that don't optimize this variable, means always fetch the value from the harddisk / main memory
- for ex: there are the cases that values could changed intermediately by some other task where as current running would be using the value which was stored in virtual memory .. so the changed value on the disk it not read in the execution == this is how the compiler optimize the program , note  changing values in run time differs from reading the disk / main memory
- to do so  , declare the variable as volatile so that compiler wont optimize it , and it will always fetch from main memory , though it is inefficient
-- use register /  volatile is there is a critical requirement , otherwise it's my opinion to not use of them ..

External Linkage

Here let us discuss about  what is external linkage ? .

In global scope, identifiers for the following kinds of entities declared without the static storage class specifier have external linkage:

• Anobject --- variable
• A function -- always external, there is no function can be defined within another function 

External linkage is just a linkage for the function declared in another file, means references to the same name..


We can do consider soft links - can say symbolic soft link

ln -s dest src file

here we create symbolic link for the file exists. it just a replicate of existing file, for convenient and version library changes we create symbolic links .. however the reference to the library is same ...

I think this would be best example for the external linkage ..

External linkage - is a linkage to the global scope identities declared on different files.

Ex:  extern int sum(int); extern int number;

 

Gratuitous ARP (GARP)

I just come across new type in ARP, i.e GARP -> ARP Announcement .

This is very interesting message type,

Here i am assuming that person reading this article knowing about ARP and uses of ARP.

I am discussing about GARP called as ARP announcement.

Gratuitous ARPs (GARP) are useful for four reasons:

• They can help detect IP conflicts. When a machine receives an ARP request containing a source IP that matches its own, then it knows there is an IP conflict.
• They assist in the updating of other machines' ARP tables. Clustering solutions utilize this when they move an IP from one NIC to another, or from one machine to another. Other machines maintain an ARP table that contains the MAC associated with an IP. When the cluster needs to move the IP to a different NIC, be it on the same machine or a different one, it reconfigures the NICs appropriately then broadcasts a gratuitous ARP reply to inform the neighboring machines about the change in MAC for the IP. Machines receiving the ARP packet then update their ARP tables with the new MAC.
• They inform switches of the MAC address of the machine on a given switch port, so that the switch knows that it should transmit packets sent to that MAC address on that switch port.
• Every time an IP interface or link goes up, the driver for that interface will typically send a gratuitous ARP to preload the ARP tables of all other local hosts. Thus, a gratuitous ARP will tell us that that host just has had a link up event, such as a link bounce, a machine just being rebooted or the user/sysadmin on that host just configuring the interface up. If we see multiple gratuitous ARPs from the same host frequently, it can be an indication of bad Ethernet hardware/cabling resulting in frequent link bounces. 

Example with Traffic Flow Following as,

• Two nodes in a cluster are configured to share a common IP address 192.168.1.1. Node A has a hardware address of 01:01:01:01:01:01 and node B has a hardware address of 02:02:02:02:02:02.
• Assume that node A currently has IP address 192.168.1.1 already configured on its NIC. At this point, neighboring devices know to contact 192.168.1.1 using the MAC 01:01:01:01:01:01.
• Using the heartbeat protocol, node B determines that node A has died.
• Node B configures a secondary IP on an interface with ifconfig eth0:1 192.168.1.1.
• Node B issues a gratuitous ARP with send_arp eth0 192.168.1.1 02:02:02:02:02:02 192.168.1.255. All devices receiving this ARP update their table to point to 02:02:02:02:02:02 for the IP address 192.168.1.1.

As a conclusion, GARP is mainly used for avoid IP Conflict, maintaining ARP cache entries with proper mac address applying ARP announcement, whenever interface/server/port got replied with new hardware assigning same IP Address.


Thanks.

Let me know for any clarification required.

IPv4 Multicast Valid Range Addressing

IPv4 Multicast Group Address should be belongs to any of the range between,

starting from 224 to 239 , rest of ranges can't be accepted as IPv4 Multicast Group Valid Addressses

Ex: 224.0.0.0/24 to 239.0.0.0/24


By the what is multicast means

A Simple Real time example:
---------------------------

Multicast can describe follows, Being member of mailing group, whenever some one post message to that group, the person part of the group too receives mail. 

Ex: group name: xyz
members:
 john
arun
rahul
ram
..etc

here members are physically present, they have their own characteristics, whereas group name: xyz, no one present, it means that its a logical address, mapped with subscribed members.

therefore whenever there is a message post to a group xyz all the members would be getting a message.

Thanks.

let me know for any clarifications.

IPv6 Multicast Group Addressing

Following are the learning from RFC about IPv6 Multicast Group Addressing

Learning IPv6

rfc4291:
=======

The type of an IPv6 address is identified by the high-order bits of
   the address, as follows:

      Address type         Binary prefix        IPv6 notation   Section
      ------------         -------------        -------------   -------
      Unspecified          00...0  (128 bits)   ::/128          2.5.2
      Loopback             00...1  (128 bits)   ::1/128         2.5.3
      Multicast            11111111             FF00::/8        2.7
      Link-Local unicast   1111111010           FE80::/10       2.5.6
      Global Unicast       (everything else)

       An IPv6 multicast address is an identifier for a group of interfaces
   (typically on different nodes).  An interface may belong to any
   number of multicast groups.  Multicast addresses have the following
   format:

   |   8    |  4 |  4 |                  112 bits                   |
   +------ -+----+----+---------------------------------------------+
   |11111111|flgs|scop|                  group ID                   |
   +--------+----+----+---------------------------------------------+

      binary 11111111 at the start of the address identifies the address
      as being a multicast address.

                                    +-+-+-+-+
      flgs is a set of 4 flags:     |0|R|P|T|
                                    +-+-+-+-+

         The high-order flag is reserved, and must be initialized to 0.

         T = 0 indicates a permanently-assigned ("well-known") multicast
         address, assigned by the Internet Assigned Numbers Authority
         (IANA).

         T = 1 indicates a non-permanently-assigned ("transient" or
         "dynamically" assigned) multicast address.
       
       
         out of 112 bits - 80 bits unused must be "zeros"
         rest 32 bits - used for group id



Unicast-Prefix-based IPv6 Multicast Addresses
===============================================
 
   |   8    |  4 |  4 |   8    |    8   |       64       |    32    |
      +--------+----+----+--------+--------+----------------+----------+
      |11111111|flgs|scop|reserved|  plen  | network prefix | group ID |
      +--------+----+----+--------+--------+----------------+----------+

                                   +-+-+-+-+
   flgs is a set of 4 flags:       |0|0|P|T|
                                   +-+-+-+-+

         o  P = 0 indicates a multicast address that is not assigned
            based on the network prefix.

         o  P = 1 indicates a multicast address that is assigned based
            on the network prefix.

         o  If P = 1, T MUST be set to 1
          plen indicates the actual number of bits in the network prefix field
          that identify the subnet when P = 1.
         
          network prefix identifies the network prefix of the unicast subnet
            owning the multicast address.  If P = 1, this field contains the
            unicast network prefix assigned to the domain owning, or allocating,
            the multicast address.  All non-significant bits of the network
            prefix field SHOULD be zero.
           
   It should be noted that the Interface Identifier requirements in
   Section 2.5.1 of [ADDRARCH] effectively restrict the length of the
   unicast prefix to 64 bits, hence the network prefix portion of the
   multicast address will be at most 64 bits.

   Group ID is set based on the guidelines outlined in [IPV6 GID].

   The scope of the unicast-prefix based multicast address MUST NOT
   exceed the scope of the unicast prefix embedded in the multicast
   address.
  
  
Embedding the Rendezvous Point (RP) Address                  
in an IPv6 Multicast Address


FF01 to FF0F are well defined multicast ipv6 address family
first 8 bit always should be '1' - for multicast addressing
next 4 bit - 0
next 4 bit  follows notation defined scope

-  0  reserved
            1  interface-local scope
            2  link-local scope
            3  reserved
            4  admin-local scope
            5  site-local scope
            6  (unassigned)
            7  (unassigned)
            8  organization-local scope
            9  (unassigned)
            A  (unassigned)
            B  (unassigned)
            C  (unassigned)
            D  (unassigned)
            E  global scope
            F  reserved


1'st 16bits
===========
FF - 8bit

4it flag
0,1,2,3,5,7 - allowed
4 - can't be

4it - scope
0 to F out of which 0,3,f are reserved

2nd 16bits
==========

8 bit reserved case of unicase prefix based/ rvcd,RRId in case of ext embedding RP
8 bit pem  - used for subnet

3,4,5,6th 16bits
===================
In case unicast prefix based  used as - Network Prefix
All bits are zeros - normally

specially 6th 16bit - can be value as '1' for defined IANA addresses

7,8th 16bits
============
Group ID - can be 32bit mac address based EUI format how system create link local ip address based on mac

8th 16bit - can't be zero - which is reserved
             and can't be 100 to 12A
             7,8th 16 bits can't be 2:7ffd/e/f, 8000,ffff