Networking

总览

The development of the Internet has led to an exponential increase in network applications and, as a consequence, to increasing the speed and productivity requirements of an operating system’s networking subsystem. The networking subsystem is not an essential component of an operating system kernel (the Linux kernel can be compiled without networking support). It is, however, quite unlikely for a computing system (or even an embedded device) to have a non-networked operating system due to the need for connectivity. Modern operating systems use the TCP/IP stack. Their kernel implements protocols up to the transport layer, while application layer protocols are tipically implemented in user space (HTTP, FTP, SSH, etc.).

用户空间的Networking

In user space the abstraction of network communication is the socket. The socket abstracts a communication channel and is the kernel-based TCP/IP stack interaction interface. An IP socket is associated with an IP address, the transport layer protocol used (TCP, UDP etc) and a port. Common function calls that use sockets are: creation (socket), initialization (bind), connecting (connect), waiting for a connection (listen, accept), closing a socket (close).

Network communication is accomplished via read/write or recv/send calls for TCP sockets and recvfrom/sendto for UDP sockets. Transmission and reception operations are transparent to the application, leaving encapsulation and transmission over network at the kernel’s discretion. However, it is possible to implement the TCP/IP stack in user space using raw sockets (the PF_PACKET option when creating a socket), or implementing an application layer protocol in kernel (TUX web server).

For more details about user space programming using sockets, see Bee’s Guide to Network Programming Using Internet Sockets.

Linux networking

The Linux kernel provides three basic structures for working with network packets: struct socket, struct sock and struct sk_buff.

The first two are abstractions of a socket:

• struct socket is an abstraction very close to user space, ie BSD sockets used to program network applications;
• struct sock or INET socket in Linux terminology is the network representation of a socket.

The two structures are related: the struct socket contains an INET socket field, and the struct sock has a BSD socket that holds it.

The struct sk_buff structure is the representation of a network packet and its status. The structure is created when a kernel packet is received, either from the user space or from the network interface.

The struct socket structure

The struct socket structure is the kernel representation of a BSD socket, the operations that can be executed on it are similar to those offered by the kernel (through system calls). Common operations with sockets (creation, initialization/bind, closing, etc.) result in specific system calls; they work with the struct socket structure.

The struct socket operations are described in net/socket.c and are independent of the protocol type. The struct socket structure is thus a generic interface over particular network operations implementations. Typically, the names of these operations begin with the sock_ prefix

Operations on the socket structure

Socket operations are:

Creation

Creation is similar to calling the socket() function in user space, but the struct socket created will be stored in the res parameter:
reate(int family, int type, int protocol, struct socket **res) creates a socket after the socket() system call;

• int sock_create_kern(struct net *net, int family, int type, int protocol, struct socket **res) creates a kernel socket;
• int sock_create_lite(int family, int type, int protocol, struct socket **res) creates a kernel socket without parameter sanity checks.

The parameters of these calls are as follows:

• net, where it is present, used as reference to the network namespace used; we will usually initialize it with init_net;
• family represents the family of protocols used in the transfer of information; they usually begin with the PF_ (Protocol Family) string; the constants representing the family of protocols used are found in linux/socket.h, of which the most commonly used is PF_INET, for TCP/IP protocols;
  type is the type of socket; the constants used for this parameter are found in linux/net.h, of which the most used are SOCK_STREAM for a connection based source-to-destination communication and SOCK_DGRAM for connectionless communication;
  protocol represents the protocol used and is closely related to the type parameter; the constants used for this parameter are found in linux/in.h, of which the most used are IPPROTO_TCP for TCP and IPPROTO_UDP for UDP.

To create a TCP socket in kernel space, you must call:

struct socket *sock;
int err;

err = sock_create_kern(&init_net, PF_INET, SOCK_STREAM, IPPROTO_TCP, &sock);
if (err < 0) {
        /* handle error */
}

and for creating UDP sockets:

struct socket *sock;
int err;

err = sock_create_kern(&init_net, PF_INET, SOCK_DGRAM, IPPROTO_UDP, &sock);
if (err < 0) {
        /* handle error */
}

A usage sample is part of the sys_socket() system call handler:

SYSCALL_DEFINE3(socket, int, family, int, type, int, protocol)
{
      int retval;
      struct socket *sock;
      int flags;

      /* Check the SOCK_* constants for consistency.  */
      BUILD_BUG_ON(SOCK_CLOEXEC != O_CLOEXEC);
      BUILD_BUG_ON((SOCK_MAX | SOCK_TYPE_MASK) != SOCK_TYPE_MASK);
      BUILD_BUG_ON(SOCK_CLOEXEC & SOCK_TYPE_MASK);
      BUILD_BUG_ON(SOCK_NONBLOCK & SOCK_TYPE_MASK);

      flags = type & ~SOCK_TYPE_MASK;
      if (flags & ~(SOCK_CLOEXEC | SOCK_NONBLOCK))
              return -EINVAL;
      type &= SOCK_TYPE_MASK;

      if (SOCK_NONBLOCK != O_NONBLOCK && (flags & SOCK_NONBLOCK))
              flags = (flags & ~SOCK_NONBLOCK) | O_NONBLOCK;

      retval = sock_create(family, type, protocol, &sock);
      if (retval < 0)
              goto out;

      return sock_map_fd(sock, flags & (O_CLOEXEC | O_NONBLOCK));
}

Closing

Close connection (for sockets using connection) and release associated resources:

• void sock_release(struct socket *sock) calls the release function in the ops field of the socket structure:

void sock_release(struct socket *sock)
{
      if (sock->ops) {
              struct module *owner = sock->ops->owner;

              sock->ops->release(sock);
              sock->ops = NULL;
              module_put(owner);
      }
      //...
}

Sending/receiving messages

The messages are sent/received using the following functions:

• int sock_recvmsg(struct socket *sock, struct msghdr *msg, int flags);
• int kernel_recvmsg(struct socket *sock, struct msghdr *msg, struct kvec *vec, size_t num, size_t size, int flags);
• int sock_sendmsg(struct socket *sock, struct msghdr *msg);
• int kernel_sendmsg(struct socket *sock, struct msghdr *msg, struct kvec *vec, size_t num, size_t size);

The message sending/receiving functions will then call the sendmsg/ recvmsg function in the ops field of the socket. Functions containing kernel_ as a prefix are used when the socket is used in the kernel.

• msg, a struct msghdr structure, containing the message to be sent/received. Among the important components of this structure are msg_name and msg_namelen, which, for UDP sockets, must be filled in with the address to which the message is sent (struct sockaddr_in);
• vec, a struct kvec structure, containing a pointer to the buffer containing its data and size; as can be seen, it has a similar structure to the struct iovec structure (the struct iovec structure corresponds to the user space data, and the struct kvec structure corresponds to kernel space data)

A usage example can be seen in the sys_sendto() system call handler:

SYSCALL_DEFINE6(sendto, int, fd, void __user *, buff, size_t, len,
              unsigned int, flags, struct sockaddr __user *, addr,
              int, addr_len)
{
      struct socket *sock;
      struct sockaddr_storage address;
      int err;
      struct msghdr msg;
      struct iovec iov;
      int fput_needed;

      err = import_single_range(WRITE, buff, len, &iov, &msg.msg_iter);
      if (unlikely(err))
              return err;
      sock = sockfd_lookup_light(fd, &err, &fput_needed);
      if (!sock)
              goto out;

      msg.msg_name = NULL;
      msg.msg_control = NULL;
      msg.msg_controllen = 0;
      msg.msg_namelen = 0;
      if (addr) {
              err = move_addr_to_kernel(addr, addr_len, &address);
              if (err < 0)
                      goto out_put;
              msg.msg_name = (struct sockaddr *)&address;
              msg.msg_namelen = addr_len;
      }
      if (sock->file->f_flags & O_NONBLOCK)
              flags |= MSG_DONTWAIT;
      msg.msg_flags = flags;
      err = sock_sendmsg(sock, &msg);

out_put:
      fput_light(sock->file, fput_needed);
out:
      return err;
}

The struct socket fields

/**
 *  struct socket - general BSD socket
 *  @state: socket state (%SS_CONNECTED, etc)
 *  @type: socket type (%SOCK_STREAM, etc)
 *  @flags: socket flags (%SOCK_NOSPACE, etc)
 *  @ops: protocol specific socket operations
 *  @file: File back pointer for gc
 *  @sk: internal networking protocol agnostic socket representation
 *  @wq: wait queue for several uses
 */
struct socket {
      socket_state            state;

      short                   type;

      unsigned long           flags;

      struct socket_wq __rcu  *wq;

      struct file             *file;
      struct sock             *sk;
      const struct proto_ops  *ops;
};

The noteworthy fields are:

• ops - the structure that stores pointers to protocol-specific functions;
• sk - The INET socket associated with it.

The struct proto_ops structure

The struct proto_ops structure contains the implementations of the specific operations implemented (TCP, UDP, etc.); these functions will be called from generic functions through struct socket (sock_release(), sock_sendmsg(), etc.)

The struct proto_ops structure therefore contains a number of function pointers for specific protocol implementations:

struct proto_ops {
      int             family;
      struct module   *owner;
      int             (*release)   (struct socket *sock);
      int             (*bind)      (struct socket *sock,
                                    struct sockaddr *myaddr,
                                    int sockaddr_len);
      int             (*connect)   (struct socket *sock,
                                    struct sockaddr *vaddr,
                                    int sockaddr_len, int flags);
      int             (*socketpair)(struct socket *sock1,
                                    struct socket *sock2);
      int             (*accept)    (struct socket *sock,
                                    struct socket *newsock, int flags, bool kern);
      int             (*getname)   (struct socket *sock,
                                    struct sockaddr *addr,
                                    int peer);
      //...
}

The initialization of the ops field from struct socket is done in the __sock_create() function, by calling the create() function, specific to each protocol; an equivalent call is the implementation of the __sock_create() function:

//...
      err = pf->create(net, sock, protocol, kern);
      if (err < 0)
              goto out_module_put;
//...

This will instantiate the function pointers with calls specific to the protocol type associated with the socket. The sock_register() and sock_unregister() calls are used to fill the net_families vector.

For the rest of the socket operations (other than creating, closing, and sending/receiving a message as described above in the Operations on the socket structure section), the functions sent via pointers in this structure will be called. For example, for bind, which associates a socket with a socket on the local machine, we will have the following code sequence:

#define MY_PORT 60000

struct sockaddr_in addr = {
      .sin_family = AF_INET,
      .sin_port = htons (MY_PORT),
      .sin_addr = { htonl (INADDR_LOOPBACK) }
};

//...
      err = sock->ops->bind (sock, (struct sockaddr *) &addr, sizeof(addr));
      if (err < 0) {
              /* handle error */
      }
//...

As you can see, for transmitting the address and port information that will be associated with the socket, a struct sockaddr_in is filled.

The struct sock structure

The struct sock describes an INET socket. Such a structure is associated with a user space socket and implicitly with a struct socket structure. The structure is used to store information about the status of a connection. The structure’s fields and associated operations usually begin with the sk_ string. Some fields are listed below:

struct sock {
      //...
      unsigned int            sk_padding : 1,
                              sk_no_check_tx : 1,
                              sk_no_check_rx : 1,
                              sk_userlocks : 4,
                              sk_protocol  : 8,
                              sk_type      : 16;
      //...
      struct socket           *sk_socket;
      //...
      struct sk_buff          *sk_send_head;
      //...
      void                    (*sk_state_change)(struct sock *sk);
      void                    (*sk_data_ready)(struct sock *sk);
      void                    (*sk_write_space)(struct sock *sk);
      void                    (*sk_error_report)(struct sock *sk);
      int                     (*sk_backlog_rcv)(struct sock *sk,
                                                struct sk_buff *skb);
      void                    (*sk_destruct)(struct sock *sk);
};

• sk_protocol is the type of protocol used by the socket;
• sk_type is the socket type (SOCK_STREAM, SOCK_DGRAM, etc.);
• sk_socket is the BSD socket that holds it;
• sk_send_head is the list of struct sk_buff structures for transmission;
• the function pointers at the end are callbacks for different situations.

Initializing the struct sock and attaching it to a BSD socket is done using the callback created from net_families (called __sock_create()). Here’s how to initialize the struct sock structure for the IP protocol, in the inet_create() function:

/*
 *    Create an inet socket.
 */

static int inet_create(struct net *net, struct socket *sock, int protocol,
                     int kern)
{

      struct sock *sk;

      //...
      err = -ENOBUFS;
      sk = sk_alloc(net, PF_INET, GFP_KERNEL, answer_prot, kern);
      if (!sk)
              goto out;

      err = 0;
      if (INET_PROTOSW_REUSE & answer_flags)
              sk->sk_reuse = SK_CAN_REUSE;


      //...
      sock_init_data(sock, sk);

      sk->sk_destruct    = inet_sock_destruct;
      sk->sk_protocol    = protocol;
      sk->sk_backlog_rcv = sk->sk_prot->backlog_rcv;
      //...
}

The struct sk_buff structure

The struct sk_buff (socket buffer) describes a network packet. The structure fields contain information about both the header and packet contents, the protocols used, the network device used, and pointers to the other struct sk_buff. A summary description of the content of the structure is presented below:

struct sk_buff {
      union {
              struct {
                      /* These two members must be first. */
                      struct sk_buff          *next;
                      struct sk_buff          *prev;

                      union {
                              struct net_device       *dev;
                              /* Some protocols might use this space to store information,
                               * while device pointer would be NULL.
                               * UDP receive path is one user.
                               */
                              unsigned long           dev_scratch;
                      };
              };

              struct rb_node  rbnode; /* used in netem & tcp stack */
      };
      struct sock             *sk;

        union {
              ktime_t         tstamp;
              u64             skb_mstamp;
      };

      /*
       * This is the control buffer. It is free to use for every
       * layer. Please put your private variables there. If you
       * want to keep them across layers you have to do a skb_clone()
       * first. This is owned by whoever has the skb queued ATM.
       */
      char                    cb[48] __aligned(8);

      unsigned long           _skb_refdst;
      void                    (*destructor)(struct sk_buff *skb);
        union {
              struct {
                      unsigned long   _skb_refdst;
                      void            (*destructor)(struct sk_buff *skb);
              };
              struct list_head        tcp_tsorted_anchor;
      };
      /* ... */

      unsigned int            len,
                              data_len;
      __u16                   mac_len,
                              hdr_len;

         /* ... */

      __be16                  protocol;
      __u16                   transport_header;
      __u16                   network_header;
      __u16                   mac_header;

      /* private: */
      __u32                   headers_end[0];
      /* public: */

      /* These elements must be at the end, see alloc_skb() for details.  */
      sk_buff_data_t          tail;
      sk_buff_data_t          end;
      unsigned char           *head,
                              *data;
      unsigned int            truesize;
      refcount_t              users;
};

where:

• next and prev are pointers to the next, and previous element in the buffer list;
• dev is the device which sends or receives the buffer;
• sk is the socket associated with the buffer;
• destructor is the callback that deallocates the buffer;
• transport_header, network_header, and mac_header are offsets between the beginning of the packet and the beginning of the various headers in the packets. They are internally maintained by the various processing layers through which the packet passes. To get pointers to the headers, use one of the following functions: tcp_hdr(), udp_hdr(), ip_hdr(), etc. In principle, each protocol provides a function to get a reference to the header of that protocol within a received packet. Keep in mind that the network_header field is not set until the packet reaches the network layer and the transport_header field is not set until the packet reaches the transport layer.

The structure of an IP header (struct iphdr) has the following fields:

struct iphdr {
#if defined(__LITTLE_ENDIAN_BITFIELD)
      __u8    ihl:4,
              version:4;
#elif defined (__BIG_ENDIAN_BITFIELD)
      __u8    version:4,
              ihl:4;
#else
#error        "Please fix <asm/byteorder.h>"
#endif
      __u8    tos;
      __be16  tot_len;
      __be16  id;
      __be16  frag_off;
      __u8    ttl;
      __u8    protocol;
      __sum16 check;
      __be32  saddr;
      __be32  daddr;
      /*The options start here. */
};

where:

• protocol is the transport layer protocol used;
• saddr is the source IP address;
• daddr is the destination IP address.

The structure of a TCP header (struct tcphdr) has the following fields:

struct tcphdr {
      __be16  source;
      __be16  dest;
      __be32  seq;
      __be32  ack_seq;
#if defined(__LITTLE_ENDIAN_BITFIELD)
      __u16   res1:4,
              doff:4,
              fin:1,
              syn:1,
              rst:1,
              psh:1,
              ack:1,
              urg:1,
              ece:1,
              cwr:1;
#elif defined(__BIG_ENDIAN_BITFIELD)
      __u16   doff:4,
              res1:4,
              cwr:1,
              ece:1,
              urg:1,
              ack:1,
              psh:1,
              rst:1,
              syn:1,
              fin:1;
#else
#error        "Adjust your <asm/byteorder.h> defines"
#endif
      __be16  window;
      __sum16 check;
      __be16  urg_ptr;
};

where:

• source is the source port;
• dest is the destination port;
• syn, ack, fin are the TCP flags used; for a more detailed view, see this diagram.

The structure of a UDP header (struct udphdr) has the following fields:

struct udphdr {
      __be16  source;
      __be16  dest;
      __be16  len;
      __sum16 check;
};

where:

• source is the source port;
• dest is the destination port.

An example of accessing the information present in the headers of a network packet is as follows:

struct sk_buff *skb;

struct iphdr *iph = ip_hdr(skb);                 /* IP header */
/* iph->saddr  - source IP address */
/* iph->daddr  - destination IP address */
if (iph->protocol == IPPROTO_TCP) {              /* TCP protocol */
        struct tcphdr *tcph = tcp_hdr(skb);      /* TCP header */
        /* tcph->source  - source TCP port */
        /* tcph->dest    - destination TCP port */
} else if (iph->protocol == IPPROTO_UDP) {       /* UDP protocol */
        struct udphdr *udph = udp_hdr(skb);      /* UDP header */
        /* udph->source  - source UDP port */
        /* udph->dest    - destination UDP port */
}

Conversions

In different systems, there are several ways of ordering bytes in a word (Endianness), including: Big Endian (the most significant byte first) and Little Endian (the least significant byte first). Since a network interconnects systems with different platforms, the Internet has imposed a standard sequence for the storage of numerical data, called network byte-order. In contrast, the byte sequence for the representation of numerical data on the host computer is called host byte-order. Data received/sent from/to the network is in the network byte-order format and should be converted between this format and the host byte-order.

For converting we use the following macros:

• u16 htons(u16 x) converts a 16 bit integer from host byte-order to network byte-order (host to network short);
• u32 htonl(u32 x) converts a 32 bit integer from host byte-order to network byte-order (host to network long);
• u16 ntohs(u16 x) converts a 16 bit integer from network byte-order to host byte-order (network to host short);
• u32 ntohl(u32 x) converts a 32 bit integer from network byte-order to host byte-order (network to host long).

netfilter

Netfilter is the name of the kernel interface for capturing network packets for modifying/analyzing them (for filtering, NAT, etc.). The netfilter interface is used in user space by iptables.

In the Linux kernel, packet capture using netfilter is done by attaching hooks. Hooks can be specified in different locations in the path followed by a kernel network packet, as needed. An organization chart with the route followed by a package and the possible areas for a hook can be found here.

The header included when using netfilter is linux/netfilter.h.

A hook is defined through the struct nf_hook_ops structure:

struct nf_hook_ops {
      /* User fills in from here down. */
      nf_hookfn               *hook;
      struct net_device       *dev;
      void                    *priv;
      u_int8_t                pf;
      unsigned int            hooknum;
      /* Hooks are ordered in ascending priority. */
      int                     priority;
};

where:

• pf is the package type (PF_INET, etc.);
• priority is the priority; priorities are defined in
  uapi/linux/netfilter_ipv4.h as follows:

enum nf_ip_hook_priorities {
      NF_IP_PRI_FIRST = INT_MIN,
      NF_IP_PRI_CONNTRACK_DEFRAG = -400,
      NF_IP_PRI_RAW = -300,
      NF_IP_PRI_SELINUX_FIRST = -225,
      NF_IP_PRI_CONNTRACK = -200,
      NF_IP_PRI_MANGLE = -150,
      NF_IP_PRI_NAT_DST = -100,
      NF_IP_PRI_FILTER = 0,
      NF_IP_PRI_SECURITY = 50,
      NF_IP_PRI_NAT_SRC = 100,
      NF_IP_PRI_SELINUX_LAST = 225,
      NF_IP_PRI_CONNTRACK_HELPER = 300,
      NF_IP_PRI_CONNTRACK_CONFIRM = INT_MAX,
      NF_IP_PRI_LAST = INT_MAX,
};

• net_device is the device (network interface) on which the capture is intended;
• hooknum is the type of hook used. When a packet is captured, the processing mode is defined by the hooknum and hook fields. For IP, hook types are defined in linux/netfilter.h:

enum nf_inet_hooks {
      NF_INET_PRE_ROUTING,
      NF_INET_LOCAL_IN,
      NF_INET_FORWARD,
      NF_INET_LOCAL_OUT,
      NF_INET_POST_ROUTING,
      NF_INET_NUMHOOKS
};

• hook is the handler called when capturing a network packet (packet sent as a struct sk_buff structure). The private field is private information handed to the handler. The capture handler prototype is defined by the struct nf_hookfn type:

struct nf_hook_state {
      unsigned int hook;
      u_int8_t pf;
      struct net_device *in;
      struct net_device *out;
      struct sock *sk;
      struct net *net;
      int (*okfn)(struct net *, struct sock *, struct sk_buff *);
};

typedef unsigned int nf_hookfn(void *priv,
                             struct sk_buff *skb,
                             const struct nf_hook_state *state);

For the nf_hookfn() capture function, the priv field is the private information with which the struct nf_hook_ops was initialized. skb is the pointer to the captured network packet. Based on skb information, packet filtering decisions are made. The function’s state parameter is the status information related to the packet capture, including the input interface, the output interface, the priority, the hook number. Priority and hook number are useful for allowing the same function to be called by several hooks.

A capture handler can return one of the constants NF_*:

/* Responses from hook functions. */
#define NF_DROP 0
#define NF_ACCEPT 1
#define NF_STOLEN 2
#define NF_QUEUE 3
#define NF_REPEAT 4
#define NF_STOP 5
#define NF_MAX_VERDICT NF_STOP

NF_DROP is used to filter (ignore) a packet, and NF_ACCEPT is used to accept a packet and forward it.

Registering/unregistering a hook is done using the functions defined in linux/netfilter.h:

/* Function to register/unregister hook points. */
int nf_register_net_hook(struct net *net, const struct nf_hook_ops *ops);
void nf_unregister_net_hook(struct net *net, const struct nf_hook_ops *ops);
int nf_register_net_hooks(struct net *net, const struct nf_hook_ops *reg,
                        unsigned int n);
void nf_unregister_net_hooks(struct net *net, const struct nf_hook_ops *reg,
                           unsigned int n);

Attention

There are some restrictions related to the use of header extraction functions from a struct sk_buff structure set as a parameter in a netfilter hook. While the IP header can be obtained each time using ip_hdr(), the TCP and UDP headers can be obtained with tcp_hdr() and udp_hdr() only for packages that come from inside the system rather than the ones that are received from outside the system. In the latter case, you must manually calculate the header offset in the package:

// For TCP packets (iph->protocol == IPPROTO_TCP)
tcph = (struct tcphdr*)((__u32*)iph + iph->ihl);
// For UDP packets (iph->protocol == IPPROTO_UDP)
udph = (struct udphdr*)((__u32*)iph + iph->ihl);

This code works in all filtering situations, so it’s recommended to use it instead of header access functions.
A usage example for a netfilter hook is shown below:

#include <linux/netfilter.h>
#include <linux/netfilter_ipv4.h>
#include <linux/net.h>
#include <linux/in.h>
#include <linux/skbuff.h>
#include <linux/ip.h>
#include <linux/tcp.h>

static unsigned int my_nf_hookfn(void *priv,
              struct sk_buff *skb,
              const struct nf_hook_state *state)
{
      /* process packet */
      //...

      return NF_ACCEPT;
}

static struct nf_hook_ops my_nfho = {
      .hook        = my_nf_hookfn,
      .hooknum     = NF_INET_LOCAL_OUT,
      .pf          = PF_INET,
      .priority    = NF_IP_PRI_FIRST
};

int __init my_hook_init(void)
{
      return nf_register_net_hook(&init_net, &my_nfho);
}

void __exit my_hook_exit(void)
{
      nf_unregister_net_hook(&init_net, &my_nfho);
}

module_init(my_hook_init);
module_exit(my_hook_exit);

netcat

When developing applications that include networking code, one of the most used tools is netcat. Also nicknamed “Swiss-army knife for TCP / IP”. It allows:

• Initiating TCP connections;
• Waiting for a TCP connection;
• Sending and receiving UDP packets;
• Displaying traffic in hexdump format;
• Run a program after establishing a connection (eg, a shell);
• Set special options in sent packages.

Initiating TCP connections:

nc hostname port

Listening to a TCP port:

nc -l -p port

Sending and receiving UDP packets is done adding the -u command line option.

Note

The command is nc; often netcat is an alias for this command. There are other implementations of the netcat command, some of which have slightly different parameters than the classic implementation. Run man nc or nc -h to check how to use it.

1-2-netfilter

filter.h

#ifndef _FILTER_H_
#define _FILTER_H_

#include <asm/ioctl.h>

/* ioctl command to pass address to filter driver */
#define MY_IOCTL_FILTER_ADDRESS		_IOW('k', 1, unsigned int)

#define MY_MAJOR			42

#endif /* _FILTER_H_ */

filter.c

/*
 * SO2 - Networking Lab (#10)
 *
 * Exercise #1, #2: simple netfilter module
 *
 * Code skeleton.
 */

#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/init.h>
#include <linux/module.h>
#include <asm/uaccess.h>
#include <linux/fs.h>
#include <linux/cdev.h>
#include <asm/atomic.h>
#include <linux/netfilter.h>
#include <linux/netfilter_ipv4.h>
#include <linux/net.h>
#include <linux/in.h>
#include <linux/skbuff.h>
#include <linux/ip.h>
#include <linux/tcp.h>

#include "filter.h"

MODULE_DESCRIPTION("Simple netfilter module");
MODULE_AUTHOR("SO2");
MODULE_LICENSE("GPL");

#define LOG_LEVEL		KERN_ALERT
#define MY_DEVICE		"filter"

static struct cdev my_cdev;
static atomic_t ioctl_set;
static unsigned int ioctl_set_addr;


/* Test ioctl_set_addr if it has been set.
 */
static int test_daddr(unsigned int dst_addr)
{
	int ret = 0;

	/* TODO 2/4: return non-zero if address has been set
	 * *and* matches dst_addr
	 */
	if (atomic_read(&ioctl_set) == 1)
		ret = (ioctl_set_addr == dst_addr);
	else
		ret = 1;

	return ret;
}

/* TODO 1/20: netfilter hook function */
static unsigned int my_nf_hookfn(void *priv,
		struct sk_buff *skb,
		const struct nf_hook_state *state)
{
	/* get IP header */
	struct iphdr *iph = ip_hdr(skb);

	if (iph->protocol == IPPROTO_TCP && test_daddr(iph->daddr)) {
		/* get TCP header */
		struct tcphdr *tcph = tcp_hdr(skb);
		/* test for connection initiating packet */
		if (tcph->syn && !tcph->ack)
			printk(LOG_LEVEL "TCP connection initiated from "
				"%pI4:%u\n",
				&iph->saddr, ntohs(tcph->source));
	}

	/* let the package pass */
	return NF_ACCEPT;
}

static int my_open(struct inode *inode, struct file *file)
{
	return 0;
}

static int my_close(struct inode *inode, struct file *file)
{
	return 0;
}

static long my_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
	switch (cmd) {
	case MY_IOCTL_FILTER_ADDRESS:
		/* TODO 2/4: set filter address from arg */
		if (copy_from_user(&ioctl_set_addr, (void *) arg,
					sizeof(ioctl_set_addr)))
			return -EFAULT;
		atomic_set(&ioctl_set, 1);
		break;

	default:
		return -ENOTTY;
	}

	return 0;
}

static const struct file_operations my_fops = {
	.owner = THIS_MODULE,
	.open = my_open,
	.release = my_close,
	.unlocked_ioctl = my_ioctl
};

/* TODO 1/6: define netfilter hook operations structure */
static struct nf_hook_ops my_nfho = {
	.hook        = my_nf_hookfn,
	.hooknum     = NF_INET_LOCAL_OUT,
	.pf          = PF_INET,
	.priority    = NF_IP_PRI_FIRST
};

int __init my_hook_init(void)
{
	int err;

	/* register filter device */
	err = register_chrdev_region(MKDEV(MY_MAJOR, 0), 1, MY_DEVICE);
	if (err != 0)
		return err;

	atomic_set(&ioctl_set, 0);
	ioctl_set_addr = 0;

	/* init & add device */
	cdev_init(&my_cdev, &my_fops);
	cdev_add(&my_cdev, MKDEV(MY_MAJOR, 0), 1);

	/* TODO 1/3: register netfilter hook */
	err = nf_register_net_hook(&init_net, &my_nfho);
	if (err)
		goto out;

	return 0;

out:
	/* cleanup */
	cdev_del(&my_cdev);
	unregister_chrdev_region(MKDEV(MY_MAJOR, 0), 1);

	return err;
}

void __exit my_hook_exit(void)
{
	/* TODO 1/1: unregister hook */
	nf_unregister_net_hook(&init_net, &my_nfho);

	/* cleanup device */
	cdev_del(&my_cdev);
	unregister_chrdev_region(MKDEV(MY_MAJOR, 0), 1);
}

module_init(my_hook_init);
module_exit(my_hook_exit);

test-1.sh

#!/bin/sh
#
# SO2 - Networking Lab (#10)
#
# Test script for exercise #1
#

# insert module
insmod ../kernel/filter.ko || exit 1

# listen for connections on localhost, port 60000 (run in background)
../../netcat -l -p 60000 &

# wait for netcat to start listening
sleep 1

# connect to localhost, port 60000, starting a connection using local
# port number 600001;
echo "Should show up in filter." | ../../netcat -q 2 127.0.0.1 60000

# look for filter message in dmesg output
echo "Check dmesg output."

# remove module
rmmod filter || exit 1

test-2.sh

#!/bin/sh
#
# SO2 - Networking Lab (#10)
#
# Test script for exercise #2
#

# insert module
insmod ../kernel/filter.ko || exit 1

# set filter IP address to 127.0.0.1
./test 127.0.0.1

# listen for connections on localhost, port 60000 (run in background)
../../netcat -l -p 60000 &

# wait for netcat to start listening
sleep 1

# connect to localhost, port 60000, starting a connection using local
# port number 600001;
echo "Should show up in filter." | ../../netcat -q 2 127.0.0.1 60000

# set filter IP address to 127.0.0.2
./test 127.0.0.2

# listen for connections on localhost, port 60000 (run in background)
../../netcat -l -p 60000 &

# wait for netcat to start listening
sleep 1

# connect to localhost, port 60000, starting a connection using local
# port number 600001;
echo "Should NOT show up in filter." | ../../netcat -q 2 127.0.0.1 60000

# look for filter message in dmesg output
echo "Check dmesg output."

# remove module
rmmod filter || exit 1

/*
 * SO2 - Networking Lab (#11)
 *
 * Test filter module for exercise #2
 *
 * Sends MY_IOCTL_FILTER_ADDRESS to filter module.
 */

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <unistd.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <errno.h>

#include "../kernel/filter.h"

#define MY_DEVICE	"/dev/filter"


static void print_usage(char *argv0)
{
	fprintf(stderr, "Usage: %s <address>\n"
			"\taddress must be a string containing "
			"an IP dotted address\n", argv0);
}

int main(int argc, char **argv)
{
	int fd;
	unsigned int addr;

	if (argc != 2) {
		print_usage(argv[0]);
		exit(EXIT_FAILURE);
	}

	/* get address */
	addr = inet_addr(argv[1]);

	/* make device node */
	if (mknod(MY_DEVICE, 0644 | S_IFCHR, makedev(MY_MAJOR, 0)) < 0) {
		if (errno != EEXIST) {
			perror("mknod " MY_DEVICE);
			exit(EXIT_FAILURE);
		}
	}

	/* open device */
	fd = open(MY_DEVICE, O_RDONLY);
	if (fd < 0) {
		perror("open " MY_DEVICE);
	} else {
		/* send ioctl */
		if (ioctl(fd, MY_IOCTL_FILTER_ADDRESS, &addr) < 0)
			perror("ioctl MY_IOCTL_FILTER_ADDRESS");

		/* close device */
		if (close(fd) < 0)
			perror("close");
	}

	/* cleanup device node */
	if (unlink(MY_DEVICE) < 0)
		perror("unlink " MY_DEVICE);

	return 0;
}

3-4-tcp-sock

tcp_sock.c

/*
 * SO2 - Networking Lab (#10)
 *
 * Exercise #3, #4: simple kernel TCP socket
 *
 * Code skeleton.
 */

#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/net.h>
#include <linux/in.h>
#include <linux/fs.h>
#include <net/sock.h>

MODULE_DESCRIPTION("Simple kernel TCP socket");
MODULE_AUTHOR("SO2");
MODULE_LICENSE("GPL");

#define LOG_LEVEL		KERN_ALERT
#define MY_TCP_PORT		60000
#define LISTEN_BACKLOG		5

#define ON			1
#define OFF			0
#define DEBUG			ON

#if DEBUG == ON
#define LOG(s)					\
	do {					\
		printk(KERN_DEBUG s "\n");	\
	} while (0)
#else
#define LOG(s)					\
	do {} while (0)
#endif

#define print_sock_address(addr)		\
	do {					\
		printk(LOG_LEVEL "connection established to "	\
				"%pI4:%d\n",	 		\
				&addr.sin_addr.s_addr,		\
				ntohs(addr.sin_port));		\
	} while (0)

static struct socket *sock;	/* listening (server) socket */
static struct socket *new_sock;	/* communication socket */

int __init my_tcp_sock_init(void)
{
	int err;
	/* address to bind on */
	struct sockaddr_in addr = {
		.sin_family	= AF_INET,
		.sin_port	= htons(MY_TCP_PORT),
		.sin_addr	= { htonl(INADDR_LOOPBACK) }
	};
	int addrlen = sizeof(addr);
	/* address of peer */
	struct sockaddr_in raddr;

	/* TODO 1/5: create listening socket */
	err = sock_create_kern(&init_net, PF_INET, SOCK_STREAM, IPPROTO_TCP, &sock);
	if (err < 0) {
		printk(LOG_LEVEL "can't create socket\n");
		goto out;
	}

	/* TODO 1/5: bind socket to loopback on port MY_TCP_PORT */
	err = sock->ops->bind(sock, (struct sockaddr *) &addr, addrlen);
	if (err < 0) {
		printk(LOG_LEVEL "can't bind socket\n");
		goto out_release;
	}

	/* TODO 1/5: start listening */
	err = sock->ops->listen(sock, LISTEN_BACKLOG);
	if (err < 0) {
		printk(LOG_LEVEL "can't listen on socket\n");
		goto out_release;
	}

	/* TODO 2/6: create new socket for the accepted connection */
	err = sock_create_lite(PF_INET, SOCK_STREAM, IPPROTO_TCP, &new_sock);
	if (err < 0) {
		printk(LOG_LEVEL "can't create new socket\n");
		goto out;
	}
	new_sock->ops = sock->ops;

	/* TODO 2/5: accept a connection */
	err = sock->ops->accept(sock, new_sock, 0, true);
	if (err < 0) {
		printk(LOG_LEVEL "can't accept new connection\n");
		goto out_release_new_sock;
	}

	/* TODO 2/6: get the address of the peer and print it */
	err = sock->ops->getname(new_sock, (struct sockaddr *) &raddr, 1);
	if (err < 0) {
		printk(LOG_LEVEL "can't find peer name\n");
		goto out_release_new_sock;
	}
	print_sock_address(raddr);

	return 0;

out_release_new_sock:
	/* TODO 2/1: cleanup socket for accepted connection */
	sock_release(new_sock);
out_release:
	/* TODO 1/1: cleanup listening socket */
	sock_release(sock);
out:
	return err;
}

void __exit my_tcp_sock_exit(void)
{
	/* TODO 2/1: cleanup socket for accepted connection */
	sock_release(new_sock);

	/* TODO 1/1: cleanup listening socket */
	sock_release(sock);
}

module_init(my_tcp_sock_init);
module_exit(my_tcp_sock_exit);

test-3.sh

#!/bin/sh
#
# SO2 - Networking Lab (#10)
#
# Test script for exercise #3
#

set -x

# insert module
insmod tcp_sock.ko || exit 1

# list all currently listening servers and active connections
# for both TCP and UDP, and don't resolve hostnames
netstat -tuan

# remove module
rmmod tcp_sock || exit 1

test-4.sh

#!/bin/sh
#
# SO2 - Networking Lab (#10)
#
# Test script for exercise #3
#

set -x

# insert module (run in background, it waits for a connection)
insmod tcp_sock.ko &

# wait for module to start listening
sleep 1

# list all currently listening servers and active connections
# for both TCP and UDP, and don't resolve hostnames
netstat -tuan

# connect to localhost, port 60000, starting a connection using local
# port number 600001;
echo "Should connect." | ../netcat -q 4 127.0.0.1 60000 -p 60001 &

# wait for connection to be established then remove module
# (and close connection)
sleep 3

# remove module
rmmod tcp_sock || exit 1

5-udp-sock

udp_sock.c

/*
 * SO2 - Networking Lab (#10)
 *
 * Bonus: simple kernel UDP socket
 *
 * Code skeleton.
 */

#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/net.h>
#include <linux/in.h>
#include <net/sock.h>

MODULE_DESCRIPTION("Simple kernel UDP socket");
MODULE_AUTHOR("SO2");
MODULE_LICENSE("GPL");

#define LOG_LEVEL		KERN_ALERT
#define MY_UDP_LOCAL_PORT	60000
#define MY_UDP_REMOTE_PORT	60001
#define MY_TEST_MESSAGE		"kernelsocket\n"

#define ON			1
#define OFF			0
#define DEBUG			ON

#if DEBUG == ON
#define LOG(s)					\
	do {					\
		printk(KERN_DEBUG s "\n");	\
	} while (0)
#else
#define LOG(s)					\
	do {} while (0)
#endif

#define print_sock_address(addr)		\
	do {					\
		printk(LOG_LEVEL "connection established to "	\
				NIPQUAD_FMT ":%d\n", 		\
				NIPQUAD(addr.sin_addr.s_addr),	\
				ntohs(addr.sin_port));		\
	} while (0)

static struct socket *sock;	/* UDP server */

/* send datagram */
static int my_udp_msgsend(struct socket *s)
{
	/* address to send to */
	struct sockaddr_in raddr = {
		.sin_family	= AF_INET,
		.sin_port	= htons(MY_UDP_REMOTE_PORT),
		.sin_addr	= { htonl(INADDR_LOOPBACK) }
	};
	int raddrlen = sizeof(raddr);
	/* message */
	struct msghdr msg;
	struct iovec iov;
	char *buffer = MY_TEST_MESSAGE;
	int len = strlen(buffer) + 1;

	/* TODO 1/7: build message */
	iov.iov_base = buffer;
	iov.iov_len = len;
	msg.msg_flags = 0;
	msg.msg_name = &raddr;
	msg.msg_namelen = raddrlen;
	msg.msg_control = NULL;
	msg.msg_controllen = 0;

	/* TODO 1/1: send the message down the socket and return the
	 * error code.
	 */
	return kernel_sendmsg(s, &msg, (struct kvec *) &iov, 1, len);

	return 0;
}

int __init my_udp_sock_init(void)
{
	int err;
	/* address to bind on */
	struct sockaddr_in addr = {
		.sin_family	= AF_INET,
		.sin_port	= htons(MY_UDP_LOCAL_PORT),
		.sin_addr	= { htonl(INADDR_LOOPBACK) }
	};
	int addrlen = sizeof(addr);

	/* TODO 1/5: create UDP socket */
	err = sock_create_kern(&init_net, PF_INET, SOCK_DGRAM, IPPROTO_UDP, &sock);
	if (err < 0) {
		printk(LOG_LEVEL "can't create socket\n");
		goto out;
	}

	/* TODO 1/5: bind socket to loopback on port MY_UDP_LOCAL_PORT */
	err = sock->ops->bind(sock, (struct sockaddr *) &addr, addrlen);
	if (err < 0) {
		printk(LOG_LEVEL "can't bind socket\n");
		goto out_release;
	}

	/* send message */
	err = my_udp_msgsend(sock);
	if (err < 0) {
		printk(LOG_LEVEL "can't send message\n");
		goto out_release;
	}

	return 0;

out_release:
	/* TODO 1/1: release socket */
	sock_release(sock);
out:
	return err;
}

void __exit my_udp_sock_exit(void)
{
	/* TODO 1/1: release socket */
	sock_release(sock);
}

module_init(my_udp_sock_init);
module_exit(my_udp_sock_exit);

test-5.sh

#!/bin/sh
#
# SO2 - Networking Lab (#10)
#
# Test script for bonus exercise
#

set -x

# listen for UDP packets on localhost, port 60001 (run in background)
../netcat -l -u -p 60001 &

# get pid of netcat
pid=$!

# wait for netcat to start listening
sleep 1

# insert module, causing the message to be sent
insmod udp_sock.ko

# remove module
rmmod udp_sock

# kill netcat
kill $pid 2>/dev/null