Traffic shaping with TC in Linux

Traffic shaping is a method to control the rate at which packets are sent and Linux does a pretty good job in doing this. I am currently developing a shell-like interface for TC in python and the following script is the framework for the application. I will update this post as I have more experience with TC. Here is the script with descriptions for which I also used tc manual;

#tc qdisc add dev eth1 root handle 1:0 htb

qdisc (queueing discipline) is elementary to understanding traffic control.
Whenever the kernel needs to send a packet to an interface, it is enqueued
to the qdisc configured for that interface. Immediately afterwards,
the kernel tries to get as many packets as possible from the qdisc,
for giving them to the network adaptor driver. We use classful HTB in this script

#tc class add dev eth1 parent 1: classid 1:1 htb rate 2mbit ceil 2mbit

Classful qdiscs contain classes which allows us to set rate limits. Class 1:1 is the root class here

#tc class add dev eth1 parent 1:1 classid 1:10 htb rate 512kbit ceil 2mbit
#tc class add dev eth1 parent 1:1 classid 1:11 htb rate 1mbit ceil 2mbit
#tc class add dev eth1 parent 1:1 classid 1:12 htb rate 512kbit ceil 2mbit

This is the leaf class 1:10 attached to the root class 1:1.
Shaping can be done in the leaf class not in the root class.
In this example of leaf classes, we allocate 512kbit,1mbit
and 512kbit of bandwidth to each class respectively.
rate: guaranteed (minimum bandwidth) which can be exceeded
ceil: burstable (maximum bandwidth) which cannot be exceeded

#tc filter add dev eth1 protocol ip parent 1:0 prio 1 handle 2 fw classid 1:10
#tc filteradd dev eth1 protocol ip parent 1:0 prio 1 handle 3 fw classid 1:11
#tc filter add dev eth1 protocol ip parent 1:0 prio 1 handle 4 fw classid 1:12

We use filters to classify the packets into an output queue and attached it to our classful htb class.
What handle means will be meaningful in the rest of the post

#tc qdisc add dev eth1 parent 1:10 handle 10: sfq quantum 1500b perturb 10
#tc qdisc add dev eth1 parent 1:11 handle 11: sfq quantum 1500b perturb 10
#tc qdisc add dev eth1 parent 1:12 handle 12: sfq quantum 1500b perturb 10

SFQ (Stochastic Fairness Queueing) is added to each of the HTB
queues and it offers equality for limiting traffic while keeping
the overhead low. I have tested a setup without SFQ and it gave
frustrating results (e.g link rates go up and down)

quantum: Amount of bytes a flow is allowed to
dequeue during a round of the round robin process. Defaults to
the MTU of the interface which is also the advised value and
the minimum value. This is what manual says but I have seen
that default value is assigned as 814. So maybe it is better
to set it to 1500

perturb: Interval in seconds for queue algorithm perturbation.
Defaults to 0, which means that no perturbation occurs. Do not
set too low for each perturbation may cause some packet
reordering. Advised value: 10

#iptables -t mangle -A POSTROUTING -s 192.168.111.2 -j MARK --set-mark 2
#iptables -t mangle -A POSTROUTING -d 192.168.111.2 -j MARK --set-mark 2
#iptables -t  mangle -A POSTROUTING -s 192.168.111.3 -j MARK --set-mark 3
#iptables -t  mangle -A POSTROUTING -d 192.168.111.3 -j MARK --set-mark 3
#iptables -t mangle -A POSTROUTING -s 192.168.111.4 -j MARK --set-mark 4
#iptables -t mangle -A POSTROUTING -d 192.168.111.4 -j MARK --set-mark 4

By using iptables we mark packets in order for TC to classify them. Those marking numbers are used in handle parameter of tc filter 

In this script we direct hosts having IP addresses 192.168.111.2,3,4 into different classes each of which has different rate. Important thing is that eth1 interface is the inner interface facing these hosts.  For example if host 192.168.111.2 is downloading a file from a remote host, eth1 becomes an egress interface for the packets coming from the remote host. If we want to limit what these hosts send another qdisc for eth0 (outside interface) and root,child classes must be added.

I still have time to complete my shell-like interface for TC. I don’t have much experience in the field with TC except some tests in several ISPs. I welcome any opinion, feedback.

References:

http://tldp.org/HOWTO/html_single/Traffic-Control-HOWTO