Software
Now virtual server program is now implemented based on Linux IP masquerading codes, and some of Steven Clarke's port
forwarding codes are reused.
The virtual server patch for 2.2 is under beta test now, it
has more features and is becoming more and more stable.
The virtual server patch for kernel 2.2
The virtual server patch v0.8 for kernel 2.2.10 (ipvs-0.8-2.2.12.tar.gz) was
released on September 1, 1999. Changes are as follows:
- Added the persistent port feature.
Users can specify whether the virtual service port is
persistent or not. It is more flexible. The original PCC
scheduling is removed.
- Added the dest server status checking.
The server status is checked before forwording a packet.
If the server is not available(down or put out of
service), the packet will be dropped and the client will
be notified immediately. The server status is also
checked while generating a masq entry based on the masq
template. If not available, the new entry won't be
created.
- Added some code in ip_masq_ftp.c to handle virtual FTP
service for VS-NAT. The passive handling code in
ip_masq_ftp.c never works.
- Fixed stepping to mSR after SYN in INPUT_ONLY table.
Thank Julian Anastasov for doing it. It make much much
harder that a LinuxDirector is synflooded to run out of
memory.
- Fixed huge masq expire bug for after bad checksum.
Thank Julian Anastasov for fixing it.
- Added the IP_MASQ_F_VS_INACTIVE flag and fixed the
connection counter
Thank Julian Anastasov for the suggestion and fix
example.
- Fixed the incorrect lookup in hash table.
The ms=NULL statement was forgot if no entry is found,
this makes the incorrect lookup, which may lead to huge
masq expire. Stupid mistake, but the result is serious.
- Fixed the incorrect slow timer vector layout
Correct layout and more efficient to use memory.
- Fixed the bug of slow timer being added twice for masq
template.
The virtual server patch v0.7 for kernel 2.2.10 (ipvs-0.7-2.2.10.tar.gz) was
released on July 9, 1999. Changes are as follows:
- Added a separate masq hash table for IPVS.
- Added slow timers to expire masq entries.
Slow timers are checked in one second by default. Most
overhead of cascading timers is avoided.
With this new hash table and slow timers, the system can
hold huge number of masq entries, but make sure that you
have enough free memory. One masq entry costs 128 bytes
memory effectively (Thank Alan Cox), if your box holds 1
million masq entries (it means that your box can receive
2000 connections per second if masq expire time is 500
seconds in average.), make sure that you have 128M free
memory. And, thank Alan for suggesting the early random
drop algorithm for masq entries that prevents the system
from running out of memory, I will design and implement
this feature in the near future.
- Fixed the unlocking bug in the ip_vs_del_dest().
Thank Ted Pavlic <tpavlic@netwalk.com> for
reporting it.
The virtual server patch v0.6 for kernel 2.2.10 (ipvs-0.6-2.2.10.tar.gz) was
released on July 1, 1999. Fixed the overflow bug in the
ip_vs_procinfo(). Thank Ted Pavlic
<tpavlic_list@netwalk.com> for reporting it. Added the
functionality to change weight and forwarding (dispatching)
method of existing real server. Added the functionality to change
scheduler of virtual service on the fly. Reorganized some code
and changed names of some functions.
The virtual server patch v0.5 for kernel 2.2.9 (ipvs-0.5-2.2.9.tar.gz) was
released on June 22, 1999. Fix the bug that LocalNode doesn't
work in vs0.4-2.2.9. Thank Changwon Kim
<chwkim@samsung.co.kr> for reporting the bug and pointing
me the checksum update problem in the code. Some code in the
ip_fw_demasquerade was reorganized for efficiency.
The virtual server patch v0.4 for kernel 2.2.9 (ipvs-0.4-2.2.9.tar.gz) was
released on June 1, 1999. Most of the code was rewritten. The
locking and refcnt was changed. The violation of "no floats
in kernel mode" rule in the weighted least-connection
scheduling was fixed. This patch is more efficient, and should be
more stable.
Peter provided the ipvs-0.2-2.2.9.tar.gz,
it was for test purpose. Peter ported the VS patch to kernel 2.2,
rewrote the code and loadable scheduling modules. Thank Peter!
Please report bugs to the Linux Virtual Server mailing list
<linux-virtualserver@iinchina.net>,
thank you!
The virtual server patch for kernel 2.0
The latest archive is the virtual server patch version 0.9 vs-0.9.tar.gz. The ChangeLog is as
follows:
- Add Virtual Server via Direct Routing
This approach was first implemented in IBM's
NetDispatcher. All real servers have their loopback alias
interface configured with the virtual IP address, the
load balancer and the real servers must have one of their
interfaces physically linked by a HUB/Switch. When the
packets destined for the virtual IP address arrives, the
load balnacer directly route them to the real servers,
the real servers processing the requests and return the
reply packets directly to the clients. Compared to the
virtual server via IP tunneling approach, this approach
doesn't have tunneling overhead(In fact, this overhead is
minimal in most situations), but requires that one of the
load balancer's interfaces and the real servers'
interfaces must be in physical segment.
- Add more satistics information
The active connection counter and the total connection
counter of each real server were added for all the
scheduling algorithms.
- Add resetting(zeroing) counters
The total connection counters of all real servers can be
reset to zero.
- Change some statements in the masq_expire function and
the ip_fw_demasquerade function, so that
ip_masq_free_ports won't become abnormal number after the
masquerading entries for virtual server are released.
- Fix the bug of "double unlock on device queue"
Remove the unnecessary function call of
skb_device_unlock(skb) in the ip_pfvs_encapsule function,
which sometimes cause "kernel: double unlock on
device queue" waring in the virtual server via
tunneling.
- Many functions of virtual server patch was splitted into
the linux/net/ipv4/ip_masq_pfvs.c.
- Upgrade ippfvsadm 1.0.2 to ippfvsadm 1.0.3
Zeroing counters is supported in the new version. The
ippfvsadm 1.0.3 can be used for all kernel with different
virtual server options without rebuilding the program.
The virtual server patch version 0.8 vs-0.8.tar.gz
was changed as follows:
- Add virtual FTP server support
The original ippfvs via IP tunneling could not be used to
build a virtual FTP server, because the real servers
could not establish data connections to clients. The code
was added to parse the port number in the ftp control
data and create the corresponding masquerading entry for
the coming data connection.
Although the original ippfvs via NAT could be used to
build a virtual server, the data connection was
established in
this way.
Real Server port:20 ----> ippfvs: allocate a free masq
port -----> the client port
It is not elegent but time-consuming. Now it was changed
as follows:
Real Server port:20 ----> ippfvs port: 20 ----> the
client port
- Change the port checking order in the
ip_fw_demasquerade()
If the size of masquerade hash table is well chosen,
checking a masquerading entry in the hash table will just
require one
hit. It is much efficient than checking port for virtual
services, and there are at least 3 incoming packets for
each connection, which require port checking. So, it is
efficient to check the masquerading hash table first and
then check
port for virtual services.
- Remove a useless statement in the ip_masq_new_pfvs()
The useless statement in the ip_masq_new_pfvs function is
ip_masq_free_ports[masq_proto_num(proto)]++; which may
disturb system.
- Change the header printing of the ip_pfvs_procinfo()
Matthew Kellett
used the virtual server patch in his Zaphod project (a
rack-mounted multiple computer web server) in the Corel Computer
Corpration, and did his modification on the vs patch. Here is his
patch matthewpatch-0.5.zip,
which is against the vs-0.5 patch. He added the load-balancing
module and the load-informed scheduling. It will compeletly
change the infrastructure of current vs patch and will require a
lot of coding to make it fit with the vs-0.9 if it is merged into
the vs-0.9 patch, and which may make the current vs patch
unstable. So, his patch won't be included in the current patch,
however the idea of flexible load-balancing module and maybe part
of his patch will be used in the new next-stage development of
the virtual server.
The virtual server patch version 0.7 vs-0.7.tar.gz
was changed as follows:
- Fix a bug in detect the finish of connection for
tunneling or NATing to the local node. Since the server
reply the client directly in tunneling or NATing to the
local node, the load balancer (LinuxDirector) can only
detect a FIN segment. It is mistake that the masq entry
is removed only if both-side FIN segments are detected,
and thus every masq entry has to expire in 15 minutes.
For the situation above, the code was changed to set the
masq entry expire in TCP_FIN_TIMEOUT (2min) when an
incoming FIN segment is detecting.
- Add the patch version printing in the ip_pfvs_procinfo()
It would be easy for users and hackers to know which
virtual server patch version they are running. Thank
Peter Kese <peter.kese@ijs.si> for the suggestion.
The virtual server patch version 0.6 vs-0.6.tar.gz
was changed as follows:
- Add the local node feature in virtual server.
If the local node feature is enabled, the load balancer
can not only redirect the packets of the specified port
to the other servers (remote nodes) to process it, but
also can process the packets locally (local node). Which
node is chosen depends on the scheduling algorithms.
Thank Peter Kese <peter.kese@ijs.si> for the idea
of "Two node Virtual Server" and his single
line patch for virtual server via IP tunneling.
- Remove a useless function call ip_send_check in the
virtual server code.
The virtual server patch version 0.5 vs-0.5.tar.gz
was changed as follows:
- Add feature to enable IP tunneling on virtual server.
If the ippfvs is enabled using IP tunneling, the load
balancer chooses a real server from a cluster based on a
scheduling algorithm, encapsules the packet and forwards
it to the chosen server. All real servers are configured
with "ifconfig tunl0 <Virtual IP Address>
up". When the chosen server receives the encapsuled
packet, it decapsules the packet, processes the request
and returns the reply packets directly to the client
without passing the load balancer. This can greatly
increase the scalability of virtual server.
- Fix a bug in the ip_portfw_del() for the weighted RR
scheduling.
The bug in version 0.4 is when the weighted round-robin
scheduling is used, deleting the last rule for a virtual
server will report "setsockopt failed: Invalid
argument" warning, in fact the last rule is deleted
but the gen_scheduling_seq() works on a null list and
causes that warning.
Old archives:
The archive is virtual server patch version 0.4 vs-0.4.tar.gz, in which a bug of memory
access error was fixed. The set_serverpointer_null() function is
added to scan all the existing ip masquerading records for its
server pointer which points to the server specified and set it
null. It is useful when administrators delete a real server or
all real servers, those pointers pointing to the server must be
set null. Otherwise, decreasing the connection counter of the
server may cause memory access error when the connection
terminates or timeout.
The virtual server patch version 0.3 vs-0.3.tar.gz
is for the Linux kernel 2.0.35. Three scheduling algorithms, the
weighted round-robin scheduling, the least-connection scheduling
and the weighted least-connection scheduling, are implemented in
this patch. You can also see the ChangeLog in the archive to
check the new features.
There are two virtual server program patch for Linux 2.0.33.
One is vs-r-2.0.34.tar.gz, which
uses round-robin scheduling algorithm; it is a little bit more
efficient in scheduling when all real servers have the same
processing capacity. The other is vs-w-2.0.34.tar.gz,
which uses the weighted round-robin scheduling algorithm; it is
better when the real servers have different processing capacity,
actually this one should be called the weighted least-connection
scheduling, the concept was clarified in the version 0.3.
Virtual server administration program ippfvsadm.c allows you
to configure the virtual server, the program is available at the
packed archives. ipvsadmcgi.c allows
you to administrate the virtual server via browsers.
Last updated: 1999/9/1
Created on: 1998/5/28