======================
VNETLAB PROJECT README
======================


----------------------
1. DIRECTORY STRUCTURE
----------------------

bin/    This directory would hold the binaries post compilation.

conf/   The conf directory contains all the configuration files.
conf/conf_examples
	This directory provides configuration examples that serve two purposes:
	first, conf files can be created easily by modifying the examples. group/
	host/ and /user-specified-nw-conf subdirs are of this type. Second, 
	grpMgr-gen-conf/ subdir contains automatically generated conf files for a
	virtual network "qrnet335". These conf files are transparent to end user 
	but give an idea how vnetlab works.
	
doc/    The doc directory contains the previous student reports and the papers
        published as a research outcome of this project.
doc/components-interface
	This directory contains interface documents for each vnetlab component:
	group-manager, host-manager, kernel module and resource-allocator
doc/resource allocator
	This direcotry contains reference and reports for the resource-allocator
	component of vnetlab.

src/    The src directory contains all the source files and the script files.
src/allocator/	   
	contains files for resource-allocator
src/common/	   
	contains files for common headers and utilities that are used by other 
	directories 
src/compiler/	   
	contains files for a compiler which consumes users' virtual network 
	configuration file
src/hostMgr/	   
	contains files for host-manager
src/kernelModule/
	contains files for kernel module
src/scripts/
	contains files for scripts
src/server/
	contains files for group-manager
src/userInterface/
	contains files for user interface programs, specifically vnet-manager and 
	group-admin
src/vix/
	contains files for working with VMware VIX API
src/webInterface/
	contains files for the web interface for users

test/
	This directory to contains test cases for resource-allocator


--------------
2. COMPILATION
--------------

Run 'make' to build the project and to get the required binaries. After the
compilation the binaries will be placed under the directory bin/. The binaries
that would be generated are vnet-manager, group-manager, host-manager,
group-admin.

To compile a specific component, run 'make' in the corresponding source directory.

-------------------------------
3. INSTALLATION & CONFIGURATION
-------------------------------

3.1 Server Setup For V-NetLab

V-NetLab project is setup on blade servers installed with GNU/Linux operating system. Our current
setup has Ubuntu 8.04 LTS server edition installed in all the server machines. The basic setup constitute
the following machines,

(0) Gateway Server (Master) - One
(1) NFS Server - One
(2) Worker Servers (Slaves) - One or more

3.1.1 Gateway Server

The Gateway server plays two different roles in our setup. In addition to acting as the gateway to our
setup, it also acts as the group-manager (refer the publication). This is the only machine in our setup
which can be accessed from the external world. So ideally this machine should have two NIC cards
with one of the them connected to the external world and the second NIC connected to other machines
(NFS and worker servers) of the V-NetLab setup.

All the users who want to use V-NetLab should have an user account in this server and the UNIX group
is being used to setup teams, if required. Teams here are used when a set of people want to share a
network. In addition to all these users, one default user must be present in this server by the username
'vnetlab'. We have exposed this user to all the Worker Servers through NIS.

3.1.2 NFS Server

All the servers in the V-NetLab setup mount their home directories (/home) from this NFS server. The
primary reason is that while starting a new network, the group-manager creates few configuration files
and saves under the user vnetlab's home directory. These files should be accessible to all the worker
servers who will be involved in hosting virtual machines for that particular network. Instead of
transferring these configuration files from the group-manager to the worker machines, we use the
mounted home directory to share these files.

3.1.3 Worker Servers

The worker servers are the slave machines which will host the required virtual machines for the virtual
networks. Each of these servers should have the kernel module 'kmt' loaded in them and should run the
binary host-manager. The kernel module is used for the packet rewriting purposes and to provide
isolation among the virtual networks. The binary group-manager running in the Gateway server will
talk with the host-manager running these worker server to send commands while starting and stopping
networks.

As said before, all the worker servers should have the user vnetlab available through the NIS service
running in the Gateway server. Ideally these worker machines will not need any other user accounts
other than vnetlab and a superuser account. The home directory for the user vnetlab should also be
mounted from the above mentioned NFS server.

Since these worker servers will host the virtual machines, it should to have VMware Server 2.0.x
installed in them. VMware Server is a freely (as in cost) available software and can be downloaded
from the VMware website. While installing the VMware Server, the user vnetlab should be mentioned
as the administrator and no networking setup is required. As all the required networking setup is done
dynamically while starting and stopping the network, the administrator doesn't have to care about those
while installing the VMware server.

Once the VMware Server installation is complete, login to the VMware Server web console
(https://worker_server:8333/ui/) with the credentials for the user vnetlab. Through the web console,
create two separate Datastores with the following details:
Name: VMLibrary
Directory Path: /src/VMLibrary
Name: VMWorkspace
Directory Path: /srv/VMWorkspace

3.2 V-NetLab Binary and Configuration Files

Once the NFS server's have been setup, following directories *need* to be created which hold the
configuration information used by V-NetLab.

The /home/vnetlab/ directory consists of :
bin/
conf/
managed/

The contents of each directory are explained below:
a. /home/vnetlab/bin
This directory is created by copying the following files from VNETLAB_RELEASE/bin after compilation.
group-manager // the group manager executable
host-manager // the host manager executable which manages each host
daemon // the resource allocator daemon
vnet-manager // the user utility


b. /home/vnetlab/conf
This directory is very important and consist of two directories:

1. /home/vnetlab/conf/group
This directory is created by copying and modifying related files from
VNETLAB_RELEASE/conf/conf_examples/group directory. It consists of information related to vnetlab 
groups which is used by group manager.

adminVsGroupID.txt : Consist of the administrator for each group. File format is :
<admin user id>:<group id>
0:408
where the admin user id would normally be 0 (refers to root user) and the group id is the group to
which the users belong to.

groupsConfig.txt : Specifies which groups are managed by which group manager.
# group-name:group-manager-host-name:group-manager-host-ip:port:static-allocation:migration
408:testfw:192.168.2.1:7777:yes:no

In the above example, the file states that group 408 is manager by group manager which runs
on the host "testfw" with ip address "192.168.2.1" at port 7777 and it supports static allocation
but no migration of networks.

usersVsGroupID.txt : Specifies which users belong to which groups
Format <user id>:<group id>
514:508
515:408

Above example states that user 514 belongs to group 508 and user 514 to group 408.

c. /home/vnetlab/managed
This is the most critical directory containing the Vnetlab management information.
1 ./home/vnetlab/managed/common
This directory is created by copying and modifying related files from
VNETLAB_RELEASE/conf/conf_examples/host directory. 

This directory contains a file which lists all the hosts to be managed by a group manager. An
example of the file is listed below: each host should be identified with hostname that matches its dns
name in /etc/hosts, should specify its IP address and MAC address, and information about cpu, memory
and disk.

host v3 {
cpu: 3.0
mem: 3000
disk: 80.0
root: "/srv/vnetlab"
ip: "192.168.2.3"
mac: "00:0f:1f:67:bf:28"
}
host v4 {
cpu: 3.0
mem: 3000
disk: 80.0
root: "/srv/vnetlab"
ip: "192.168.2.4"
mac: "00:0f:1f:67:bf:54"
}

2. /home/vnetlab/managed/scripts
This directory is created by copying and modifying related files from VNETLAB_RELEASE/src/scripts directory. 
It holds all the scripts used by VNetLab.

Edit the file /home/vnetlab/managed/scripts/vmwareserver2_vixscript.sh and update the password field
(VM_PASS & GU_PASS) with the password that had been set while setting up the VMWare Server.

Edit the file /home/vnetlab/managed/scripts/restart-host-manager.sh file and update the line where the
insmod command is issued with the correct MAC address of the machine which would act as the
group-manager.

3. /home/vnetlab/managed/teamNetworks/ : When V-Netlab is used to register and run virtual
networks, this directory will hold all the information about the running and stopped team networks.
NOTE : No need to add files to this directory, these files are generated automatically.

For each team (team is nothing but a group), a directory with team id will be created. Example, if
team 786 has registered a network vnetwork1, then following directory will be created.
/managed/teamNetworks/786/vnetwork1
If vnetwork1 has virtual machines on hosts v3 and v4, following directories and files will be created
when the network is started.
netConfigInfo : Script to configure the virtual machines as specified by user
.uid : The user id of the user who last started the network
v3 : Directory containing configuration information for all VMs on host v3
v4 : Directory containing configuration information for all VMs on host v4
vmSrc.map : Specifies which virtual machine is on which host

Within directory for each host, following files are maintained.
host1.cfg : The vmware configuration file for virtual machine host1
host2.cfg : The vmware configuration file for virtual machine host2
port2vm.map : Specifies port forwarding information from physical to virtual machine.
pt.map : The participation table (sibling closure) for this virtual network
vinf.map : Virtual ethernet switch configuration information used by Vnetlab
vm2hub.map : Specifies which VM is connected to which Virtual Ethernet Switch
vm.loc : Specifies the actual location of the virtual machine on physical host

4. /home/vnetlab/managed/vmSrc : Add images of virtual machines to this directory. 
We currently support Linux Je0S Virtual Machine Image. 

5. VNETLAB_RELEASE/src/kernelmodule/tools: We need to apply a patch to generate a script file. 

Make sure you have installed the vmserver2.
Then copy the vmware-config.pl (normally in /usr/bin directory)
to vmware_vnctl and vmware_config directories.

Then cd to vmware_vnctl and apply the following command:

patch -p1 <../vnctl.patch

Then you get the management tool vnconf.pl that V-Netlab will use.


6. /home/vnetlab/managed/tools/obj/ : After make, Copy the kernel module 
VNETLAB_RELEASE/src/kernelmodule/kmt.ko into this folder. And copy the files under 
VNETLAB_RELEASE/src/kernelmodule/tools into /home/vnetlab/managed/tools folder.


3.3 Web Sign Up
web sign up component needs a web service installed on the GATEWAY server. In this guidance, we use apache as the example

Please follow the steps below:
1. copy all the web files on /vnetROOT/src/webInterface/signup/www into /var/www
2. copy the whole directory from /vnetROOT/src/webInterface/signup/vnetlab into /var
3. chown www-data.www-data /var/vnetlab 
4. chmod 700 /var/vnetlab

*Note: the reason for step 3 and 4 is because /var/vnetlab contains some important system data that should not be leaked out to normal users



--------
4. USAGE
--------

4.1 Start up the servers

Before a user can start networks, the admin has to start the group manager, the host manager, and the resource
allocator daemon.

4.1.1 group-manager

group-manager serves as the server in V-Netlab. It runs on the gateway and handles the user requests and transfer
them to host-managers

To start the group manager, chdir to /home/vnetlab/bin on gateway, and issue the following command:

	./group-manager [-h PATH_TO_HOST] [-l LOG_NUM] [-d]

Note: 
PATH_TO_HOST: the path to the physical host config file which indicates the resource capacity of each host. An example of
this file can be found at VNETLAB_RELEASE/conf/conf_examples/host/test.hosts
LOG_NUM: the level of log output to the stdout
DEBUG: -d option indicates whether invoke group-manager in debug mode where group-manager could be start without host-mangers

MANAGEMENT SCRIPT: copy-restart-grpmgr.sh (development); restart-group-manager.sh (release)

4.1.2 host-manager

host-manager serves as the agent in each physical host that performs the operations to the vms. To start host-manager, on each
worker server, chdir to /home/vnetlab/bin and issue the following command:

	./host-manager GATEWAY $HOSTMGR_IP

Note:
GATEWAY: gateway is the ip address from which host-manager can connect to group-manager. Normally it should be the internal 
private address of the gateway
HOSTMGR_IP: is the ip address that host-manager will bind to. 

MANAGEMENT SCRIPT: copy-restart-hostmgr.sh(development); restart-host-manager.sh(release)

4.1.3 resource allocator

resource allocator decides the location(s) of the vms generated from the user request. To start resource allocator, chdir to 
/home/vnetlab/bin on gateway and issue the following command:

	./daemon

Note:
resource allocator should be running with group-manager in the SAME MACHINE, which is the gateway.

4.1.4 kernel module

kernel module is the component that performs the 2-layer network isolation/virtualization. To load the kernel module, on each
worker server, issue the following command:

	insmod /path/to/kernel_module   gw_ip=GATEWAY str_gw_mac="xx","xx","xx","xx","xx","xx"
	rmmod  kmt

Note:
when insmod, gw_ip and str_gw_mac refer to gateway ip and gateway mac address
when doing rmmod, probably it fails if there are some reference counts to it. This possibility mainly come from: 
vmnet-netifup, host-manager, thus kill all of them before you rmmod.

4.2 Use vnet-manager to manage virtual networks

vnet-manager is the front end tool for user/administrator to start requests. After the servers are all started up,
vnet-manager can be used to manage user virtual networks.

To use vnet-manager, the user should first login to gateway:
ssh user@gateway
-X option is needed if the user wants vnc connection. 
Users' accounts on the gateway can be set up during the sign up phase.

Usage for vnetmgr is shown as follows:

  vnetmgr (v1.0) - Command Line Interface for VNetLab

SYNOPSIS
 user mode: allows users to manage networks belonging to them
    vnetmgr [create | destroy | shudown | start | suspend | resume | query | restart] [OPTIONS...]

 admin mode: provides admin the full view of system resource allocation 
    vnetmgr

DESCRIPTION
 vnetmgr is the command line interface for Virtual Network Lab
 It allows users to create/destroy/start/shutdown/suspend/resume networks
 Each of the commands to vnetmgr take some arguments.

 CREATE: Allows users to create a virtual network
 USAGE:
   vnetmgr create -c <config-file-path> -n <network-name> [-t <team-id>]
   where,
      -c  specifies path to user's config file. An example of the file can be found at
	  VNETLAB_RELEASE/conf/conf_examples/user-specified-nw-conf/qr10.cfg
      -n  specifies unique network name
      -t  specifies the team id if user wants to share the network with other team members

 DESTROY: Allows users to destroy a running network
 USAGE:
    vnetmgr destroy -n <network-name>
    where,
      -n  specifies name of already running network

 START: Allows users to start a virtual network
 USAGE:
   vnetmgr start -n <network-name>
   where,
      -n  specifies unique network name

 SHUTDOWN: Allows users to shutdown a running network
 USAGE:
    vnetmgr shutdown -n <network-name>
    where,
      -n  specifies name of already running network

 SUSPEND: Allows users to suspend a running network
 USAGE:
    vnetmgr suspend -n <network-name>
    where,
      -n  specifies name of already running network

 RESUME: Allows users to resume a suspended network
 USAGE:
    vnetmgr resume -n <network-name>
    where,
      -n  specifies name of suspended network

 RESTART: Allows users to restart invidual VMs of a network
 USAGE:
    vnetmgr restart -n <network-name> -v vm1...vmN
    where,
      -n  specifies name of already running network
      -v  specifies name of a VM(s) vm1 through vmN

 QUERY: Allows users to query all networks belonging to them
 USAGE:
    vnetmgr query

4.3 Web based user sign up

All the user sign up data is in the /var/vnetlab directory, along with them are some scripts that the administrator should use manage user sign up data.

*Note: Before starting to use the system, MAKE SURE this directory can ONLY be read from root

4.3.1 user sign up data

There are 3 types of files for user sign up. 
XXX.txt: e.g.:509.txt contains all the user data for course cse 509
	data format:
	name ID

courses: contains all the course information
	data format:
	courseID1
	courseID2
	......

password: contains the password for web based user to sign up. The passwd is plaintext


4.3.2 User singup Management scipts

The following are the management scripts for administrator:

foreground_courseMgr.pl
=======================
this script is to help administrator to create courese for users(students) to signup from the web interface.
Usage:
        ./foreground_courseMgr.pl [add|del] [courseID]
        ./foreground_courseMgr.pl list

NOTE: list option will display all the courses that are OPEN to the web users


backgroundd_courseMgr.pl
========================
this script is to help create courses along with a administrator for V-NetLab system
Usage: 
        ./background_courseMgr.pl [add|del] [courseID]
        ./background_courseMgr.pl list

Note:
1) add option will "create" a course for V-Netlab AND create a Unix group for this course. The name will be cse + courseID. ie. cse509
2) list option will list all the courses that exists in V-NetLab


registerUsers.pl
===============
This script is to help create local user accounts for each students that has signed up from the web
Usage:
        ./registerUsers.pl [add|del] [courseID]

Note: add option will create users that are in the courseID.txt (ie. 509.txt) by using the SIGNUP NAME as the account name and ID as the password.



4.4 Use group-admin to manage teams and users

NAME
 group-admin (v1.0) - Group Administrator utility for VNetLab

SYNOPSIS
 group-admin [add | del | mod | query] [OPTIONS...]

DESCRIPTION
 group-admin is the group administrator utility for Virtual Network Lab
 It allows the administrator to add/delete/modifiy users (or) teams.
 It also provides a query interface to the admin to check networks, etc
 The options to Group admin utility are as follows:

 ADD: Allows administrator to add a user (or) team
 USAGE:
    group-admin add -u <user> -p <passwd>
    group-admin add -t <team>
    where,
         -u  username to be added
         -p  password of the user to be added
         -t  team-name to be added

 DELETE: Allows administrator to delete a user (or) team
 USAGE:
    group-admin del -u <user>
    group-admin del -t <team>
    where,
         -u  username to be deleted
         -t  teamname to be deleted

 MODIFY: Allows administrator to add/remove a user from a group
 USAGE:
    group-admin mod -t <team> -a <user>
    group-admin mod -t <team> -r <user>
    where,
         -t  teamname to be modified
         -a  username to be added in the team
         -r  username to be removed from the team

 QUERY: Allows administrator to check users/teams/hosts/networks
 USAGE:
    group-admin query -s [allusers | allteams | allhosts | allnetworks]
    where,
         -s  specifies the type of query
