The DTN (Data Transfer Node) Focus Group has performed several tests on GTS (GÉANT Testbed Service) in order to get useful results that the NRENs can compare and replicate for their own tests. The following matrix shows a summary of the set-up or each test, the parameters tuned, the software installed and the performance achieved, as well as the links to the information on how to install each software in GTS. Finally, some comments related to the setup and the test are included.

The DTN Focus Group has performed several tests on GTS in order to get useful results that the NRENs can compare and replicate for their own tests using up to four Bare Metal Servers (BMS) with two different setups. Both BMS setups were connected directly with 10Gbps links. It has also produced examples of tests and setups in Bare Metal Server (BMS) and virtualised environments using both VMs (provided as setup from the GTS testbed administration page) and Dockers.

More test details:

In the first investigation, DTNs on virtualised environments using VMs were tested. For the direct measurement of the links between the client site and the server, ipref was used. The results in virtualised environments showed that all programs evaluated achieve high bandwidth utilization except XRootd. The reason for not achieving high bandwidth utilization was the virtualisation itself. Xrootd is highly dependant on hardware resources and the virtualised environment with the virtual hypervisor on the physical system and the lack of direct access of Xrootd to resources (i.e. the physical link, not direct access to disks and buffer memory) had as result the low bandwidth utilization. On the other hand, the rest of the DTN services and client programs investigated (i.e. FDT, gridFTP) achieved higher bandwidth utilization. This first experiment was executed in virtual servers with different distances between the client and the server. In this case, both VMs were in Amsterdam for the shortest distance tests and one VM was in Amsterdam and the other one in London for a the longest distance test.Additionally, our investigation shown that in VM testing better results are achieved with VM with 2xCPU than 4xCPU. The aforesaid differences in results are due to different reasons. More specifically, it is due to different server architecture, Hyper-V VM Load Balancing, or that DTN tools do not know "how to" better use the CPU resources. It is important to note that this investigation did  not have any access to change VM parameters.

As mentioned before, the results may be affected by the fact that the links are not dedicated and other tests could have been running in GTS simultaneously. This could explain why the tests run between Amsterdam and London with 4 CPUs offered worse results than all the rest, as they were run in different days. When doing the tests, they were repeated at different time slots, to see if a pattern could be found regarding performance, but no significant changes were found.

¹: AMS: Amsterdam; LON: London; BMS: Bare Metal Server; VM: Virtual Machine

At the next investigation, the examination of DTN services (same as above) focused on the usage of physical Bare Metal Serves (BMS). At the beginning, the performance of the DTN services and applications at BMS was examined at a short distance (Hamburg-Paris), with the latest technology servers available in GTS (Dell R430). The results for all the DTN services and applications were good an all the services performed well in terms of transferring huge amounts of data among them (as usual, more than 250GB). Also, hardware was not a bottleneck at the process.

Afterwards, longer distance data transfers were also investigated over the GTS test bed between London and Prague with the use of BMS. The London and Prague nodes, with longest optical distance and BMS availability, were used with a combination of Dell R430 and Dell 520 hardware. Therefore, the available system was not the same as for the previous tests, it was used a Dell R520 that was acting as a client with a Dell R430 that was acting as a server. The Dell 520  is weaker in terms of CPU and Memory than the R430 that was acting as client in the previous tests. The results of the test  (as shown in the results table)  showed that the bandwidth utilization dropped for all the DTN services and applications, highlighting the importance of the hardware to get the best results, which also highlight the importance of the hardware for data transfer. An important note that needs to be mentioned is that during the test run with Dell R520 Geant GTS team were implementing hardware replacements and made additional network configurations so it also contributed to poorer results.

Dockerised Tests

From the ctop results we can see that each container required more memory and cpu than the docker already assigned to it. This is due to the requirements of the DTN services.

For Dockerised environments, the best data rates were achieved also with FDT, reaching 9.2 Gbps, gridFTP followed with 8.6 Gb/s and at the end XrootD with 8.1 Gb/s. As well as for the regular tests, all services need the servers to have DISKs RAIDS, that are as fast as 6 GB/s in order to achieve the required speed. Also, there is a large demand in memory for the packet transfers to be saved in the memory buffer and then at the disks.

Comparing the results achieved with and without Docker, with a different number of CPUs in a VMs and with long and short distances, shown in the tables above, some comments to summarise them are: