Dear Zeyu,

Each "tar.gz" file in the datasets (e.g., results_nsfnetbw_10-12_Routing-SP-k-0_0_124.tar.gz) contains 125 simulations with a given routing configuration and a traffic intensity range (<lower lambda max>-<upper lambda max>). Then, for a given traffic intensity we generate 125 different traffic matrices and make one simulation for each of them. The method to generate these matrices is described in [1] (Section 4.1).

Each line in the "simulationResults.txt" file contains all the data resulting from every simulation. In other words, all the data of a simulation is flattened into a vector (hereafter "line"). Then, the expression you mention points to the positions where the values calculated by our simulator are (within the "simulationResults.txt" file). Maybe it would be more correct to rewrite the expression as:

bandwidth[src_node][dst_node] = line[(src_node∗n+dst_node)*3]

If you want to extract the whole traffic matrix (i.e., bandwidth[:][:]), you can make a 2-layer loop to evaluate all the combinations of "src_node" and "dst_node" in the network (whose range is [0-13] in the case of the nsfnet dataset).

Regarding the naming of the routing configurations, basically there are two types of routings: "SP" are variants of the shortest path configuration, while "W" includes longer paths than in the case of "SP". In this context, "k_x" is just an identifier to differentiate every "SP" or "W" configuration. However, it doesn't have any meaning about the generation process. Lastly, "0_124" indicates that in this file there are 125 simulations in total (i.e., [0-124]).

I hope I answered all your questions.


[1] Krzysztof Rusek, José Suárez-Varela, Albert Mestres, Pere Barlet-Ros, Albert Cabellos-Aparicio; "Unveiling the potential of Graph Neural Networks for network modeling and optimization in SDN," in and in ACM Symposium on SDN Research (SOSR) , pp. 140-151, 2019. (https://arxiv.org/pdf/1901.08113.pdf)

Regards,

José

On 17/02/20 15:47, tjuzeyuluan wrote:
Dear José,
        
       I am so appreciated that you can offer me so much guidance. During these days, I gradually understand the structure of dataset_v1, i.e. nsfnetbw.tar.gz. 
       


        First, I imported the .ned file into omnet++ and got the topology as shown in the figure below.
      


     Second, I checked the link connection between a pair of output ports of its corrsponding switches as shown in the figure below. 
    


     Third, I decoded the meaning of element in Routing.txt. Actually, it can be seen as a collection of routing tables  of all the switches. For instance, line 0 represents the routing table of switch 0. For each row, 14 columns represent 14 destinations respectively from switch 0. And the element in the position (row i, column j) means, in switch i’s routing table, which output port of switch i should be chosen in order to forward a packet to arrive at destination j. For example in the file results_nsfnetbw_8-10_Routing-SP-k-0_0_124/Routing.txt, the element (1,2) = 1 means, in order to forward a packet to switch 2, switch 1 should forward the packet through its output port 1.
 
     

       These three steps above seem to be with no doubt till now. However, I don’t understand the data structure yet. In the Readme file of GitHub (https://github.com/knowledgedefinednetworking/NetworkModelingDatasets/tree/master/datasets_v1). It says there are 125 lines in each simulationResults.txt, representing 125 different levels of traffic intensity in 125 simulations.

        And there are 11 key performance indices listed below. For example, bandwidth [src_node][dst_node] seems to be a 2-layer for-loop, the outer layer is the index of src_node from 0-13 meanwhile the inner layer is the index of dst_node from 0-13. So there should be 14*14 = 196 elements. But what does the highlight expression mean in the figure snapshot?
       


      I thought the value should be sampled from the simulation environment rather than calculated. For instance, the  bandwidth between src_node = switch 1 and dst_node = switch 4 should be recorded in the position bandwidth[1][4], but what does bandwidth[1][4] = (1*14 + 4) *3 = 54 mean? 

     

        Another question is about the naming manner of each file as shown below. I know 8-10 and 14-16 represent the <lower bound , upper bound> of traffic intensity. However, what dose SP-k-x_0 and W-1-k-x_0 mean? Does it mean shortest path first and widest path first routing scheme. But how can 0 path exist? 


  

       Could you please give me a bit hint to these two questions? Thanks a lot!

Best Regards,

Zeyu

--------- Forwarded Message ---------

Date： 02/17/2020 20:39
Subject： Re: [Kdn-users] asking for Dataset

Dear Zeyu,

Thank you for your interest in this work. Unfortunately, the datasets are quite large and cannot be shared via e-mail. Note that files under the "nsfnetbw/tfrecords/train/" directory include 889M of data.

The main difference between both datasets you mention (v0 and v1) is in the topologies they include (see README files with the descriptions). Also, 'datasets_v0' include 500 iterations for each combination of routing+traffic intensity. Note that each iteration uses a different input traffic matrix (TM) of a given traffic intensity (TI). The method to generate these traffic matrices is described in Section 4.1 of [1].  In the case of 'datasets_v1', each file includes 125 iterations. In this case, a file includes a collection of traffic matrices with a range of traffic intensities (<lower lambda max>-<upper lambda max>). Also, these latter datasets include the following information:

"5.- Average per-packet neperian logarithm of the delay over the packets transmitted in each source-destination pair".

Which can be useful to make probabilistic modeling. For instance, to parameterize a Gamma distribution that models the delay distribution on each source-destination pair.

Overall, if you want to reproduce the experiments of a paper I recommend you use the datasets used in the paper. Otherwise, you will need to modify the code to read datasets with a different format. For instance:

"Challenging the generalization capabilities of Graph Neural Networks for network modeling" -> datasets_v0 (https://github.com/knowledgedefinednetworking/NetworkModelingDatasets/tree/master/datasets_v0)

Also, for the paper "Unveiling the potential of Graph Neural Networks for network modeling and optimization in SDN" you should use the datasets at the following link:

https://github.com/knowledgedefinednetworking/Unveiling-the-potential-of-GNN-for-network-modeling-and-optimization-in-SDN/tree/master/datasets

This paper presents the first version of RouteNet (https://github.com/knowledgedefinednetworking/net2vec/tree/RouteNet-SOSR/routenet), which did not have support for variable link capacity. For this reason, in these latter datasets all the links in the different topologies have the same capacity. You can check the link capacities used in the "*.ned" files that describe each topology.


[1] Krzysztof Rusek, José Suárez-Varela, Albert Mestres, Pere Barlet-Ros, Albert Cabellos-Aparicio; "Unveiling the potential of Graph Neural Networks for network modeling and optimization in SDN," in and in ACM Symposium on SDN Research (SOSR) , pp. 140-151, 209. Link: https://github.com/knowledgedefinednetworking/Unveiling-the-potential-of-GNN-for-network-modeling-and-optimization-in-SDN


Regards,

José


On 14/02/20 08:00, tjuzeyuluan wrote:
Dear Professor,
    I’m Zeyu, a PhD student from UC Berkeley. I am really interested in your work related in Graph Neural Networks -based routing optimization. I am trying to repeat your experiment. However, the download speed from the URL(path = '/home/datasets/SIGCOMM/nsfnetbw/tfrecords/train/') is so slow. Could you please transfer the zip package via the e-mail? Thank you very much! 
    Another question is that what’s the difference between dataset v0 and dataset v1. I am a little confused. Could you please explain further? Thanks a lot!


_______________________________________________
Kdn-users mailing list
Kdn-users@knowledgedefinednetworking.org
https://mail.n3cat.upc.edu/cgi-bin/mailman/listinfo/kdn-users