- Challenge-kdn - example.com

Graph Neural Networking challenge 2021 – Creating a Scalable Network Digital Twin
by José Suárez-Varela 31 May '21

31 May '21

Dear Colleagues, We are glad to announce the second edition of the "*Graph Neural Networking challenge*", organized as part of the "ITU AI/ML in 5G Challenge". Challenge description: *Creating a Scalable Network Digital Twin* We provide several resources and tips to easily start working on the challenge (See “Main Resources”). Top-3 teams will have access to the global round of the "ITU AI/ML in 5G Challenge <https://aiforgood.itu.int/ai-ml-in-5g-challenge/>". This competition attracted over 1,300 participants around the world in the last edition, and there were attractive cash prizes for winners. Registration is now open and free of charge for all participants (Deadline Aug 31st). Website: https://bnn.upc.edu/challenge/gnnet2021 <https://bnn.upc.edu/challenge/gnnet2021> Registration: https://bnn.upc.edu/challenge/gnnet2021/registration <https://bnn.upc.edu/challenge/gnnet2021/registration> If you have any questions or comments, please do not hesitate to contact us at: gnnetchallenge(a)bnn.upc.edu <mailto:gnnetchallenge@bnn.upc.edu> ===================================================== [Overview] Graph Neural Networks (GNN) have produced groundbreaking applications in many fields where data is fundamentally structured as graphs (e.g., chemistry, physics, biology, recommender systems). In the field of computer networks, this new type of neural networks is being rapidly adopted for a wide variety of use cases, particularly for those involving complex graphs (e.g., performance modeling, routing optimization, resource allocation in wireless networks). The Graph Neural Networking challenge 2021 brings a fundamental limitation of existing GNNs: their lack of generalization capability to larger graphs. In order to achieve production-ready GNN-based solutions, we need models that can be trained in network testbeds of limited size (e.g., at the vendor’s networking lab), and then be directly ready to operate with guarantees in real customer networks, which are often much larger in number of nodes. In this challenge, participants are asked to design GNN-based models that can be trained on small network scenarios (up to 50 nodes), and after that scale successfully to larger networks not seen before, up to 300 nodes. Solutions with better scalability properties will be the winners. [Problem statement] The goal of this challenge is to create a scalable Network Digital Twin based on neural networks, which can accurately estimate QoS performance metrics given a network state snapshot. More in detail, solutions must predict the per-path mean delay given: (i) a network topology, (ii) a traffic matrix, and (iii) a routing configuration. As a baseline, we provide RouteNet, a GNN model recently proposed to estimate end-to-end performance metrics (e.g., delay, jitter, loss) in networks. However, RouteNet does not scale well to networks considerably larger than those observed during the training phase. As a result, it does not perform well when applied to this challenge. Participants are encouraged to update RouteNet or submit their own neural network models. [Main resources] - Summary slides (with some tips for participants) - Baseline GNN model: Open source implementation of RouteNet, including a tutorial on how to use it and modify fundamental characteristics of the model - Training/validation datasets - Python API to easily read and process the datasets - Mailing list for participants and people interested (Link to subscribe:https://mail.bnn.upc.edu/cgi-bin/mailman/listinfo/challenge2021 <https://mail.bnn.upc.edu/cgi-bin/mailman/listinfo/challenge2021>). [Timeline] * Registration open to participants: Deadline Aug 31st * Evaluation phase: Sep 16th * Winners (top 3) official announcement: Oct 31st * Final awards and presentation: Dec 2021 *_Important notice_**:* You have received this email because you are subscribed to the challenge-kdn(a)mail.knowledgedefinednetworking.org <mailto:challenge-kdn@mail.knowledgedefinednetworking.org>mailing list. From June 15^th this list will be removed, and all members will be moved to a new mailing list related to the Graph Neural Networking challenge: challenges(a)mail.bnn.upc.edu <mailto:challenges@mail.bnn.upc.edu> Please, let us know if you do not want to be included in this new list before June 15^th by sending an email to gnnetchallenge(a)bnn.upc.edu <mailto:gnnetchallenge@bnn.upc.edu>. Otherwise, we will add all members by default. Regards, José Suárez-Varela ------------------------ Barcelona Neural Networking center (BNN-UPC) Universitat Politècnica de Catalunya

1 0

Fwd: ITU AI/ML in 5G Challenge experience survey
by José Suárez-Varela 13 Nov '20

13 Nov '20

Dear participants, Please take some time to fill this short survey from ITU. Your feedback will be very useful to plan future editions of the challenge. Thank you in advance. Best regards, José Suárez-Varela Barcelona Neural Networking center Universitat Politècnica de Catalunya -------- Mensaje reenviado -------- Asunto: ITU AI/ML in 5G Challenge experience survey Fecha: Fri, 13 Nov 2020 08:51:08 +0000 De: AI-5G-Challenge, ITU <ai5gchallenge(a)itu.int> Para: AI-5G-Challenge, ITU <ai5gchallenge(a)itu.int> Dear Participants, As the first edition of the ITU AI/ML in 5G Challenge is drawing to a close, we would like to understand your experience in this edition. This helps us to plan towards future editions of the Challenge. Please take some time to respond to our questions in the survey below (e.g. on your motivations, clarity of problem statements, availability of resources, etc) so that we can improve your experience in the upcoming edition of the Challenge. We would like to get responses from everyone (whether you submitted a solution or not). Please follow the link here to answer the survey <https://forms.office.com/Pages/ResponsePage.aspx?id=12TkI-YEh0uRPCS9iSGf0-y…>. */Team leaders/*: make sure that you share this survey with your team members. Looking forward to receiving your responses. Regards Thomas ITU AI/ML in 5G Challenge

1 0

End of the evaluation phase and provisional ranking
by José Suárez-Varela 16 Oct '20

16 Oct '20

Dear participants of the Graph Neural Networking challenge 2020, First of all, thank you very much again on behalf of the BNN organizing team for participating in this challenge. We hope you enjoyed it a lot. Also, we expect it was a good opportunity for some of you to get introduced to the topics of network modeling and Graph Neural Networks. Once the evaluation phase has finished, we are glad to announce a provisional ranking with all the teams that submitted solutions for the challenge. You can find it at this link: https://bnn.upc.edu/challenge2020 We are very happy to see that you have made a really active participation. For the top-5 teams, please expect a separate email with detailed instructions on how to prepare the required documentation and the code to reproduce your solutions. For your interest, from the BNN team we have also prepared a possible solution for this challenge. Particularly, we have implemented a GNN model that introduces the queue entity as part of the internal GNN architecture. This enables to effectively model the impact of various queuing policies on delay. For more information, we have posted a short paper describing the solution and including some evaluation results on other datasets than those used in the challenge (https://arxiv.org/abs/2010.06686). Best regards, José Suárez-Varela Barcelona Neural Networking center Universitat Politècnica de catalunya

1 0

End of the evaluation phase
by José Suárez-Varela 14 Oct '20

14 Oct '20

Dear participants, I would like to remind you that the end of the evaluation phase is approaching. You can make submissions until *October 15th at 23:59:59 (GMT+2)*. Once the evaluation finishes, we will post in our website a ranking (on Oct 16th) with all the teams that submitted solutions and their scores. Also, we will send instructions to the top-3 teams with details on how to prepare the documentation and the code to reproduce the training of their solutions. Best regards, José Suárez-Varela Barcelona Neural Networking center Universitat Politècnica de Catalunya

1 0

Correction in the instructions for the evaluation phase
by José Suárez-Varela 03 Oct '20

03 Oct '20

Dear participants, We found a small error in the document with the instructions for the evaluation phase. Particularly, to generate the output CSV format with your solutions you can use the following pseudo-code: String = “” For node_src in range (19): For node_dst in range (19): *If node_src != node_dst:* String += str(delay[node_src, node_dst])+”;” # New sample, new line String +=”\n” This skips the node pairs where the source and destination match, thus resulting in 342 src-dst delay values (i.e., 19 sources x 18 destinations = 342 values). Please find attached a new version of the instruction document fixing this error. Also, to help you validate the format of your submission files, I attach a ZIP file with the expected format when applying a solution to our toy validation dataset (https://github.com/knowledgedefinednetworking/RouteNet-challenge/tree/maste…). Note that in this case the results correspond to a 17-node network, so we have 17x16=272 src-dst delay values per line. However, with the final test dataset we should expect 342 values per line. *I would like to take this opportunity to encourage you to register on our evaluation platform in advance to avoid potential problems before the evaluation phase starts **(October 1st)**.* Best regards, José Suárez-Varela Barcelona Neural Networking center Universitat Politècnica de Catalunya

2 3

Test dataset release today at 12h (CEST)
by José Suárez-Varela 30 Sep '20

30 Sep '20

Dear participants, We are glad to announce that the final test dataset will be publicly available at the following link from today (Sep 30^th ) at 12:00 (CEST): https://challenge.bnn.upc.edu/downloads?download=5:gnnet-data-set-evaluation Please note that the evaluation phase will open on Oct 1^st at 00:00:00 (CEST), so it will NOT be possible to make submissions before this time. However, you can start generating the submission files with your solutions. Remember that you can only make a maximum of 5 submissions each day, and in total each team can make up to 20 submissions. The evaluation platform will accept submissions until October 15^th at 23:59:59 (CEST). In the evaluation platform you will be able to find an anonymized ranking with the 5 best scores at the moment. This may help assess how good are your results compared to other teams. Each team will be considered for its best score, regardless of whether it is the last submission or not. Also, you will have access to a log with all the submissions of your team registered in our platform and the scores obtained. Please do not hesitate to contact us if you have any problems or need help during the evaluation process. Enjoy the challenge! Challenge website: https://bnn.upc.edu/challenge2020 Best regards, José Suárez-Varela Barcelona Neural Networking center Universitat Politècnica de Catalunya

1 0

Awards from ITU and instructions for the evaluation phase
by José Suárez-Varela 23 Sep '20

23 Sep '20

Dear participants, As maybe some of you may not have received this information, we would like to share with you that ITU announced officially the following awards for the winners of the global round: - *1st prize**: 5,000 CHF *(approx. $5,510 /4,640 €) - *2nd prize**: 3,000 CHF* (approx. $3,306 / 2,784€) - *3rd prize**: 2,000 CHF* (approx. $2,204 / 1,856€) - *3 Runners up*: *1,000 CHF* (approx. $1,102 / 928€) each. Note that top-3 solutions of the Graph Neural Networking challenge will have access to the global round. Also, the winners of this challenge will receive some amounts directly, which will be announced shortly. *Please, remember that the evaluation phase starts on **October 1st**, and **try to register on our evaluation platform with some time in advance to avoid potential issues. *Also, we recommend to validate the output format of your submissions with the instructions we provide in the attached document. *We encourage all of you to participate in the evaluation*. Even if you think your solution may not be among the top ones, you have nothing to lose and eventually you may be gladly rewarded. Also, keep in mind that there are three winning positions. Please do not hesitate to contact us if you have any comments or questions. Enjoy the challenge! Challenge website: https://bnn.upc.edu/challenge2020 Best regards, José Suárez-Varela Barcelona Neural Networking center Universitat Politècnica de Catalunya

1 0

Graph Neural Networking challenge 2020 – Instructions for the evaluation phase
by José Suárez-Varela 18 Sep '20

18 Sep '20

Dear participants, Thank you again for taking part in this challenge. The evaluation process will open soon (Oct 1^st -Oct15^th ), and from the organizing team of the Barcelona Neural Networking center we would like to share with you some detailed instructions on how it will be done. Please, read carefully the attached document and *try to register on our platform before the start of the evaluation phase (Oct 1st). Also, we recommend to validate in advance the output format of your submissions with the example dataset and the script we provide*. Do not hesitate to contact us if you have any comments or questions about all this process. Enjoy the challenge! Challenge website: https://bnn.upc.edu/challenge2020 Regards, José Suárez-Varela Barcelona Neural Networking center Universitat Politècnica de Catalunya

1 0

Update the code: datanetAPI bug
by Albert López 20 Jul '20

20 Jul '20

Dear participants, We have found a small bug in the API we provide to read the datasets (datanetAPI.py). This particularly affects the functions "get_maxAvgLambda()" and "get_global_delay()" of the API. If you are using RouteNet (https://github.com/knowledgedefinednetworking/RouteNet-challenge), please update the code with the latest version of the repository. You can do it with "git pull" from the terminal or downloading the code directly from the GitHub website. Alternatively, if you have already made major modifications to the code, you can replace only the file "datanetAPI.py" under the directory "RouteNet-challenge/code/". The same applies if you are using the API code provided at https://github.com/knowledgedefinednetworking/datanetAPI. Sorry for the inconvenience. Enjoy the challenge! And remember, if you have any questions/comments don't hesitate to use this mailing list to share them. Best Regards, Albert López Barcelona Neural Networking center Universitat Politècnica de Catalunya

1 0

Re: [Challenge-kdn] Question about RouteNet code on GitHub
by José Suárez-Varela 16 Jul '20

16 Jul '20

Dear Minh, Please, find my answers inline: > Dear GNNet challenge 2020 organizing team, > > My name is Minh, one of the participants in the challenge. I have > questions about the RouteNet code on GitHub that I want to ask as follows: > > 1) Can you please provide a requirements.txt file of your Python > environment? I'm aware that you wrote the codes depend on > tensorflow=2.1.0, networks>=2.4, and pandas >=0.24, but I often get > different errors when running on different machines, so I think a > requirements.txt file will make it easier not only for me but also for > all other participants. We are aware that the process to install all the libraries may be a bit confusing. However, we decided not to put a 'requirements.txt' file because we think it can be more error-prone. The main problem comes when you install TensorFlow in a machine with a CUDA-enabled GPU card, since you need to have the right CUDA and CuDNN versions pre-installed. These libraries depend on the operating system and the version you use. Thus, the most laborious part is to install correctly CUDA and CuDNN, but you can do it in a pretty systematic way following this TensorFlow tutorial: https://www.tensorflow.org/install/gpu > 2) What is the terminating point of RouteNet as a baseline for the > challenge? From what I see, it only stops after 5 million steps, which > takes an incredibly long time, should this be the terminating point of > the algorithm? And the results after 5 million steps are the baseline > results for the competition? This is just an arbitrary upper bound limit we set in the implementation. The idea is that you can stop the execution whenever you want based on the training progress, and the last models (checkpoints) are automatically saved in the "../logs/model_log" directory. Note that after 400k steps the model has iterated over all the training dataset. Further iterations can only help slightly refine the model. You can take the baseline as a reference implementation to develop your model. However, with this baseline you can expect a MAPE (Mean Absolute Percentage Error) above 100%. One main reason is that it does not encode information about queue scheduling, and this has a great impact on network delay. * * > I look forward to hearing from you soon. Thank you. > > Best regards, > > Minh Nguyen. > > King Abdullah University of Science and Technology (KAUST) > Al-Khawarizmi Applied Math. Building (Bldg. #1) | Level 3, Table > 3139-WS01 > Thuwal 23955-6900 | Makkah Province > Kingdom of Saudi Arabia I hope I answered all your questions. Please, let me know if you have any more questions/comments. Regards, José Suárez-varela Barcelona Neural Networking center - UPC

1 0

2026

2025

2024

2023

2022

2021

2020

Challenge-kdn