MLOps Blog

Graph Neural Networks: Libraries, Tools, and Learning Resources

Amal Menzli

4 min

17th August, 2023

Machine Learning Tools ML Model Development

Graphs are a kind of data structure that models a set of objects (nodes) and their relationships (edges). Lots of learning tasks deal with graph data that have rich relationships and mutual dependency between objects. Graphs have a lot of practical uses — in social networks, natural science (physics systems), chemistry, medicine, and many other research areas. This fuels the growing interest of deep learning researchers in the structure of graph data.

We’ll describe Graph Neural Networks (GNNs), cover popular GNN libraries, and we’ll finish with great learning resources to get you started in this field.

Prerequisites: This article assumes a basic understanding of Machine Learning (ML), Deep Learning (DL), and GNNs. For a more in-depth understanding of GNN basics and applications, feel free to check out my previous article.

Graph Neural Networks (GNN) overview

Graph Neural Networks (GNNs) came to life quite recently. They’re a class of deep learning models for learning on graph-structured data.

GNNs are neural networks designed to make predictions at the level of nodes, edges, or entire graphs. For example, a prediction at a node level could solve a task like spam detection. An edge-wise prediction task could be link prediction, a common scenario in recommender systems. A graph-wise prediction task could be predicting the chemical properties of molecular graphs.

List of GNN Python libraries

Let’s explore some high-quality open-source libraries for graph neural networks that will help you in your GNN journey.

1) PyTorch Geometric

PyTorch Geometric (PyG) is a Python library for deep learning on irregular structures like graphs. The project was developed and released by two Ph.D. students from TU Dortmund University, Matthias Fey and Jan E. Lenssen.

Along with general graph data structures and processing methods, it has a variety of recently published methods from the domains of relational learning and 3D data processing. PyTorch Geometric achieves high data throughput by leveraging sparse GPU acceleration, providing dedicated CUDA kernels, and introducing efficient mini-batch handling for input examples of different sizes.

2) Deep Graph Library (DGL)

Deep Graph Library(DGL) is another easy-to-use, high-performance, and scalable Python library for deep learning on graphs. It’s the product of a group of deep learning enthusiasts called the Distributed Deep Machine Learning Community. It has a very clean and concise API. DGL introduces a useful higher-level abstraction, allowing for auto-batching.

3) Graph Nets

Graph Nets is DeepMind’s library for building graph networks in Tensorflow and Sonnet. The library works with both the CPU and GPU versions of TensorFlow. It offers the flexibility that almost any existing GNN can be implemented using 6 core functions, and it can be extended to Temporal Graphs. Graph Nets require TensorFlow 1, so it feels outdated even though it’s only about 3 years old.

4) Spektral

Spektral is an open-source Python graph deep learning library, based on the Keras API and TensorFlow 2. The main goal of this library is to provide a simple, flexible framework for creating GNNs. You can use Spektral to classify the users of a social network, predict molecular properties, generate new graphs with GANs, cluster nodes, predict links, and any other task where data is described by graphs. It implements some of the most popular layers for graph deep learning. This library is designed according to the guiding principles of Keras to make things extremely simple for beginners while maintaining flexibility for experts. Unfortunately, there’s a trade-off for the simplicity of using Spektral, which is the slowness in training speeds for most tasks compared to other libraries like DGL and PyG.

Overview of GNN libraries

Library Name	License	Stars	Programming Language	Main Contributor(s)
Pytorch Geometric	MIT	11.2k	Python, PyTorch	Matthias Fey
Deep Graph Library	Apache 2.0	7.4k	Python, PyTorch, TF, MxNet	Distributed MLC
Graph Nets	Apache 2.0	4.9k	Python, PyTorch	DeepMind
Spektral	MIT	1.8k	Python, TF2/Keras	Daniele Grattarola

Which GNN library should you choose?

It’s about choosing the library that meets your needs, and this choice is usually influenced by a previous choice of deep learning libraries made by you or by your manager/teammate.

For example, if you’ve worked before, or you’re used to working with Keras and Tensorflow, then Spektral may be a good library for you. For Graph Nets DeepMind library, I don’t recommend starting a new GNN project with it due to TensorFlow 1. At the same time, it’s a reasonable choice if you’re working on legacy projects.

If you want a fast, capable library at a relatively established and mature state of development, with the ease of integration of common benchmark datasets to implementation of other papers, then PyTorch Geometric is a good choice.

May be useful

If you use TensorFlow/Keras for model training, check how you can keep track of this process here.
If you use PyTorch, on the other hand, you can learn how to track your runs here.

Best learning resources for Graph Neural Networks

After my first GNN article, I got a lot of messages asking for the best resources to understand this topic. Since the GNN field has been growing very quickly, up-to-date knowledge is not always easily available.

Here’s the list of best resources that you need to bookmark if you want to get hands-on practical experience in this field.

1) Stanford Course: CS224W Machine Learning with Graphs

I think this course is a must if you want to grow your knowledge about Graph Neural Networks. It has publicly available slides from lectures, along with well recommended readings. This course is taught by GraphSage author, Jurij Leskovec, himself. I highly recommend starting with this course.

2) ‘Network Science’ by Albert-László Barabási

This book is the result of a big collaboration that shaped everything, from content to visualizations and interactive tools. Although this book is not about GNNs, it’s an excellent resource to get a solid foundation on graphs.

3) Graph Representation Learning Book by William L. Hamilton

This book is available as a pre-publication online. It provides a brief but comprehensive introduction to graph representation learning, including methods for embedding graph data, graph neural networks, and deep generative models of graphs. It has almost all the required theories of graph neural networks.

Graph Representation Learning Book - GNN resources

4) Github Repository: Collection of Recent GNN Papers

The repository contains the recent GNN papers subdivided by topics like Models and Applications of GNN (Chemistry, NLP, Traffic network and Adversarial Attach, etc.). It’s worth checking if you’re interested in new papers focusing on specific applications of GNNs.

5) Graph Neural Network Papers With Code

If you want to find Graph Neural Network models with code implementation that you can use, Paper With Code (PwC) is the best place to search.

It’s a website that organizes access to technical papers. It has grown immensely in the past few years. Coupled with the increase of publicly available datasets, modern research has started to converge back towards full transparency and credibility. PwC has been consistently improving its website as well. You can easily navigate the state of the art via browsing, either by task or by method (e.g. attention, transformers).

Conclusion

Over the past few years, GNNs have become powerful and practical tools for machine learning tasks in the graph domain. This article is just a simple overview of graph neural networks. We’ve summarised popular GNN libraries, and listed the best learning resources to ease your way into this boundless field.

I hope that you enjoyed this article! In case you have questions or need any kind of assistance, feel free to contact me.