Graphs are a kind of data structure that models a set of objects (nodes) and their relationships (edges). Lots of learning tasks deal with graph data that have rich relationships and mutual dependency between objects. Graphs have a lot of practical uses — in social networks, natural science (physics systems), chemistry, medicine, and many other research areas. This fuels the growing interest of deep learning researchers in the structure of graph data.
We’ll describe Graph Neural Networks (GNNs), cover popular GNN libraries, and we’ll finish with great learning resources to get you started in this field.
Prerequisites: This article assumes a basic understanding of Machine Learning (ML), Deep Learning (DL), and GNNs. For a more in-depth understanding of GNN basics and applications, feel free to check out my previous article.
Graph Neural Networks (GNN) overview
Graph Neural Networks (GNNs) came to life quite recently. They’re a class of deep learning models for learning on graph-structured data.
GNNs are neural networks designed to make predictions at the level of nodes, edges, or entire graphs. For example, a prediction at a node level could solve a task like spam detection. An edge-wise prediction task could be link prediction, a common scenario in recommender systems. A graph-wise prediction task could be predicting the chemical properties of molecular graphs.
List of GNN Python libraries
Let’s explore some high-quality open-source libraries for graph neural networks that will help you in your GNN journey.
1) PyTorch Geometric
PyTorch Geometric (PyG) is a Python library for deep learning on irregular structures like graphs. The project was developed and released by two Ph.D. students from TU Dortmund University, Matthias Fey and Jan E. Lenssen.
Along with general graph data structures and processing methods, it has a variety of recently published methods from the domains of relational learning and 3D data processing. PyTorch Geometric achieves high data throughput by leveraging sparse GPU acceleration, providing dedicated CUDA kernels, and introducing efficient mini-batch handling for input examples of different sizes.
2) Deep Graph Library (DGL)
Deep Graph Library(DGL) is another easy-to-use, high-performance, and scalable Python library for deep learning on graphs. It’s the product of a group of deep learning enthusiasts called the Distributed Deep Machine Learning Community. It has a very clean and concise API. DGL introduces a useful higher-level abstraction, allowing for auto-batching.
3) Graph Nets
Graph Nets is DeepMind’s library for building graph networks in Tensorflow and Sonnet. The library works with both the CPU and GPU versions of TensorFlow. It offers the flexibility that almost any existing GNN can be implemented using 6 core functions, and it can be extended to Temporal Graphs. Graph Nets require TensorFlow 1, so it feels outdated even though it’s only about 3 years old.
Spektral is an open-source Python graph deep learning library, based on the Keras API and TensorFlow 2. The main goal of this library is to provide a simple, flexible framework for creating GNNs. You can use Spektral to classify the users of a social network, predict molecular properties, generate new graphs with GANs, cluster nodes, predict links, and any other task where data is described by graphs. It implements some of the most popular layers for graph deep learning. This library is designed according to the guiding principles of Keras to make things extremely simple for beginners while maintaining flexibility for experts. Unfortunately, there’s a trade-off for the simplicity of using Spektral, which is the slowness in training speeds for most tasks compared to other libraries like DGL and PyG.
Overview of GNN libraries
|Library Name||License||Stars||Programming Language||Main Contributor(s)|
|Pytorch Geometric||MIT||11.2k||Python, PyTorch||Matthias Fey|
|Deep Graph Library||Apache 2.0||7.4k||Python, PyTorch, TF, MxNet||Distributed MLC|
|Graph Nets||Apache 2.0||4.9k||Python, PyTorch||DeepMind|
|Spektral||MIT||1.8k||Python, TF2/Keras||Daniele Grattarola|
Which GNN library should you choose?
It’s about choosing the library that meets your needs, and this choice is usually influenced by a previous choice of deep learning libraries made by you or by your manager/teammate.
For example, if you’ve worked before, or you’re used to working with Keras and Tensorflow, then Spektral may be a good library for you. For Graph Nets DeepMind library, I don’t recommend starting a new GNN project with it due to TensorFlow 1. At the same time, it’s a reasonable choice if you’re working on legacy projects.
If you want a fast, capable library at a relatively established and mature state of development, with the ease of integration of common benchmark datasets to implementation of other papers, then PyTorch Geometric is a good choice.
May be useful
Best learning resources for Graph Neural Networks
After my first GNN article, I got a lot of messages asking for the best resources to understand this topic. Since the GNN field has been growing very quickly, up-to-date knowledge is not always easily available.
Here’s the list of best resources that you need to bookmark if you want to get hands-on practical experience in this field.
I think this course is a must if you want to grow your knowledge about Graph Neural Networks. It has publicly available slides from lectures, along with well recommended readings. This course is taught by GraphSage author, Jurij Leskovec, himself. I highly recommend starting with this course.
This book is the result of a big collaboration that shaped everything, from content to visualizations and interactive tools. Although this book is not about GNNs, it’s an excellent resource to get a solid foundation on graphs.
This book is available as a pre-publication online. It provides a brief but comprehensive introduction to graph representation learning, including methods for embedding graph data, graph neural networks, and deep generative models of graphs. It has almost all the required theories of graph neural networks.
The repository contains the recent GNN papers subdivided by topics like Models and Applications of GNN (Chemistry, NLP, Traffic network and Adversarial Attach, etc.). It’s worth checking if you’re interested in new papers focusing on specific applications of GNNs.
If you want to find Graph Neural Network models with code implementation that you can use, Paper With Code (PwC) is the best place to search.
It’s a website that organizes access to technical papers. It has grown immensely in the past few years. Coupled with the increase of publicly available datasets, modern research has started to converge back towards full transparency and credibility. PwC has been consistently improving its website as well. You can easily navigate the state of the art via browsing, either by task or by method (e.g. attention, transformers).
Over the past few years, GNNs have become powerful and practical tools for machine learning tasks in the graph domain. This article is just a simple overview of graph neural networks. We’ve summarised popular GNN libraries, and listed the best learning resources to ease your way into this boundless field.
I hope that you enjoyed this article! In case you have questions or need any kind of assistance, feel free to contact me.
Where Can You Learn About MLOps? What Are the Best Books, Articles, or Podcasts to Learn MLOps?ML Experiment Tracking: What It Is, Why It Matters, and How to Implement It
4 mins read | Paweł Kijko | Updated May 31st, 2021
MLOps is not a piece of cake. Especially in today’s changing environment. There are many challenges—construction, integrating, testing, releasing, deployment, and infrastructure management. You need to follow good practices and know how to adjust to the challenges.
And if you don’t learn and develop your knowledge, you’ll fall out of the loop. The right resources can help you follow the best practices, discover helpful tips, and learn about the latest trends.
You don’t have to look far, we’ve got you covered! Here’s your list of the best go-to resources about MLOps—books, articles, podcasts, and more. Let’s dive in!
1. Introducing MLOps from O’Reilly
Introducing MLOps: How to Scale Machine Learning in the Enterprise is a book written by Mark Treveil and the Dataiku Team (collective authors). It introduces the key concepts of MLOps, shows how to maintain and improve ML models over time, and tackles the challenges of MLOps.
The book was written specifically for analytics and IT operations team managers—the people directly facing the task of scaling machine learning (ML) in production. It’s a guide for creating a successful MLOps environment, from the organizational to the technical challenges involved.
The book is divided into three parts:
- An introduction to the topic of MLOps, how and why it has developed as a discipline, who needs to be involved to execute MLOps successfully, and what components are required.
- The second part follows the machine learning model life cycle, with chapters on developing models, preparing for production, deploying to production, monitoring, and governance.
- Provides tangible examples of how MLOps looks in companies today, so readers can understand the setup and implications in practice.