UniTG: A Unified System for Efficient and Seamless Textual Graph Learning

Abstract

It has become critical to utilize language models (LMs) for representation learning on text-attributed graphs. They enhance the original graph neural networks (GNNs) by delicately modeling text attributes alongside graph structure learning. Despite these algorithmic breakthroughs, existing LM-based graph learning still fails in practical deployment due to several critical defects, namely time and resource inefficiency, inflexible decoupled architectures, limited model scale, and the omission of graph properties. In this paper, we propose UniTG, the first unified system that fuses the LM and GNN phases into a single end-to-end procedure through three co-designed components spanning the runtime, algorithm, and execution levels. At the runtime level, UniTG introduces Affinity-aware Flow Parallelism, exploiting graph affinity to scale the training of large graph neural networks. At the algorithm level, a novel Collaborative Learning strategy integrates both text and graph modalities to enable accurate joint training. At the execution level, the Streamlined Pipeline Schedule squeezes pipeline bubbles by interleaving LM fine-tuning into the GNN pipeline, boosting overall efficiency and resource utilization. Extensive experiments demonstrate that, compared with state-of-the-art LM-based graph learning systems, UniTG dramatically reduces learning makespan by up to 17.3x without compromising model quality.