Meta-programming for cross-domain tensor optimizations
- 7 April 2020
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM SIGPLAN Notices
- Vol. 53 (9), 79-92
- https://doi.org/10.1145/3393934.3278131
Abstract
Many modern application domains crucially rely on tensor operations. The optimization of programs that operate on tensors poses difficulties that are not adequately addressed by existing languages and tools. Frameworks such as TensorFlow offer good abstractions for tensor operations, but target a specific domain, i.e. machine learning, and their optimization strategies cannot easily be adjusted to other domains. General-purpose optimization tools such as Pluto and existing meta-languages offer more flexibility in applying optimizations but lack abstractions for tensors. This work closes the gap between domain-specific tensor languages and general-purpose optimization tools by proposing the Tensor optimizations Meta-Language (TeML). TeML offers high-level abstractions for both tensor operations and loop transformations, and enables flexible composition of transformations into effective optimization paths. This compositionality is built into TeML's design, as our formal language specification will reveal. We also show that TeML can express tensor computations as comfortably as TensorFlow and that it can reproduce Pluto's optimization paths. Thus, optimized programs generated by TeML execute at least as fast as the corresponding Pluto programs. In addition, TeML enables optimization paths that often allow outperforming Pluto.Keywords
This publication has 23 references indexed in Scilit:
- Generating performance portable code using rewrite rules: from high-level functional expressions to high-performance OpenCL codePublished by Association for Computing Machinery (ACM) ,2015
- Cross-Loop Optimization of Arithmetic Intensity for Finite Element Local AssemblyACM Transactions on Architecture and Code Optimization, 2015
- Loo.pyPublished by Association for Computing Machinery (ACM) ,2014
- Unified form languageACM Transactions on Mathematical Software, 2014
- HalidePublished by Association for Computing Machinery (ACM) ,2013
- NWChem: A comprehensive and scalable open-source solution for large scale molecular simulationsComputer Physics Communications, 2010
- Semi-Automatic Composition of Loop Transformations for Deep Parallelism and Memory HierarchiesInternational Journal of Parallel Programming, 2006
- A Language for the Compact Representation of Multiple Program VersionsLecture Notes in Computer Science, 2006
- Facilitating the search for compositions of program transformationsPublished by Association for Computing Machinery (ACM) ,2005
- Single Assignment C: efficient support for high-level array operations in a functional settingJournal of Functional Programming, 2003