By Samuel Drapeau, Professor Shanghai Jiao Tong University
Title: Utilization of Transformers for Time Series Modeling
Abstract: Purely Markovian or recurrent models often fail to capture the complex, non-linear dependencies inherent in many financial time series, such as LOB dynamics. With recent advances in computational resources and AI libraries, new models open the door to alternative approaches. Transformers have revealed themselves as astonishingly efficient in handling the sequential nature of language modeling and are increasingly applied to time series modeling.
In this talk, we first present Transformer architectures as global functional approximators governed by quadratic interaction kernels, utilizing positional encodings to induce temporal order. We address specific time-series challenges, particularly the necessity of strict causal masking to preserve the filtration, as well as scaling issues and the artifacts arising from projecting continuous-time events onto fixed time grids. To resolve this, we present a multidimensional framework modeling both state values and stochastic timing. By decomposing dynamics into clocks and positions, our approach avoids fixed-grid discretization artifacts. While this generic framework admits various architectures, Transformers prove particularly computationally efficient for handling the long-range dependencies characteristic of complex high-frequency data. We finally present preliminary results on real high-frequency data and discuss the critical aspect of validation metrics—and their pitfalls—focusing on conditional Wasserstein and Skorohod distances. We demonstrate how this approach, integrated with a custom network-driven matching engine, allows for the replication of exchange behavior on a tick-by-tick basis.