📋 Job Overview

The role involves leading the architecture design and implementation of end-to-end inference systems for deep learning algorithms in the Deep Synergy Algorithm Team, focusing on high throughput and low latency for large models. Responsibilities include deep performance analysis of the full inference chain, optimization through operator tuning, quantization, and resource scheduling to maximize throughput while controlling costs. Additionally, it entails optimizing the underlying inference framework, enhancing modules like dynamic batching and caching, and building engineering capabilities for usability and debuggability to support stable large-scale inference services.