Career Profile
I’m a Ph.D. student in the Department of Computer Science and Engineering, University of Minnesota, Twin Cities. I’m currently working under the supervision of Professor Arindam Banerjee. I have broad interests in Machine Learning and Deep Learning. My work includes large scale learning, stochastic gradient algorithms, and Sequence-to-Sequence model for sub-seaonal forecasting.
Experiences
- Study the role of active parameters of Deep Neural Networks (DNNs) through the concept of re-initializing
- Discover the layer-wise distribution of the active parameters.
- Propose Layer-Wise Sparse (LWS) SGD, which only updates some subsets of a DNN’s layers during training.
- Demonstrate that LWS-SGD can match the generalization performance of vanilla SGD and the back propagation time can be 2-5 times more efficient.
- Propose noisy truncated SGD (NT-SGD) which aims to reduce the communication cost in distributed setting and takes the advantage of both the fast-decaying structure of stochastic gradient components and the power of additional Gaussian noise.
- Design and perform extensive empirical study to demonstrate that NT-SGD matches the speed and accuracy of vanilla SGD, and can successfully escape sharp minima.
- Build novel Sequence-to-sequence models for sub-seasonal forecasting (forecasting 14-28 days ahead of time) that better handles the long-term temporal dependencies in climate time series.
- Design proper evaluation pipeline to validate model performances on small-scale time series dataset.
- Collect and propose a useful benchmark dataset as the ‘Imagenet’ dataset for sub-seasonal forecasting.
- Wrote 1000+ lines of python/pytorch-code implementing 5+ baseline models, e.g. LSTM-Autoencoder.
- Studied the properties of loss surfaces of deep Relu network through the Hessian matrix
- Unveiled a novel observation of the overlap between the principal eigen-spaces of the Hessian and the second moment of the stochastic gradients.
- Implemented a python/pytorch toolbox for efficient computation of eigenvalues and eigenvectors of large-scale matrices with millions of entries.
- Designed deep neural network for land climate prediction using a small climate dataset containing high-resolution (high dimensional) sea surface temperature data.
- Applied transfer learning on pre-trained VGG-Net and ResNet to solve the same climate prediction problem.
- Built deep neural network with special feature descriptor to perform diagnosis on malignant and benign neoplasms of the endometrium, prostate, breast, ovary and myometrium, squamous intraepithelial lesions of the lower female genital tract, and histopathologic diagnosis of non-neoplastic pulmonary diseases, with average prediction accuracy over 90%.
publications
Experiments with Rich Regime Training for Deep Learning
arXiv preprint arXiv:2103.00075, 2021.
Noisy Truncated SGD: Optimization and Generalization
arXiv preprint arXiv:2102.13522, 2021.
Sub-Seasonal Climate Forecasting via Machine Learning: Challenges, Analysis, and Advances
The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI), 2021.
Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization
SIAM International Conference on Data Mining (SDM), 2020.
Interpretable Predictive Modeling for Climate Variables with Weighted Lasso
The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), 2019.