Research Areas

Introduction

Generative deep neural networks (DNNs) such as Generative Adversarial Nets and Variational Autoencoders have been widely used for data generation problems, ranging from image generation to text generation. In most recent few years, deep neural networks have been extended to generate graphs, thanks to the development of graph deep learning.

Beyond the domain of graph generation which focuses on generating graphs unconditionally, graph transformation focuses on (stochastically) generate graphs in target domain, given a graph in source domain. This domain can be considered as an extension from traditional data traslation problems including image translation, natural language translation, and programming language translation.

Graph transformation is a promising domain with wide potential applications as shown in the following examples:




Methods and Codes

Problem Method

Paper

 Code
Graph topology transformation GT-GAN [arXiv] [code]
Directed acyclic graph transformation [AISTATS] [code]
Multi-attributed graph transformation NEC-DGT [ICDM 19 Best Paper] [code]
Domain specific Molecule optimization VJTNN [ICLR 19] [code]
Chemical reaction GTPN [KDD 19] [code]

Datasets (Our Benchmark Dataset Website is online now!)

Name Number of Samples Source Graph Target Graph

Paper
Cyber-network transformation
N/A
 benign network malicious network [arXiv]
Scale-free transformation
25,000
 scale-free graph grown scale-free graph [arXiv]
Random graph transformation
1,500
 random graph grown random graph [arXiv]
Attributed graph transformation
~1,500
 attributed graph new attributed graph [ICDM 19 Best Paper]
Internet of Things transformation
~500
 current network future network [ICDM 19 Best Paper]
Chemical reaction
7,180
 reactant molecules product molecules [ICDM 19 Best Paper]
Penalized logP
178,000
 molecule a new molecule [ICLR 19]
Drug likeness (QED)
88,000
 molecule more drug-like molecule [ICLR 19]
Dopamine Receptor (DRD2)
34,000
 molecule a new molecule [ICLR 19]
USPTO-15k
15,000
 Reactant molecules product molecules [KDD 19]
Graph copy (derived from here)
N/A
 source graph the same source graph [AISTATS]

Surveys

  • Xiaojie Guo and Liang Zhao. 2020. A Systematic Survey on Deep Generative Models for Graph Generation. arXiv preprint arXiv:2007.06686.

    • Introduction

    Spatio-temporal societal event forecasting, which has traditionally been prohibitively challenging, is now becoming possible and experiencing rapid growth thanks to the big data from Open Source Indicators (OSI) such as social media, news sources, blogs, economic indicators, and other meta-data sources. Spatio-temporal societal event forecasting and their precursor discovery benefit the society in various aspects, such as political crises, humanitarian crises, mass violence, riots, mass migrations, disease outbreaks, economic instability, resource shortages, responses to natural disasters, and others.

    Different from traditional event detection that identifies ongoing events, event forecasting focuses on predicting future events yet to happen. Also different from traditional spatio-temporal predictions on numerical indices, spatio-temporal event forecasting needs to leverage the heterogeneous information from OSI to discover the predictive indicators and mappings to future societal events. While studying large scale societal events, policy makers and practitioners aim to identify precursors to such events to help understand causative attributes and ensure accountability. The resulting problems typically require the predictive modeling techniques that can jointly handle semantic, temporal, and spatial information, and require a design of efficient and interpretable algorithms that scale to high-dimensional large real-world datasets. For event prediction in general, visit our EPIDE website.

    Methods and Codes

    Type of Events
    		 Method Category

    Method and Codes
    Temporal Event Forecasting Causal dependency mining Predefined causality [12, 22, 3]
    Optimized causality [17, 16, 2, 11]
    Temporal dependency mining Markov decision processes [15, 20]
    Deep neural networks [7, 14, 24, 8]
    Anormaly mining Scan-Statistic based [9,33,34]
    Distance based [35]
    Spatio-temporal Event Forecasting Discriminative Models Multi-task models
    [39, 29, 30, 6, 43, 46, 49]
    Multi-level models [32, 27]
    Multi-view models [31]
    Multi-layer models [37, 38, 48]
    Spatio-autoregressive [44]
    Generative & Mechanistic Models Generative Models [19, 26,40,47]
    Mechanistic Models [41]
    Ensemble Models Data-driven Models [12, 18]
    Data-Mechanistic-driven Models
    [28, 42, 45]

    [1] Somayyeh Aghababaei and Masoud Makrehchi. Mining social media content for crime prediction. In Web Intelligence (WI), 2016 IEEE/WIC/ACM International Conference on, pages 526-531. IEEE, 2016

    [2] Marta Arias, Argimiro Arratia, and Ramon Xuriguera. Forecasting with Twitter data. ACM Transactions on Intelligent Systems and Technology (TIST), 5(1):8, 2013.

    [3] Johan Bollen, Huina Mao, and Xiaojun Zeng. Twitter mood predicts the stock market. Journal of Computational Science, 2(1):1, 2011.

    [4] Feng Chen and Daniel B Neill. Non-parametric scan statistics for event detection and forecasting in heterogeneous social media graphs. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1166-1175. ACM, 2014

    [5] Mahtab Jahanbani Fard, Ping Wang, Sanjay Chawla, and Chandan K Reddy. A bayesian perspective on early stage event prediction in longitudinal data. IEEE Transactions on Knowledge and Data Engineering, 28(12):3126-3139, 2016.

    [6] Yuyang Gao and Liang Zhao. Incomplete label multi-task ordinal regression for spatial event scale forecasting. In AAAI Conference on Artificial Intelligence, pages 2999-3006, 2018.

    [7] Mark Granroth-Wilding and Stephen Clark. What happens next? event prediction using a compositional neural network model. In Thirtieth AAAI Conference on Artificial Intelligence, 2016.

    [8] Linmei Hu, Juanzi Li, Liqiang Nie, Xiao-Li Li, and Chao Shao. What happens next? future subevent prediction using contextual hierarchical lstm. In AAAI Conference on Artificial Intelligence, 2017.

    [9] Hyeon-Woo Kang and Hang-Bong Kang. Prediction of crime occurrence from multi-modal data using deep learning. PloS one, 12(4):e0176244, 2017.

    [10] Gizem Korkmaz, Jose Cadena, Chris J Kuhlman, Achla Marathe, Anil Vullikanti, and Naren Ramakrishnan. Combining heterogeneous data sources for civil unrest forecasting. In Advances in Social Networks Analysis and Mining (ASONAM), 2015 IEEE/ACM International Conference on, pages 258-265. IEEE, 2015

    [11] Canasai Kruengkrai, Kentaro Torisawa, Chikara Hashimoto, Julien Kloetzer, Jong-Hoon Oh, and Masahiro Tanaka. Improving event causality recognition with multiple background knowledge sources using multi-column convolutional neural networks. In AAAI Conference on Artificial Intelligence, 2017.

    [12] Sathappan Muthiah, Patrick Butler, Rupinder Paul Khandpur, Parang Saraf, Nathan Self, Alla Rozovskaya, Liang Zhao, Jose Cadena, ChangTien Lu, Anil Vullikanti, Achla Marathe, Kristen Summers, Graham Katz, Andy Doyle, Jaime Arredondo, Dipak K. Gupta, David Mares, and Naren Ramakrishnan. Embers at 4 years: Experiences operating an open source 7 indicators forecasting system. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2016, pages 205-214, New York, NY, USA, 2016. ACM.

    [13] Yue Ning, Sathappan Muthiah, Huzefa Rangwala, and Naren Ramakrishnan. Modeling precursors for event forecasting via nested multi-instance learning. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1095-1104. ACM, 2016

    [14] Karl Pichotta and Raymond J Mooney. Learning statistical scripts with lstm recurrent neural networks. In AAAI, pages 2800-2806, 2016.

    [15] Fengcai Qiao, Pei Li, Xin Zhang, Zhaoyun Ding, Jiajun Cheng, and Hui Wang. Predicting social unrest events with hidden markov models using gdelt. Discrete Dynamics in Nature and Society, 2017, 2017.

    [16] Kira Radinsky and Sagie Davidovich. Learning to predict from textual data. Journal of Artificial Intelligence Research, 45(1):641-684, 2012.

    [17] Kira Radinsky and Eric Horvitz. Mining the web to predict future events. In WSDM, pages 255-264, 2013

    [18] Naren Ramakrishnan, Patrick Butler, Sathappan Muthiah, Nathan Self, Rupinder Khandpur, Parang Saraf, Wei Wang, Jose Cadena, Anil Vullikanti, Gizem Korkmaz, et al. 'beating the news'?with embers: forecasting civil unrest using open source indicators. In KDD 2014, pages 1799-1808. ACM, 2014.

    [19] Theodoros Rekatsinas, Saurav Ghosh, Sumiko R Mekaru, Elaine O Nsoesie, John S Brownstein, Lise Getoor, and Naren Ramakrishnan. Sourceseer: Forecasting rare disease outbreaks using multiple data sources. In Proceedings of the 2015 SIAM International Conference on Data Mining, pages 379-387. SIAM, 2015.

    [20] Philip A Schrodt. Forecasting conflict in the balkans using hidden markov models. In Programming for Peace, pages 161-184. Springer, 2006.

    [21] Minglai Shao, Jianxin Li, Feng Chen, Hongyi Huang, Shuai Zhang, and Xunxun Chen. An efficient approach to event detection and forecasting in dynamic multivariate social media networks. In Proceedings of the 26th International Conference on World Wide Web, pages 1631-1639. International World Wide Web Conferences Steering Committee, 2017.

    [22] Andranik Tumasjan, Timm Oliver Sprenger, Philipp G Sandner, and Isabell M Welpe. Predicting elections with Twitter: What 140 characters reveal about political sentiment. ICWSM, 10:178-185, 2010.

    [23] Xiaofeng Wang, Matthew S Gerber, and Donald E Brown. Automatic crime prediction using events extracted from Twitter posts. In Social Computing, Behavioral-Cultural Modeling and Prediction, pages 231-238. Springer, 2012.

    [24] Wang, Zhongqing, and Yue Zhang. "DDoS event forecasting using Twitter data." Proceedings of the 26th International Joint Conference on Artificial Intelligence. AAAI Press, 2017.

    [25] Qian Zhang, Nicola Perra, Daniela Perrotta, Michele Tizzoni, Daniela Paolotti, and Alessandro Vespignani. Forecasting seasonal influenza fusing digital indicators and a mechanistic disease model. In Proceedings of the 26th International Conference on World Wide Web, pages 311-319. International World Wide Web Conferences Steering Committee, 2017.

    [26] Liang Zhao, Feng Chen, Chang-Tien Lu, and Naren Ramakrishnan. Spatiotemporal event forecasting in social media. In SDM 15, pages 963-971. SIAM, 2015

    [27] Liang Zhao, Feng Chen, Chang-Tien Lu, and Naren Ramakrishnan. Multiresolution spatial event forecasting in social media. In Data Mining (ICDM), 2016 IEEE 16th International Conference on, pages 689-698. IEEE, 2016.

    [28] Liang Zhao, Jiangzhuo Chen, Feng Chen, Wei Wang, Chang-Tien Lu, and Naren Ramakrishnan. Simnest: Social media nested epidemic simulation via online semi-supervised deep learning. In Data Mining (ICDM), 2015 IEEE International Conference on, pages 639-648. IEEE, 2015

    [29] Liang Zhao, Qian Sun, Jieping Ye, Feng Chen, Chang-Tien Lu, and Naren Ramakrishnan. Multi-task learning for spatio-temporal event forecasting. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1503-1512. ACM, 2015.

    [30] Liang Zhao, Qian Sun, Jieping Ye, Feng Chen, Chang-Tien Lu, and Naren Ramakrishnan. Feature constrained multi-task learning models for spatiotemporal event forecasting. IEEE Transactions on Knowledge and Data Engineering, 29(5):1059-1072, 2017

    [31] Liang Zhao, Junxiang Wang, and Xiaojie Guo. Distant-supervision of heterogeneous multitask learning for social event forecasting with multilingual indicators. In AAAI Conference on Artificial Intelligence, pages 4498-4505, 2018.

    [32] Liang Zhao, Jieping Ye, Feng Chen, Chang-Tien Lu, and Naren Ramakrishnan. Hierarchical incomplete multi-source feature learning for spatiotemporal event forecasting. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 2085-2094. ACM, 2016.

    [33] Chen, F., & Neill, D. B. (2014, August). Non-parametric scan statistics for event detection and forecasting in heterogeneous social media graphs. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 1166-1175). ACM.

    [34] Chen, F., & Neill, D. B. (2015). Human rights event detection from heterogeneous social media graphs. Big Data, 3(1), 34-40.

    [35] Rozenshtein, P., Anagnostopoulos, A., Gionis, A., & Tatti, N. (2014, August). Event detection in activity networks. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining(pp. 1176- 1185). ACM.

    [36] Chen, F., & Neill, D. B. (2015). Human rights event detection from heterogeneous social media graphs. Big Data, 3(1), 34-40.

    [37] Wu, Congyu, and Matthew S. Gerber. "Forecasting Civil Unrest Using Social Media and Protest Participation Theory." IEEE Transactions on Computational Social Systems 5, no. 1 (2018): 82-94.

    [38] Zhuoning Yuan, Xun Zhou, Tianbao Yang. Hetero-ConvLSTM: A Deep Learning Approach to Traffic Accident Prediction on Heterogeneous Spatio-Temporal Data. In 24th ACM SIGKDD International Conference on Knowledge Discovery from Data (KDD), 2018 (Accepted).

    [39] Yuyang Gao, Liang Zhao, Lingfei Wu, Yanfang Ye, Hui Xiong, Chaowei Yang. Incomplete Label Multi-task Deep Learning for Spatio-temporal Event Subtype Forecasting.Thirty-third AAAI Conference on Artificial Intelligence (AAAI 2019), Hawaii, USA, Feb 2019, to appear.

    [40] Liang Zhao, Feng Chen, Chang-Tien Lu, and Naren Ramakrishnan. "Online Spatial Event Forecasting in Microblogs.", ACM Transactions on Spatial Algorithms and Systems (TSAS), Volume 2 Issue 4, Acticle No. 15, pp. 1-39, November 2016.

    [41] Fang Jin, Rupinder Khandpur, Nathan Self, Edward Dougherty, Sheng Guo, Feng Chen, B. Aditya Prakash, Naren Ramakrishnan. Modeling Mass Protest Adoption in Social Network Communities using Geometric Brownian Motion, in Proceedings of the 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD'14), Aug 2014.

    [42] Ting Hua, Chandan Reddy, Lijing Wang, Liang Zhao, Lei Zhang, Chang-Tien Lu, and Naren Ramakrishnan. Social Media based Simulation Models for Understanding Disease Dynamics. the 27th International Joint Conference on Artificial Intelligence (IJCAI 2018) (acceptance rate: 20.6%), Stockholm, Sweden, Jul 2018, to appear.

    [43] Kaiqun Fu, Taoran Ji, Liang Zhao, and Chang-Tien Lu."TITAN: A Spatiotemporal Feature Learning Framework for Traffic Incident Duration Prediction",��the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems 2019 (SIGSPATIAL 2019), long paper, (acceptance rate: 21.7%), Chicago, Illinois, USA, to appear.

    [44] Liang Zhao, Olga Gkountouna, and Dieter Pfoser. Spatial Auto-regressive Dependency Interpretable Learning Based on Spatial Topological Constraints. ACM Transactions on Spatial Algorithms and Systems (TSAS), to appear.

    [45] Liang Zhao, Jiangzhuo Chen, Feng Chen, Fang Jin, Wei Wang, Chang-Tien Lu, and Naren Ramakrishnan. Online Flu Epidemiological Deep Modeling on Disease Contact Network. GeoInformatica (impact factor: 2.392), to appear.

    [46] Liang Zhao, Feng Chen, and Yanfang Ye. Efficient Learning with Exponentially-Many Conjunctive Precursors for Interpretable Spatial Event Forecasting. IEEE Transactions on Knowledge and Data Engineering (TKDE), (impact factor: 2.775), to appear, 2019.

    [47] Maya Okawa, Tomoharu Iwata, Takeshi Kurashima, Yusuke Tanaka, Hiroyuki Toda, Naonori Ueda. Deep Mixture Point Processes: Spatio-temporal Event Prediction with Rich Contextual Information, in Proceedings of the 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD'19), Aug 2019.

    [48] Songgaojun Deng, Huzefa Rangwala, Yue Ning. Learning Dynamic Context Graphs for Predicting Social Events, in Proceedings of the 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD'19), Aug 2019.

    [49] Renhe Jiang, Xuan Song, Dou Huang, Xiaoya Song, Tianqi Xia, Zekun Cai, Zhaonan Wang, Kyoung-Sook Kim and Ryosuke Shibasaki. DeepUrbanEvent: A System for Predicting Citywide Crowd Dynamics at Big Events, in Proceedings of the 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD'19), Aug 2019.

    Datasets

    Name
    		 Descriptions

    Papers
    Influenza Event Forcasting Datasets Forecast the occurrence of future influenza
    outbreaks    		 [TKDE] [Geoinformatica]
    Civil Unrest Event Forecasting Datasets Forecast the occurrence of future protests [KDD 2015]
    Traffic Flow Forecasting Forecast traffic flow [TSAS]
    Water Pollution Forecasting Predict water quality [TSAS]
    Event Scale Forecasting Forecast the scales of future events [AAAI 2018]
    Event Subtype Forecasting Forecast the subtypes of future events [AAAI 2019]
    Hyperlocal Price Event Forecasting* User real-time crowdsourced commodity prices
    to forecast social events    		 [CIKM 2017]
    Online Activism Platforms Forecast whether an online petition will succeed [ICDM 2018]
    *The dataset websites will be available soon.

    Tutorials

  • Ning, Yue, Liang Zhao, Feng Chen, Chang-Tien Lu, and Huzefa Rangwala. "Spatio-temporal Event Forecasting and Precursor Identification." In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3237-3238. ACM, 2019.

  • Liang Zhao and Feng Chen. "Big Data Analytics for Societal Event Forecasting", in IEEE International Conference on Big Data (IEEE BigData 2018), December 10, 2018, Seattle, WA, USA.

    • Surveys

  • Liang Zhao. Event Prediction in the Big Data Era: A Systematic Survey. ACM Computing Surveys. 54, 5, Article 94 (June 2021), 37 pages. DOI: https://doi.org/10.1145/3450287


    • Introduction

    Social Media is the use of electronic and Internet tools for the purpose of sharing and discussing information and experiences with other human beings in more efficient ways. Social media can be considered as the sensor of the society and hence effective utilization of such data is highly beneficial to the social good.

    Data analytics in social media is the process of representing, analyzing, and extracting meaningful patterns from social media data. In recent decades, this topic has attracted extensive attention. However, data analytics in social media is very challenging due to unique characteristics of social media data, as illustrated in the following table. Our group have been working on handling various major challenges of social media mining where the methods, codes, and datasets are shared to the public in the following.

    Methods and Codes

    Data Challenges
    		 Research Problems

    Methods and Codes
    Dynamics Dynamic inputs New features (e.g., new hashtags) auto-detection

    Dynamic Query Expansion
    [KDD 14][PloS ONE]
    [ComCom]
    Dynamic outputs Open Set Detection (e.g., new hot topics to detect) Open Set Recognition
    [CIKM 19]
    Noisiness

    Corrupted inputs

    		Leverage cleaner data in other domains

    Transfer Learning
    [WWW 18] [JBMS]
    Corrupted outputs Auto-detect corruption data while training Robust Machine Learning
    [CIKM 17][ICDM 17]
    [IJCAI 18][ICDM 18]
    [IJCAI 17]
    Incomplete inputs Complete the missing inputs while training Multi-task learning
    [KDD 16][CIKM 17]
    Incomplete outputs Complete the missing labels while training Semi-supervised learning
    [AAAI 19][ICDM 18]
    [AAAI 18]
    Streaming

    Incremental samples

    		 Update models with new training samples on the fly

    Online learning
    [ICDM 15][Geoinformatica]
    [TSAS]

    Incremental features Update models with new features on the fly Incremental feature learning
    [ICDM 18][ICDM 18]
    Heterogeneity Spatial heterogeneity Jointly handle spatial dependency and heterogeneity

    Spatial Multitask learning
    [KDD 15][TKDE][PIEEE]

    Temporal heterogeneity Different data sources have different time durations Temopral Multask learning
    [KDD 16]
    Network heterogeneity Multi-type entities in social media are connected Heterogeneous info network
    [CIKM 19][IJCAI 19]
    [WWW 19]
    Text heterogeneity Postings have different languages Multilingual learning
    [AAAI 18]
    High-dimension

    Training efficiency

    		 Efficienct selecting exponentially-many features

    Active set methods
    [TKDE]
    Predicting efficiency Jointly maximize prediction accuracy and efficiency Multi-objective learning
    [KDD 18]
    Multi-scale

    Multi-scale input

    		 Different postings come with different geo-resolution

    Multi-level models
    [KDD 16]
    Multi-scale output Different tasks require different geo-resolution Multi-resolution models
    [ICDM 16]

    Datasets

    Name
    		 Descriptions

    Papers
    Online Activism Platforms Online petition website big data [ICDM 2018]
    Online Health Forums Breast cancer communication large network [ICDM 2019]
    Influenza Tweet Datasets Influenza outbreaks indicated by tweets [ICDM 2015] [TKDE]
    Vaccine Adverse Event Datasets* Flu adverse reaction detection by tweets [WWW 2018]
    Protest Tweet Datasets Tweets that are indicative of future protests [KDD 2015] [KDD 2016]
    Crowdsourcing Prcie Datasets* Hyperlocal price data collected by crowdsourcing [CIKM 2017]
    Multilingual Tweet Datasets Multilingual social indicators for event forecasting [AAAI 2018]
    Rescue Tweet Datasets* Open-world situational awareness by tweets [CIKM 2019]
    Social Coding Platforms Stack exchange webste family [IJCAI 2019]

    *The dataset websites will be available soon.


    Introduction

    Currently most of the training strategies for deep neural networks (DNNs) are based on gradient descent. Although they are popular, they are far from being perfect and people are complaining about the drawbacks such as gradient-vanishing, poor conditioning, biological implausibility, and low concurrency.
    Gradient-free optimization for training DNN contains a number of young yet promising topics. For example, we are focusing on alternating optimization based optimizers for training deep neural networks, which first transfer a DNN training problem:

    $$\small \min_{W_l,b_l,a_l} = R(f_l(W_l\cdot f_{l-1}(\cdots)+b_l);y)+\sum\nolimits_{l=1}^L\Omega_l(W_l)$$

    into the following equivalent problem:

    $$\small \min_{W_l,b_l,z_l,a_l} = R(z_L;y)+\sum\nolimits_{l=1}^L\Omega_l(W_l)$$ $$\small s.t.\ \ z_l=W_la_{l-1}+b_l,\ \ (l=1,\cdots,L-1)$$$$\small\ a_l=f_l(z_l)\ \ (l=1,\cdots,L-1)$$

    where \(W,b,z,a\) are weights, bias, auxiliary variables, and activations, respectively. \(f_l(\cdot)\) is the \(l\)-th layer's activation function. \(R(\cdot)\) and \(\Omega(\cdot)\) are empirical loss and regularization term (if at all), respectively.

    Methods and Codes

    However, the above equivalent form is highly nonconvex with nonconvex constraints and is difficult to solve with good theoretical properties. Therefore, existing works typically further do relaxation or asymptotic approximation with different forms as follows*:

    Method
    		 Asymptotic Approximation Strategy Methods

    Codes
    ADMMNets $$\small \min_{W_l,b_l,z_l,a_l}R(z_L;y)+\langle z_L,\lambda\rangle+\beta_L\|z_L-WLa_{L-1}\|_2^2$$$$\small+\sum\nolimits_{l=1}^{L-1}(\gamma_l\|a_l-h_l(z_l)\|_2^2+\beta_l\|a_l-W_la_{l-1}\|_2^2)$$ [ICML 2016] [code]
    dlADMM $$\small \min_{W_l,b_l,z_l,a_l}R(z_L;y)+\sum\nolimits_{l=1}^L\Omega_l(W_l)+(v/2)\sum\nolimits_{l=1}^{L-1}(\|z_l-W_la_{l-1}-b_l\|_2^2+\|a_l-f_l(z_l)\|_2^2)$$ $$\small s.t.,\ z_L=W_La_{L-1}+b_L$$ [KDD 2019] [code]
    DLAM $$\small \min_{W_l,b_l,z_l,a_l}R(z_L;y)+\sum\nolimits_{l=1}^L\Omega_l(W_l)+(v/2)\sum\nolimits_{l=1}^{L}\frac{\rho}{2}\|z_l-W_la_{l-1}-b_l\|_2^2$$$$\small+\sum\nolimits_{l=1}^{L-1}\mathbb{I}(h_l(z_l)-\varepsilon\le a_l\le h_l(z_l)+\varepsilon)$$ [Arxiv] [code]
    (available soon)

    *As this is a promising and active domain, we will keep updating new methods and codes into this table all the time.

    Ongoing Grants

  • NSF 2318829 (Co-PI, PI: William West): Collaborative Research: IIBR: Innovation: Bioinformatics: Linking Chemical and Biological Space: Deep Learning and Experimentation for Property-Controlled , $319,050, 2023-2026, National Science Foundation.
  • NSF 2007716 (sole-PI): III: Small: Deep Generative Models for Temporal Graph Generation and Interpretation, $498,050, 2020-2023, National Science Foundation.
  • DHS-CINA project (PI): "Cross-platforms Cybercrime Detection on Inter-connected Heterogeneous Networks", $322,828. 2022-2024, Criminal Investigations and Network Analysis, DHS.
  • Computing Innovation Fellows Program (PI), with CIFellow Yun Li: Utilizing Deep Learning and Multi-Source Earth Observations for Air Quality Data Reconstruction and Downscaling, $257,681, 2021-2023, Computing Research Association (CRA) and the Computing Community Consortium (CCC).
  • Oracle for Research Grant Award (sole-PI): Steering Deep Model Reasoning with Visual Explanation Intervention, $58,973 2023-2024, Oracle Corporation.
  • Cisco Faculty Research Award (sole-PI) on Ethics in AI, $68,743 2023-2024, Cisco Inc.
  • NSF 2007976 (PI): OAC Core: DeepJIMU: Model-Parallelism Infrastructure for Large-scale Deep Learning by Gradient-Free Optimization, NSF, $499,800, 2020-2023, National Science Foundation.
  • Knowledge Design Company (Co-PI): Secure Model and Learning protected Hardware Design, $510,000, 2020-2022, Knowledge Design Company.
  • NSF 1942594 (sole-PI): III: CAREER: Spatial Network Deep Generative Modeling, Transformation, and Interpretation, $549,656, 2020-2025, National Science Foundation.
  • Amazon Research Award (PI): Distributed Large-scale Graph Deep Learning by Gradient-free Optimization, $75,000, Amazon Science.
  • NSF 1907805 (PI): III: Small: Graph Generative Deep Learning for Protein Structure Prediction, NSF, $499,800, 2019-2023, National Science Foundation.
  • Research Gift (PI): "Graph Neural Networks for Natural Language Processing", $90,000/Year, NEC Lab America.
  • NSF 1841520 (Co-PI): "Phase II I/UCRC: Center for Spatiotemporal Thinking, Computing and Applications", $750,000. 2019-2024, National Science Foundation.

    • Acknowledgement