Ligong Han   韩立功

I am a Research Scientist at the MIT-IBM Watson AI Lab, working on Generative AI with a focus on controllable and precise generation, particularly in diffusion models and LLMs. I obtained my PhD in Computer Science from Rutgers University in 2024, advised by Prof. Dimitris Metaxas. During my PhD, I've spent time at Google Research, MIT-IBM Watson AI Lab, Snap Research, NEC Labs America, Tencent, and the Robotics Institute working as a research intern.

Previously, I earned my master's degree from Carnegie Mellon University and my bachelor's from Chien-Shiung Wu College, Southeast University.

Email: lastnamefirstname [at] gmail [dot] com or firstname.lastname [at] rutgers [dot] edu

Email  /  CV  /  Google Scholar  /  Github  /  LinkedIn  /  Twitter

Selected publications are highlighted. (* equal contribution, † corresponding author)

TLDR: Discrete diffusion inversion for precise and flexible content editing by recording noise sequences and masking patterns during the reverse process.

TLDR: An automatic prompt engineering algorithm for LLM-based relevance ranking in IR, significantly reducing human effort and outperforming existing manual prompts.

TLDR: We derive a novel Bayes by Backprop (BBB) framework for Low Rank Adaptation (LoRA) of Large Language Models (LLMs).

TLDR: A framework for parameter-efficient fine-tuning by adjusting both singular values and their basis vectors, balancing computational efficiency and representation capacity.

TLDR: Improves upon traditional In-context Learning by extracting a context vector from demonstration examples and injecting it into the model's activation space, achieving few-shot performance with zero-shot cost.

TLDR: Solving inverse problems for 3D human pose and shape reconstruction with score guidance in the latent space of a diffusion model.

TLDR: We introduced proximal guidance to enhance diffusion-based tuning-free real image editing in two frameworks, Negative Prompt Inversion and Mutual Self-Attention Control. Our algorithms, ProxNPI and ProxMasaCtrl, overcome limitations and achieve high-quality editing with computational efficiency.

TLDR: This paper presents a theoretical framework and a novel algorithm for Continual Meta-Learning (CML) that effectively balances stability to prevent forgetting previous tasks and plasticity for learning from new tasks.

TLDR: This paper presents an approach for customizing (T2I) diffusion models, fine-tuning singular values of weight matrices to reduce overfitting and model storage, while introducing a text-based single-image editing framework and a data-augmentation technique for multi-subject generation.

TLDR: We propose a morphology-guided diffusion model for 3D cardiac volume reconstruction, by interpolation in its latent space. The model outperforms strong baselines including GAN- and DiffAE-based methods.

TLDR: This study proposes a method to assimilate generated views with expert transformations in contrastive learning, improving the state-of-the-art by up to 3.6% on three datasets and providing a comprehensive analysis of various view generation and assimilation methods.

TLDR: This work proposes a model-based guidance technique for single-image editing using pre-trained diffusion models, addressing overfitting issues and enabling content creation with only one given image, while also introducing a patch-based fine-tuning method for generating images of arbitrary resolution.

TLDR: The paper introduces a method for monocular 3D reconstruction of articulated objects with minimal labeled data, using category-specific keypoint estimators and data selection to improve performance.

TLDR: This paper demonstrates the use of score distillation sampling as a critic to adapt GAN generators to new domains using text prompts, leveraging large-scale text-to-image diffusion models and achieving high quality and controllability in domain adaptation for both 2D and 3D image generation.

TLDR: The paper presents a multimodal video generation framework using a bidirectional transformer and improved techniques to generate high-quality, diverse video sequences, achieving state-of-the-art results on four datasets.

TLDR: Training a style-based autoencoder end-to-end, resulting in a more disentangled latent space and improved image inversion and generation quality.

TLDR: The paper proposes a Counterfactual Self-Training (CST) algorithm that uses pseudolabeling to address the challenge of partial feedback in settings like pricing, online marketing, and precision medicine, treating it as a domain adaptation problem, and demonstrates its effectiveness on both synthetic and real datasets.

TLDR: The paper introduces a self-supervised hierarchical pre-training scheme with Hi-TRS, capturing multi-level dependencies and achieving state-of-the-art performance in skeleton-based tasks.

TLDR: R-WAE is a framework for unsupervised disentangled sequential representation learning, outperforming baselines in disentanglement and video generation by optimizing Wasserstein distance and mutual information.

TLDR: Dual Projection GAN (P2GAN) balances data matching and label matching in cGANs, improving class separability and sample quality.

TLDR: The paper introduces a novel human-machine collaboration approach in a bandit feedback setting that improves decision-making performance by exploiting human-machine complementarity and personalizing routing for multiple human decision-makers.

TLDR: PC-GAN is a novel generative adversarial network using weak supervision (via the proposed Elo rating network) from pairwise comparisons for image attribute editing, achieving robust performance and comparable results to fully-supervised methods.

TLDR: We propose Unbiased Auxiliary GANs (UAC-GAN) that leverage the Mutual Information Neural Estimator (MINE) and a novel projection-based statistics network architecture to address the biased distribution issue in AC-GANs, resulting in improved performance on three datasets.

TLDR: We propose a Renyi entropy regularization (RER) framework for unsupervised domain adaptation, which adapts from source to target domain by calibrating predictive uncertainties using variational Bayes learning, and demonstrate its effectiveness on three domain-adaptation tasks.

TLDR: We propose a two-stage and an end-to-end supervised GAN approaches for generating realistic electron microscope images with densely annotated sub-cellular structures.

Align axis labels nicely in parallel with axes in MATLAB (3-D) plots. This file was selected as MATLAB Central Pick of the Week.

Website source from Jon Barron.