论文阅读记录

IE-HERCL

• 标题：Image-Enhanced Hybrid Encoding with Reinforced Contrastive Learning for Spatial Domain Identification in Spatial Transcriptomics
• 作者：Central South University
• 发表会议/期刊：IJCAI
• 年份：2025
• 链接：IE-HERCL

主要内容简介

• Motivation:

  Exisisting methods fail to account for complex interdependencies between modalities.

方法与创新点

• 总体框架：

• 方法概述：

  1. Multimodal Feature Representation Learning

     • utilize AE to extract & Loss function L_rec:

       $$
       L^g_{rec} = | X_{g,i} - f^g_{\theta_d}(f^g_{\theta_e}(X_{g,i})) |^2_F
       $$

     • ultilize ResNet50 to extract $Z_{net}$ then use AE:
       

     • GraphSAGE encoder:

       $$
       Z^v_i = \sigma(W_e \cdot \text{Concat}(X_i, \text{Aggregate}({X_j \mid j \in N(i)})))
       $$

     • intra-modal attention & cross-modal attention:

       • Intra-modal attention:
         $$
         Z_f^g = \alpha_{ae_g} Z_g + \alpha_{gs_g} Z_{gs}^g
         $$

         $$
         Z_f^{img} = \alpha_{ae}^{img} Z_{img} + \alpha_{gs_img} Z^{img}_{gs}
         $$

       • Cross-modal attention:
         $$
         Z = \beta_{g} Z^g_f + \beta_{img} Z^{img}_f
         $$

     • Reconstruction:

     $$
     \hat{X}_{g,i} = \sigma(W_d \cdot \text{Concat}(Z_i, \text{Aggregate}({Z_j \mid j \in N(i)})))
     $$

     $$
     L_{rec}^{gene} = | X_{g,i} - \hat{X}_{g,i} |^2_F
     $$

  2. Negative Sample Mitigation Strategy in Contrastive Learning

     • contrastive loss :
       
       

  3. Optimal Transport-Based Representation Optimization

     • Clustering and Auxiliary Distributions
       

     • Optimal Transport Objective
       
       

     • 借鉴way博客补充最优传输的思想：
       

     • Overall Optimization Objective
       

实验

• Evaluation Metrics：
  • 调整兰德指数(ARI)、标准化互信息(NMI)、调整互信息(AMI)、Fowlkes-Mallows指数(FMI)和同质性评分(HS)