Protein language models and social behavior models share more than you think
Q1I've been working on protein variant prediction using transformers, and recently attended a talk on behavioral modeling for social networks. The parallels are uncanny: both deal with variable-length sequences, both rely on attention to capture long-range dependencies, and both face the challenge of out-of-distribution generalization. A rare mutation in a protein is statistically analogous to unusual user behavior.
Q2Exactly. And here's where I think the social domain could learn from bioinformatics: we use evolutionary conservation scores to estimate a mutation's likely impact. Highly conserved positions are functionally important, so mutations there are high-impact. Could social platforms use a similar 'behavioral conservation' concept to identify which user actions are most indicative of intent?