US Patent 9715486 Annotation probability distribution based on a factor graph

In order to address annotation bias in batch annotations, obtained via crowdsourcing, on a set of comments on user posts in a social network, a system determines an annotation probability distribution based on a factor-graph model of the batch annotations. In particular, during operation the system computes the factor-graph model that represents relationships between feature vectors that represent the comments and the annotations for the comments. Note that, for a given batch of k comments, the factor-graph model may include a statistically dependent combination of statistically independent models of the interrelationships between the feature vectors and the annotations for the k comments. Then, the system calculates the annotation probability distribution based on model parameters associated with the factor-graph model, a mapping function that maps from the feature vectors to the annotations, and an indicator function that represents the annotations for the comments in the batches.