通过比较 Pandas 数据帧中的所有行同时跟踪正在比较的行来获取 Jaccard 相似度

Question

您好，我想获得数据框中所有行之间的杰卡德相似度。

我已经有了一个像下面这样的 jaccard 相似性函数，它接受两个列表，但我无法理解如何跟踪正在进行比较的用户。

def jaccard_similarity(x,y):
  """ returns the jaccard similarity between two lists """
  intersection_cardinality = len(set.intersection(*[set(x), set(y)]))
  union_cardinality = len(set.union(*[set(x), set(y)]))
  return intersection_cardinality/float(union_cardinality)

我想针对数据框中的所有行运行此函数。

措辞	用户
苹果、香蕉、橙子、梨	adeline
香蕉、菠萝蜜、浆果、苹果	ericko
浆果、葡萄、西瓜	玛丽

如何生成如下所示的输出，以便跟踪所比较的用户？

user1	user2	相似度
adeline	eriko	0.5
adeline	mary	0.2

非常感谢您的指导。

sentences = ['apple,banana,orange,pears', 'banana,jackfruit,berries,apple']
sentences = [sent.lower().split(",") for sent in sentences]
jaccard_similarity(sentences[0], sentences[1])

输出：0.3333333333333333

运行上面的代码将使我得到我想要的值，但我只是坚持如果我有 100 行数据，如何跟踪数据框中正在比较的用户。

谢谢

Answer 1

可能的解决方案如下：

import itertools
import pandas as pd

# copied from OP above
def jaccard_similarity(x, y):
    """ returns the jaccard similarity between two lists """
    intersection_cardinality = len(set.intersection(*[set(x), set(y)]))
    union_cardinality = len(set.union(*[set(x), set(y)]))
    return intersection_cardinality/float(union_cardinality)

# set initial data and create dataframe
data = {"wordings": ["apple,banana,orange,pears", "banana,jackfruit,berries,apple", "berries,grapes,watermelon"], "users": ["adeline", "ericko", "mary"]}
df = pd.DataFrame(data)

# create list of tuples like [(wording, user), (wording, user)]
wordings_users = list(zip(df["wordings"], df["users"]))

result = []

# create list of all possible combinations between sets of (wording, user) and loop through them
for item in list(itertools.combinations(wordings_users, 2)):
    similarity = jaccard_similarity(item[0][0], item[1][0])
    data = {"user1": item[0][1], "user2": item[1][1], "similarity": similarity}
    result.append(data)

df1 = pd.DataFrame(result)
df1

返回