Pandas字符串，替换没有for循环的多个单词[重复]

Question

在Pandas df中，我有大约130万个字符串（代表用户向IT帮助台发送邮件时的需求）。我还有一系列29813个名称，我想从这些字符串中删除，这样我只剩下描述问题的单词。这里有一个数据的小例子-它可以工作，但需要太长时间。我正在寻找一种更有效的方法来实现这一结果：

输入：

List1 = ["George Lucas has a problem logging in", 
         "George Clooney is trying to download data into a spreadsheet", 
         "Bart Graham needs to logon to CRM urgently", 
         "Lucy Anne George needs to pull management reports"]
List2 = ["Access Team", "Microsoft Team", "Access Team", "Reporting Team"]

df = pd.DataFrame({"Team":List2, "Text":List1})

xwords = pd.Series(["George", "Lucas", "Clooney", "Lucy", "Anne", "Bart", "Graham"])

for word in range(len(xwords)):
    df["Text"] = df["Text"].str.replace(xwords[word], "! ")

# Just using ! in the example so one can clearly see the result

输出：

Team                Text
0   Access Team     ! ! has a problem logging in
1   Microsoft Team  ! ! is trying to download data into a spreadsheet
2   Access Team     ! ! needs to logon to CRM urgently
3   Reporting Team  ! ! ! needs to pull management reports

我已经努力寻找答案很长一段时间了：如果我因为缺乏经验而错过了某个地方，请温柔地告诉我！

非常感谢：）

Answer 1

我建议标记文本并为名称使用一组：

xwords = set(["George", "Lucas", ...])
df["Text"] = ' '.join(filter(lambda x: x not in xwords, df["Text"].str.split(' ')))

根据字符串的不同，标记化需要比仅仅在空格上分割更复杂。

也许有一种熊猫特有的方式来做到这一点，但我对此没有什么经验；)

Answer 2

熊猫。系列.str.替换可以采用已编译的正则表达式作为模式

import re
patt = re.compile(r'|'.join(xwords))
df["Text"] = df["Text"].str.replace(patt, "! ")

也许这能有所帮助？不过，我没有使用如此长的正则表达式的经验。

Answer 3

感谢Ciprian Tomoiagé将我指向在Python 3中加速数百万正则表达式替换的帖子。Eric Duminil提供的选项，请参阅“如果您想要最快的解决方案，请使用此方法（带有设置查找）”，在Pandas环境中使用序列而不是列表同样有效-下面重复此问题的示例代码，在我的大型数据集上，整个事情在2.54秒内完成！

输入：

import re

banned_words = set(word.strip().lower() for word in xwords)

def delete_banned_words(matchobj):
    word = matchobj.group(0)
    if word.lower() in banned_words:
        return ""
    else:
        return word

sentences = df["Text"]

word_pattern = re.compile('\w+')

df["Text"] = [word_pattern.sub(delete_banned_words, sentence) for sentence in sentences]
print(df)

输出：

Team              Text
Access Team       has a problem logging in
Microsoft Team    is trying to download data into a spreadsheet
Access Team       needs to logon to CRM urgently
Reporting Team    needs to pull management reports