在進行文本分析、提取關(guān)鍵詞時，新聞評論等文本通常是中英文及其他語言的混雜，若不加處理直接分析，結(jié)果往往差強人意。
下面對中英文文本進行分離做一下總結(jié)：
1、超短文本，ASCII識別。
s = "China's Legend Holdings will split its several business arms to go public on stock markets, the group's president Zhu Linan said on Tuesday.該集團總裁朱利安周二表示，中國聯(lián)想控股將分拆其多個業(yè)務(wù)部門在股市上市。"result = "".join(i for i in s if ord(i) < 256)print(result)
out:China's Legend Holdings will split its several business arms to go public on stock markets, the group's president Zhu Linan said on Tuesday.
2、unicode編碼識別
import res = "China's Legend Holdings will split its several business arms to go public on stock markets, the group's president Zhu Linan said on Tuesday.該集團總裁朱利安周二表示，中國聯(lián)想控股將分拆其多個業(yè)務(wù)部門在股市上市。"uncn = re.compile(r'[/u0061-/u007a,/u0020]')en = "".join(uncn.findall(s.lower()))print(en)
out:chinas legend holdings will split its several business arms to go public on stock markets, the groups president zhu linan said on tuesday
中文的編碼范圍是：/u4e00-/u9fa5，相應(yīng)的[^/u4e00-/u9fa5]可匹配非中文。
匹配英文時，需要將空格[/u0020]加入，不然單詞之間沒空格了。
以上這篇利用Python將文本中的中英文分離方法就是小編分享給大家的全部內(nèi)容了，希望能給大家一個參考，也希望大家多多支持VEVB武林網(wǎng)。
新聞熱點
疑難解答
圖片精選
