本文實例講述了html">Python基于lxml模塊解析html獲取頁面內所有葉子節點xpath路徑功能。分享給大家供大家參考，具體如下：

因為需要使用葉子節點的路徑來作為特征，但是原始的lxml模塊解析之后得到的卻是整個頁面中所有節點的xpath路徑，不是我們真正想要的形式，所以就要進行相關的處理才行了，差了很多網上的博客和文檔也沒有找到一個是關于輸出html中全部葉子節點的API接口或者函數，也可能是自己沒有那份耐心，沒有找到合適的資源，只好放棄了尋找，但是這并不說明沒有其他的方法了，在對頁面全部節點的xpath輸出之后觀察得到的結果就是：

1.路徑之間存在包含性

2.葉子節點的路徑必然包含上一個葉子節點路徑到下一個葉子節點路徑之間的路徑

3.所有的葉子節點均不存在包含性

基于這些觀察就可以實踐了，我采用的方法是，設置一個標志位，如果當前路徑被下一條路徑包含的話就把下標加1處理，直到遇上不包含的情況是這一條路徑就是一條葉子節點路徑，加入結果列表即可，經檢驗和原始路徑列表對比后，結果路徑列表均不存在包含性，暫可認為均為葉子節點的路徑，不敢確定的說是因為：竟沒有大量做實驗觀察是否有另類，不過方法是可行的，下面是具體實現：

#!usr/bin/env python#encoding:utf-8'''''__author__:沂水寒城功能：得到頁面的葉子節點的xpath'''def get_leaf_node_xpath(one_page_xpath):  '''''  輸入：一個頁面的原始xpath路徑列表  輸出：只包含頁面的葉子節點的xpath列表  '''  one_page_xpath.append('0')  leaf_node_xpath_list=[]  for i in range(len(one_page_xpath)-1):    j=i+1    one_xpath=one_page_xpath[i]    two_xpath=one_page_xpath[j]    if one_xpath in two_xpath:      one_xpath=two_xpath      two_xpath=one_page_xpath[j+1]    else:      leaf_node_xpath_list.append(one_xpath)  return leaf_node_xpath_listif __name__ == '__main__':  with open('baidu.txt') as f:    html=f.read()  htree, one_page_xpath=get_clean_allnodes_xpath(html)  leaf_node_xpath_list=get_leaf_node_xpath(one_page_xpath)  print leaf_node_xpath_list  with open('data/dom_leaf_xpath/original_xpath.txt', 'w') as f1:    for one_line in one_page_xpath:      f1.write(one_line.strip()+'/n')  with open('data/dom_leaf_xpath/leaf_xpath.txt', 'w') as f2:    for one_line in leaf_node_xpath_list:      f2.write(one_line.strip()+'/n')

打開保存的文件內容為：

original_xpath.txt內容為：

/html
/html/head
/html/head/meta[1]
/html/head/meta[2]
/html/head/meta[3]
/html/head/meta[4]
/html/head/title
/html/body
/html/body/p
/html/body/p/comment()[1]
/html/body/p/comment()[2]
/html/body/p/comment()[3]
/html/body/p/meta
/html/body/div[1]
/html/body/div[1]/div[1]
/html/body/div[1]/div[1]/div
/html/body/div[1]/div[1]/div/div[1]
/html/body/div[1]/div[1]/div/div[1]/div
/html/body/div[1]/div[1]/div/div[1]/div/div[1]
/html/body/div[1]/div[1]/div/div[1]/div/a
/html/body/div[1]/div[1]/div/div[1]/div/form
/html/body/div[1]/div[1]/div/div[1]/div/form/span[1]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[2]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/div
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/div/span
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[1]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[1]/a
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[2]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[2]/a
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[3]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[4]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[4]/a
/html/body/div[1]/div[1]/div/div[1]/div/div[2]
/html/body/div[1]/div[1]/div/div[2]
/html/body/div[1]/div[1]/div/div[2]/a[1]
/html/body/div[1]/div[1]/div/div[2]/a[2]
/html/body/div[1]/div[1]/div/div[2]/a[3]
/html/body/div[1]/div[1]/div/div[3]
/html/body/div[1]/div[1]/div/div[3]/a[1]
/html/body/div[1]/div[1]/div/div[3]/a[2]
/html/body/div[1]/div[1]/div/div[3]/a[3]
/html/body/div[1]/div[1]/div/div[3]/a[4]
/html/body/div[1]/div[1]/div/div[3]/a[5]
/html/body/div[1]/div[1]/div/div[3]/a[6]
/html/body/div[1]/div[1]/div/div[3]/a[7]
/html/body/div[1]/div[1]/div/div[3]/a[8]
/html/body/div[1]/div[1]/div/div[3]/a[9]
/html/body/div[1]/div[2]
/html/body/div[1]/div[2]/a[1]
/html/body/div[1]/div[2]/a[2]
/html/body/div[1]/div[2]/a[3]
/html/body/div[1]/div[2]/a[4]
/html/body/div[1]/div[2]/a[5]
/html/body/div[1]/div[2]/a[6]
/html/body/div[1]/div[2]/a[7]
/html/body/div[1]/div[2]/a[8]
/html/body/div[1]/div[2]/a[9]
/html/body/div[1]/div[3]
/html/body/div[1]/div[3]/div
/html/body/div[1]/div[3]/div/div
/html/body/div[1]/div[3]/div/div/div[1]
/html/body/div[1]/div[3]/div/div/div[2]
/html/body/div[1]/div[3]/div/div/div[2]/p
/html/body/div[1]/div[4]
/html/body/div[1]/div[4]/div
/html/body/div[1]/div[4]/div/div
/html/body/div[1]/div[4]/div/div/p[1]
/html/body/div[1]/div[4]/div/div/p[1]/a[1]
/html/body/div[1]/div[4]/div/div/p[1]/a[2]
/html/body/div[1]/div[4]/div/div/p[1]/a[3]
/html/body/div[1]/div[4]/div/div/p[1]/a[4]
/html/body/div[1]/div[4]/div/div/p[2]
/html/body/div[1]/div[4]/div/div/p[2]/a[1]
/html/body/div[1]/div[4]/div/div/p[2]/a[2]
/html/body/div[1]/div[4]/div/div/p[2]/a[3]
/html/body/div[1]/div[5]
/html/body/div[2]
/html/body/div[3]
/html/body/div[4]
0

最后一行的0只是為了代碼中列表能夠遍歷到最后一個路徑而不出現索引錯誤的一個簡單的處理，人為的添加了一個元素，沒有實際意義，也不會參與處理

leaf_xpath.txt內容為：

/html/head/meta[1]
/html/head/meta[2]
/html/head/meta[3]
/html/head/meta[4]
/html/head/title
/html/body/p/comment()[1]
/html/body/p/comment()[2]
/html/body/p/comment()[3]
/html/body/p/meta
/html/body/div[1]/div[1]/div/div[1]/div/div[1]
/html/body/div[1]/div[1]/div/div[1]/div/a
/html/body/div[1]/div[1]/div/div[1]/div/form/span[1]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[2]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/div/span
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[1]/a
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[2]/a
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[3]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[4]/a
/html/body/div[1]/div[1]/div/div[1]/div/div[2]
/html/body/div[1]/div[1]/div/div[2]/a[1]
/html/body/div[1]/div[1]/div/div[2]/a[2]
/html/body/div[1]/div[1]/div/div[2]/a[3]
/html/body/div[1]/div[1]/div/div[3]/a[1]
/html/body/div[1]/div[1]/div/div[3]/a[2]
/html/body/div[1]/div[1]/div/div[3]/a[3]
/html/body/div[1]/div[1]/div/div[3]/a[4]
/html/body/div[1]/div[1]/div/div[3]/a[5]
/html/body/div[1]/div[1]/div/div[3]/a[6]
/html/body/div[1]/div[1]/div/div[3]/a[7]
/html/body/div[1]/div[1]/div/div[3]/a[8]
/html/body/div[1]/div[1]/div/div[3]/a[9]
/html/body/div[1]/div[2]/a[1]
/html/body/div[1]/div[2]/a[2]
/html/body/div[1]/div[2]/a[3]
/html/body/div[1]/div[2]/a[4]
/html/body/div[1]/div[2]/a[5]
/html/body/div[1]/div[2]/a[6]
/html/body/div[1]/div[2]/a[7]
/html/body/div[1]/div[2]/a[8]
/html/body/div[1]/div[2]/a[9]
/html/body/div[1]/div[3]/div/div/div[1]
/html/body/div[1]/div[3]/div/div/div[2]/p
/html/body/div[1]/div[4]/div/div/p[1]/a[1]
/html/body/div[1]/div[4]/div/div/p[1]/a[2]
/html/body/div[1]/div[4]/div/div/p[1]/a[3]
/html/body/div[1]/div[4]/div/div/p[1]/a[4]
/html/body/div[1]/div[4]/div/div/p[2]/a[1]
/html/body/div[1]/div[4]/div/div/p[2]/a[2]
/html/body/div[1]/div[4]/div/div/p[2]/a[3]
/html/body/div[1]/div[5]
/html/body/div[2]
/html/body/div[3]
/html/body/div[4]

針對這個問題暫時可以想到的方法就是這樣的，利用包含性來解決，沒有其他的新思路，如果還有新的思路歡迎交流學習！