python & snmp
用python獲取snmp信息有多個現(xiàn)成的庫可以使用，其中比較常用的是netsnmp和pysnmp兩個庫。網(wǎng)上有較多的關(guān)于兩個庫的例子。
本文重點在于如何并發(fā)的獲取snmp的數(shù)據(jù)，即同時獲取多臺機(jī)器的snmp信息。
netsnmp
先說netsnmp。python的netsnmp，其實是來自于net-snmp包。
python通過一個c文件調(diào)用net-snmp的接口獲取數(shù)據(jù)。
因此，在并發(fā)獲取多臺機(jī)器的時候，不能夠使用協(xié)程獲取。因為使用協(xié)程，在get數(shù)據(jù)的時候，協(xié)程會一直等待net-snmp接口返回數(shù)據(jù)，而不會像socket使用時那樣在等待數(shù)據(jù)時把CPU切換給其他協(xié)程使用。從這點上來說，使用協(xié)程和串行獲取沒有區(qū)別。
那么如何解決并發(fā)獲取的問題呢？可以使用線程，多線程獲取(當(dāng)然也可以使用多進(jìn)程)。多個線程同時調(diào)用net-snmp的接口獲取數(shù)據(jù)，然后cpu在多個線程之間不停切換。當(dāng)一個線程獲取一個結(jié)果后，可以繼續(xù)調(diào)用接口獲取下一個snmp數(shù)據(jù)。
這里我寫了一個樣例程序。首先把所有的host和oid做成任務(wù)放到隊列里，然后啟動多個線程，去執(zhí)行獲取任務(wù)。程序樣例如下：
import threadingimport timeimport netsnmpimport Queuestart_time = time.time()hosts = ["192.20.150.109", "192.20.150.110", "192.20.150.111", "192.20.150.112", "192.20.150.113", "192.20.150.114", "192.20.150.115", "192.20.150.116", "192.20.150.117", "192.20.150.118", "192.20.150.119", "192.20.150.120", "192.20.150.121", "192.20.80.148", "192.20.80.149", "192.20.96.59", "192.20.82.14", "192.20.82.15", "192.20.82.17", "192.20.82.19", "192.20.82.12", "192.20.80.139", "192.20.80.137", "192.20.80.136", "192.20.80.134", "192.20.80.133", "192.20.80.131", "192.20.80.130", "192.20.81.141", "192.20.81.140", "192.20.82.26", "192.20.82.28", "192.20.82.23", "192.20.82.21", "192.20.80.128", "192.20.80.127", "192.20.80.122", "192.20.81.159", "192.20.80.121", "192.20.80.124", "192.20.81.151", "192.20.80.118", "192.20.80.119", "192.20.80.113", "192.20.80.112", "192.20.80.116", "192.20.80.115", "192.20.78.62", "192.20.81.124", "192.20.81.125", "192.20.81.122", "192.20.81.121", "192.20.82.33", "192.20.82.31", "192.20.82.32", "192.20.82.30", "192.20.81.128", "192.20.82.39", "192.20.82.37", "192.20.82.35", "192.20.81.130", "192.20.80.200", "192.20.81.136", "192.20.81.137", "192.20.81.131", "192.20.81.133", "192.20.81.134", "192.20.82.43", "192.20.82.45", "192.20.82.41", "192.20.79.152", "192.20.79.155", "192.20.79.154", "192.25.76.235", "192.25.76.234", "192.25.76.233", "192.25.76.232", "192.25.76.231", "192.25.76.228", "192.25.20.96", "192.25.20.95", "192.25.20.94", "192.25.20.93", "192.24.163.14", "192.24.163.21", "192.24.163.29", "192.24.163.6", "192.18.136.22", "192.18.136.23", "192.24.193.2", "192.24.193.19", "192.24.193.18", "192.24.193.11", "192.20.157.132", "192.20.157.133", "192.24.212.232", "192.24.212.231", "192.24.212.230"]oids = [".1.3.6.1.4.1.2021.11.9.0",".1.3.6.1.4.1.2021.11.10.0",".1.3.6.1.4.1.2021.11.11.0",".1.3.6.1.4.1.2021.10.1.3.1", ".1.3.6.1.4.1.2021.10.1.3.2",".1.3.6.1.4.1.2021.10.1.3.3",".1.3.6.1.4.1.2021.4.6.0",".1.3.6.1.4.1.2021.4.14.0", ".1.3.6.1.4.1.2021.4.15.0"]myq = Queue.Queue()rq = Queue.Queue()#把host和oid組成任務(wù)for host in hosts: for oid in oids: myq.put((host,oid))def poll_one_host(): while True: try: #死循環(huán)從隊列中獲取任務(wù)，直到隊列任務(wù)為空 host, oid = myq.get(block=False) session = netsnmp.Session(Version=2, DestHost=host, Community="cluster",Timeout=3000000,Retries=0) var_list = netsnmp.VarList() var_list.append(netsnmp.Varbind(oid)) ret = session.get(var_list) rq.put((host, oid, ret, (time.time() - start_time))) except Queue.Empty: breakthread_arr = []#開啟多線程num_thread = 50for i in range(num_thread): t = threading.Thread(target=poll_one_host, kwargs={}) t.setDaemon(True) t.start() thread_arr.append(t)#等待任務(wù)執(zhí)行完畢f(xié)or i in range(num_thread): thread_arr[i].join()while True: try: info = rq.get(block=False) print info except Queue.Empty: print time.time() - start_time breaknetsnmp除了支持get操作之外，還支持walk操作，即遍歷某個oid。
但是walk使用的時候需要謹(jǐn)慎，以免導(dǎo)致高延時等問題，具體可以參見之前的一篇snmpwalk高延時問題分析的博客。
pysnmp
pysnmp是用python實現(xiàn)的一套snmp協(xié)議的庫。其自身提供了對于異步的支持。
import timeimport Queuefrom pysnmp.hlapi.asyncore import *t = time.time()myq = Queue.Queue()#回調(diào)函數(shù)。在有數(shù)據(jù)返回時觸發(fā)def cbFun(snmpEngine, sendRequestHandle, errorIndication, errorStatus, errorIndex, varBinds, cbCtx): myq.put((time.time()-t, varBinds))hosts = ["192.20.150.109", "192.20.150.110", "192.20.150.111", "192.20.150.112", "192.20.150.113", "192.20.150.114", "192.20.150.115", "192.20.150.116", "192.20.150.117", "192.20.150.118", "192.20.150.119", "192.20.150.120", "192.20.150.121", "192.20.80.148", "192.20.80.149", "192.20.96.59", "192.20.82.14", "192.20.82.15", "192.20.82.17", "192.20.82.19", "192.20.82.12", "192.20.80.139", "192.20.80.137", "192.20.80.136", "192.20.80.134", "192.20.80.133", "192.20.80.131", "192.20.80.130", "192.20.81.141", "192.20.81.140", "192.20.82.26", "192.20.82.28", "192.20.82.23", "192.20.82.21", "192.20.80.128", "192.20.80.127", "192.20.80.122", "192.20.81.159", "192.20.80.121", "192.20.80.124", "192.20.81.151", "192.20.80.118", "192.20.80.119", "192.20.80.113", "192.20.80.112", "192.20.80.116", "192.20.80.115", "192.20.78.62", "192.20.81.124", "192.20.81.125", "192.20.81.122", "192.20.81.121", "192.20.82.33", "192.20.82.31", "192.20.82.32", "192.20.82.30", "192.20.81.128", "192.20.82.39", "192.20.82.37", "192.20.82.35", "192.20.81.130", "192.20.80.200", "192.20.81.136", "192.20.81.137", "192.20.81.131", "192.20.81.133", "192.20.81.134", "192.20.82.43", "192.20.82.45", "192.20.82.41", "192.20.79.152", "192.20.79.155", "192.20.79.154", "192.25.76.235", "192.25.76.234", "192.25.76.233", "192.25.76.232", "192.25.76.231", "192.25.76.228", "192.25.20.96", "192.25.20.95", "192.25.20.94", "192.25.20.93", "192.24.163.14", "192.24.163.21", "192.24.163.29", "192.24.163.6", "192.18.136.22", "192.18.136.23", "192.24.193.2", "192.24.193.19", "192.24.193.18", "192.24.193.11", "192.20.157.132", "192.20.157.133", "192.24.212.232", "192.24.212.231", "192.24.212.230"]oids = [".1.3.6.1.4.1.2021.11.9.0",".1.3.6.1.4.1.2021.11.10.0",".1.3.6.1.4.1.2021.11.11.0",".1.3.6.1.4.1.2021.10.1.3.1", ".1.3.6.1.4.1.2021.10.1.3.2",".1.3.6.1.4.1.2021.10.1.3.3",".1.3.6.1.4.1.2021.4.6.0",".1.3.6.1.4.1.2021.4.14.0", ".1.3.6.1.4.1.2021.4.15.0"] snmpEngine = SnmpEngine()#添加任務(wù)for oid in oids: for h in hosts: getCmd(snmpEngine, CommunityData('cluster'), UdpTransportTarget((h, 161), timeout=3, retries=0,), ContextData(), ObjectType(ObjectIdentity(oid)), cbFun=cbFun)time1 = time.time() - t#執(zhí)行異步獲取snmpsnmpEngine.transportDispatcher.runDispatcher()#打印結(jié)果while True: try: info = myq.get(block=False) print info except Queue.Empty: print time1 print time.time() - t breakpysnmp本身只支持最基礎(chǔ)的get和getnext命令，因此如果想使用walk，需要自己進(jìn)行實現(xiàn)。
性能測試
在同一個環(huán)境下，對兩者進(jìn)行了性能測試。兩者對198個host，10個oid進(jìn)行采集。
| 測試組 | 耗時(sec) |
|---|---|
| netsnmp(20線程) | 6.252 |
| netsnmp(50線程) | 3.269 |
| netsnmp(200線程) | 3.265 |
| pysnmp | 4.812 |
可以看到netsnmp的采集速度跟線程數(shù)有關(guān)。當(dāng)線程數(shù)增大到一定程度，采集時間不再縮短。因為開辟線程同樣會消耗時間。而已有的線程已經(jīng)足夠處理。
pysnmp性能較之略差一下。詳細(xì)分析pysnmp在添加任務(wù)(執(zhí)行g(shù)etCmd時)消耗了約1.2s，之后的采集約消耗3.3秒。
在增加了oid數(shù)，在進(jìn)行實驗。host仍然是198個，oid是42個。
| 測試組 | 耗時(sec) |
|---|---|
| netsnmp(20線程) | 30.935 |
| netsnmp(50線程) | 12.914 |
| netsnmp(200線程) | 4.044 |
| pysnmp | 11.043 |
可以看到差距被進(jìn)一步拉大。在線程足夠多的情況下，netsnmp的效率要明顯強(qiáng)于pysnmp。
因為二者都支持可以并行采集多個host，從易用性來說，netsnmp更為簡單一些，且netsnmp支持walk功能。本文更加推薦netsnmp。
安裝netsnmp需要安裝net-snmp。如果centos，則使用yum會較為方便。
以上就是本文的全部內(nèi)容，希望對大家的學(xué)習(xí)有所幫助，也希望大家多多支持武林網(wǎng)。
新聞熱點
疑難解答
圖片精選
