国产探花免费观看_亚洲丰满少妇自慰呻吟_97日韩有码在线_资源在线日韩欧美_一区二区精品毛片,辰东完美世界有声小说,欢乐颂第一季,yy玄幻小说排行榜完本

首頁 > 學院 > 開發設計 > 正文

PythonRequests庫:HTTPforHumans

2019-11-14 17:36:19
字體:
來源:轉載
供稿:網友

Python標準庫中用來處理HTTP的模塊是urllib2,不過其中的API太零碎了,requests是更簡單更人性化的第三方庫。

用pip下載:

pip install requests

或者git:

git clone git://github.com/kennethreitz/requests.git

發送請求:

GET方法

>>> import requests>>> r = requests.get('https://api.github.com/events')

POST方法:

>>> r = requests.post("http://httpbin.org/post")

也可以使用其它方法:

>>> r = requests.put("http://httpbin.org/put")>>> r = requests.delete("http://httpbin.org/delete")>>> r = requests.head("http://httpbin.org/get")>>> r = requests.options("http://httpbin.org/get")

也可以將請求方法放在參數中:

>>> import requests>>> req = requests.request('GET', 'http://httpbin.org/get')

傳遞參數或上傳文件:

1.如果要將參數放在url中傳遞,使用params參數,可以是字典或者字符串:

>>> payload = {'key1': 'value1', 'key2': 'value2'}>>> r = requests.get("http://httpbin.org/get", params=payload)>>> r.urlu'http://httpbin.org/get?key2=value2&key1=value1'

2.如果要將參數放在request body中傳遞,使用data參數,可以是字典,字符串或者是類文件對象。

使用字典時將發送form-encoded data:

>>> payload = {'key1': 'value1', 'key2': 'value2'}>>> r = requests.post("http://httpbin.org/post", data=payload)>>> PRint(r.text){  ...  "form": {    "key2": "value2",    "key1": "value1"  },  ...}

使用字符串時將直接發送數據:

>>> import json>>> url = 'https://api.github.com/some/endpoint'>>> payload = {'some': 'data'}>>> r = requests.post(url, data=json.dumps(payload))

流上傳:

with open('massive-body', 'rb') as f:    requests.post('http://some.url/streamed', data=f)

Chunk-Encoded上傳:

def gen():    yield 'hi'    yield 'there'requests.post('http://some.url/chunked', data=gen())

3.如果要上傳文件,可以使用file參數發送Multipart-encoded數據,file參數是{ 'name': file-like-objects}格式的字典 (or {'name':('filename', fileobj)}) :

>>> url = 'http://httpbin.org/post'>>> files = {'file': open('report.xls', 'rb')}>>> r = requests.post(url, files=files)>>> r.text{  ...  "files": {    "file": "<censored...binary...data>"  },  ...}

也可以明確設置filename, content_type and headers:

>>> url = 'http://httpbin.org/post'>>> files = {'file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-Excel', {'Expires': '0'})}>>> r = requests.post(url, files=files)>>> print r.text{  "args": {},   "data": "",   "files": {    "file": "1/t2/r/n"  },   "form": {},   "headers": {    "Content-Type": "multipart/form-data; boundary=e0f9ff1303b841498ae53a903f27e565",     "Host": "httpbin.org",     "User-Agent": "python-requests/2.2.1 CPython/2.7.3 Windows/7",   },   "url": "http://httpbin.org/post"}

一次性上傳多個文件:

>>> url = 'http://httpbin.org/post'>>> multiple_files = [('images', ('foo.png', open('foo.png', 'rb'), 'image/png')),                      ('images', ('bar.png', open('bar.png', 'rb'), 'image/png'))]>>> r = requests.post(url, files=multiple_files)>>> r.text{  ...  'files': {'images': 'data:image/png;base64,iVBORw ....'}  'Content-Type': 'multipart/form-data; boundary=3131623adb2043caaeb5538cc7aa0b3a',  ...}

設置Headers

>>> import json>>> url = 'https://api.github.com/some/endpoint'>>> payload = {'some': 'data'}>>> headers = {'content-type': 'application/json'}>>> r = requests.post(url, data=json.dumps(payload), headers=headers)

Response對象:

 獲取unicode字符串,會自動根據響應頭部的字符編碼(r.encoding)進行解碼,當然也可以自己設定r.encoding:

>>> r = requests.get('https://github.com/timeline.json')>>> r.textu'{"message":"Hello there, wayfaring stranger...

獲取bytes字符串,會自動解碼gzip和deflate數據:

>>> r.content'{"message":"Hello there, wayfaring stranger. ..

要存儲web圖片,可以:

>>> from PIL import Image>>> from StringIO import StringIO>>> i = Image.open(StringIO(r.content))

可以解碼json對象:

>>> r.json(){u'documentation_url': u'https://developer...

返回raw response,需要在requests請求中將stream設為True:

>>> r = requests.get('https://github.com/timeline.json', stream=True)>>> r.raw<requests.packages.urllib3.response.HTTPResponse object at 0x101194810>>>> r.raw.read(10)'/x1f/x8b/x08/x00/x00/x00/x00/x00/x00/x03'

如果不想一次性處理全部的數據,可以:

tarball_url = 'https://github.com/kennethreitz/requests/tarball/master'r = requests.get(tarball_url, stream=True)if int(r.headers['content-length']) < TOO_LONG:  content = r.content  ...

也可以迭代的處理數據:

with open(filename, 'wb') as fd:    for chunk in r.iter_content(chunk_size):        fd.write(chunk)

或者:

import jsonimport requestsr = requests.get('http://httpbin.org/stream/20', stream=True)for line in r.iter_lines():    # filter out keep-alive new lines    if line:        print(json.loads(line))

獲取響應代碼:

>>> r = requests.get('http://httpbin.org/get')>>> r.status_code200

獲取響應headers:

>>> r.headers{    'content-encoding': 'gzip',    'transfer-encoding': 'chunked',    'connection': 'close',    'server': 'nginx/1.0.4',    'x-runtime': '148ms',    'etag': '"e1ca502697e5c9317743dc078f67693f"',    'content-type': 'application/json'}

獲取發送的headers

>>> r.request.headers{'Accept-Encoding': 'identity, deflate, compress, gzip','Accept': '*/*', 'User-Agent': 'python-requests/1.2.0'}

Cookie

獲取cookie,返回CookieJar對象:

>>> url = 'http://www.baidu.com'>>> r = requests.get(url)>>> r.cookies

將CookieJar轉為字典:

>>> requests.utils.dict_from_cookiejar(r.cookies){'BAIDUID': '84722199DF8EDC372D549EC56CA1A0E2:FG=1', 'BD_HOME': '0', 'BDSVRTM': '0'}

將字典轉為CookieJar:

requests.utils.cookiejar_from_dict(cookie_dict, cookiejar=None, overwrite=True)

上傳自己設置的cookie,使用cookies參數,可以是字典或者CookieJar對象:

>>> url = 'http://httpbin.org/cookies'>>> cookies = dict(cookies_are='working')>>> r = requests.get(url, cookies=cookies)>>> r.text'{"cookies": {"cookies_are": "working"}}'

如果需要在會話中保留cookie,需要用到后面要說的session

Redirection and History

可以用history屬性來追蹤redirection

>>> r = requests.get('http://github.com')>>> r.url'https://github.com/'>>> r.status_code200>>> r.history[<Response [301]>]

Session

要在會話中保留狀態,可以使用request.Session()。

Session可以使用get,post等方法,Session對象在請求時允許你保留一定的參數和自動設置cookie

s = requests.Session()s.get('http://httpbin.org/cookies/set/sessioncookie/123456789')   #cookie保留在s中r = s.get("http://httpbin.org/cookies") #再次訪問時會保留cookieprint(r.text)# '{"cookies": {"sessioncookie": "123456789"}}'

也可以自己設置headers,cookies:

s = requests.Session()s.auth = ('user', 'pass')s.headers.update({'x-test': 'true'})s.get('http://httpbin.org/headers', headers={'x-test2': 'true'})    #  'x-test' and 'x-test2' 都會被發送

預設Request

可以在發送request前做些額外的設定

from requests import Request, Sessions = Session()req = Request('GET', url,    data=data,    headers=header)prepped = req.prepare()# do something with prepped.body# do something with prepped.headersresp = s.send(prepped,    stream=stream,    verify=verify,    proxies=proxies,    cert=cert,    timeout=timeout)print(resp.status_code) 

驗證

Basic Authentication

>>> from requests.auth import HTTPBasicAuth>>> requests.get('https://api.github.com/user', auth=HTTPBasicAuth('user', 'pass'))<Response [200]>

因為HTTP Basic Auth很常用,所以也可以直接驗證:

>>> requests.get('https://api.github.com/user', auth=('user', 'pass'))<Response [200]>

Digest Authentication

>>> from requests.auth import HTTPDigestAuth>>> url = 'http://httpbin.org/digest-auth/auth/user/pass'>>> requests.get(url, auth=HTTPDigestAuth('user', 'pass'))<Response [200]>

OAuth 1 Authentication

>>> import requests>>> from requests_oauthlib import OAuth1>>> url = 'https://api.twitter.com/1.1/account/verify_credentials.json'>>> auth = OAuth1('YOUR_APP_KEY', 'YOUR_APP_SECRET',                  'USER_OAUTH_TOKEN', 'USER_OAUTH_TOKEN_SECRET')>>> requests.get(url, auth=auth)<Response [200]>

也可以使用自己寫的驗證類。比如某個web服務接受將X-Pizza報頭設置成密碼的驗證,可以這樣寫驗證類:

from requests.auth import AuthBaseclass PizzaAuth(AuthBase):    """Attaches HTTP Pizza Authentication to the given Request object."""    def __init__(self, username):        # setup any auth-related data here        self.username = username    def __call__(self, r):        # modify and return the request        r.headers['X-Pizza'] = self.username        return r

使用:

>>> requests.get('http://pizzabin.org/admin', auth=PizzaAuth('kenneth'))<Response [200]>

SSL證書驗證

檢查主機的ssl證書:

>>> requests.get('https://kennethreitz.com', verify=True)    raise ConnectionError(e)ConnectionError: HTTPSConnectionPool(host='kennethreitz.com', port=443): Max retries exceeded with url: / (Caused by <class 'socket.error'>: [Errno 10061] )

github是有的:

>>> requests.get('https://github.com', verify=True)<Response [200]>

如果你設置驗證設置為False,也可以忽略驗證SSL證書。

可以讀取驗證文件:

>>> requests.get('https://kennethreitz.com', cert=('/path/server.crt', '/path/key'))

代理

使用代理:

import requestsproxies = {  "http": "http://10.10.1.10:3128",  "https": "http://10.10.1.10:1080",}requests.get("http://example.org", proxies=proxies)

可以設置環境變量:

$ export HTTP_PROXY="http://10.10.1.10:3128"$ export HTTPS_PROXY="http://10.10.1.10:1080"$ python>>> import requests>>> requests.get("http://example.org")

如果代理需要驗證:

proxies = {    "http": "http://user:pass@10.10.1.10:3128/",}

  

 


發表評論 共有條評論
用戶名: 密碼:
驗證碼: 匿名發表
主站蜘蛛池模板: 尚义县| 襄城县| 开封县| 揭阳市| 山东省| 昌平区| 杭锦旗| 武山县| 南开区| 黎城县| 阿鲁科尔沁旗| 措勤县| 泗洪县| 扎兰屯市| 青田县| 台中县| 邻水| 霍林郭勒市| 星子县| 岚皋县| 克拉玛依市| 宽城| 高邮市| 新田县| 福泉市| 栾城县| 新宾| 鹤庆县| 钦州市| 孟州市| 新野县| 林西县| 乌拉特前旗| 寿阳县| 旬邑县| 山阴县| 石河子市| 南康市| 夏津县| 汽车| 奈曼旗|