国产探花免费观看_亚洲丰满少妇自慰呻吟_97日韩有码在线_资源在线日韩欧美_一区二区精品毛片,辰东完美世界有声小说,欢乐颂第一季,yy玄幻小说排行榜完本

首頁 > 語言 > PHP > 正文

PHPCrawl爬蟲庫實現抓取酷狗歌單的方法示例

2024-05-05 00:01:32
字體:
來源:轉載
供稿:網友

本文實例講述了PHPCrawl爬蟲庫實現抓取酷狗歌單的方法。分享給大家供大家參考,具體如下:

本人看了網絡爬蟲相關的視頻后,手癢癢,想爬點什么。最近Facebook上表情包大戰很激烈,就想著把所有表情包都爬下來,卻一時沒有找到合適的VPN,因此把酷狗最近一月精選歌曲和簡單介紹抓取到本地。代碼寫得有點亂,自己不是很滿意,并不想放上來丟人現眼。不過轉念一想,這好歹是自己第一次爬蟲,于是...就有了如下不堪入目的代碼~~~(由于抓取的數據量較小,所以沒有考慮多進程什么的,不過我看了一下PHPCrawl的文檔,發現PHPCrawl庫已經把我能想到的功能都封裝好了,實現起來很方便)

<?phpheader("Content-type:text/html;charset=utf-8");// It may take a whils to crawl a site ...set_time_limit(10000);include("libs/PHPCrawler.class.php");class MyCrawler extends PHPCrawler {  function handleDocumentInfo($DocInfo) {    // Just detect linebreak for output ("/n" in CLI-mode, otherwise "<br>").    if (PHP_SAPI == "cli") $lb = "/n";    else $lb = "<br />";    $url = $DocInfo->url;    $pat = "/http:////www/.kugou/.com//yy//special//single///d+/.html/";    if(preg_match($pat,$url) > 0){    $this->parseSonglist($DocInfo);    }    flush();  }  public function parseSonglist($DocInfo){    $content = $DocInfo->content;    $songlistArr = array();    $songlistArr['raw_url'] = $DocInfo->url;    //解析歌曲介紹    $matches = array();    $pat = "/<span>名稱:<//span>([^(<br)]+)<br/";    $ret = preg_match($pat,$content,$matches);    if($ret>0){      $songlistArr['title'] = $matches[1];    }else{      $songlistArr['title'] = '';    }    //解析歌曲    $pat = "/<a    $matches = array();    preg_match_all($pat,$content,$matches);    $songlistArr['songs'] = array();    for($i = 0;$i < count($matches[0]);$i++){      $song_title = $matches[1][$i];      array_push($songlistArr['songs'],array('title'=>$song_title));    }    echo "<pre>";    print_r($songlistArr);    echo "</pre>";    }  }$crawler = new MyCrawler();// URL to crawl$start_url="http://www.kugou.com/yy/special/index/1-0-2.html";$crawler->setURL($start_url);// Only receive content of files with content-type "text/html"$crawler->addContentTypeReceiveRule("#text/html#");//鏈接擴展$crawler->addURLFollowRule("#http://www/.kugou/.com/yy/special/single//d+/.html$# i");$crawler->addURLFollowRule("#http://www.kugou/.com/yy/special/index//d+-/d+-2/.html$# i");// Store and send cookie-data like a browser does$crawler->enableCookieHandling(true);// Set the traffic-limit to 1 MB(1000 * 1024) (in bytes,// for testing we dont want to "suck" the whole site)//爬取大小無限制$crawler->setTrafficLimit(0);// Thats enough, now here we go$crawler->go();// At the end, after the process is finished, we print a short// report (see method getProcessReport() for more information)$report = $crawler->getProcessReport();if (PHP_SAPI == "cli") $lb = "/n";else $lb = "<br />";echo "Summary:".$lb;echo "Links followed: ".$report->links_followed.$lb;echo "Documents received: ".$report->files_received.$lb;echo "Bytes received: ".$report->bytes_received." bytes".$lb;echo "Process runtime: ".$report->process_runtime." sec".$lb; ?>

 

希望本文所述對大家PHP程序設計有所幫助。


注:相關教程知識閱讀請移步到PHP教程頻道。
發表評論 共有條評論
用戶名: 密碼:
驗證碼: 匿名發表

圖片精選

主站蜘蛛池模板: 灌阳县| 磐安县| 神农架林区| 绍兴市| 宁阳县| 渝中区| 中超| 阿拉善盟| 平和县| 武平县| 靖远县| 云龙县| 滁州市| 晋城| 凤冈县| 阳高县| 宁强县| 富裕县| 漳平市| 即墨市| 虞城县| 惠安县| 黑河市| 涿州市| 临夏市| 信丰县| 银川市| 奈曼旗| 富平县| 和林格尔县| 霍城县| 拉孜县| 文登市| 正阳县| 南江县| 和田县| 观塘区| 平南县| 浮梁县| 正宁县| 常熟市|