lyq/crawl_mrdx: 新闻电讯爬虫。 http://mrdx.cn/ @ 4b265fa82aebe895e43c141dda0a4bc9ffdd1b9d - 芝舟开源

新闻电讯爬虫。 http://mrdx.cn/

4 Commits

1 Branches

0 Releases

liuyuqi-dellpc 4b265fa82a 添加多线程		5 years ago
data	efdb552c5a init	5 years ago
shell	f9e082fbc0 增加脚本	5 years ago
test	efdb552c5a init	5 years ago
utils	efdb552c5a init	5 years ago
.gitignore	4b265fa82a 添加多线程	5 years ago
README.md	efdb552c5a init	5 years ago
main.py	4b265fa82a 添加多线程	5 years ago
requirements.txt	efdb552c5a init	5 years ago

新闻电讯爬虫

初步完成，单线程，文明爬虫（每次爬虫1-3s休息）。

cd my_project_dir
virtualenv -p /opt/python/bin/python3 venv
source venv/bin/activate
pip install -r requirements.txt
python main.py