Non-Standard Python Libraries:
Created with Python 3.6
This tool was inspired by a recent talk by Bryce Kunz. The talk Blue Cloud of Death: Red Teaming Azure takes us through some of the lesser known common information disclosures outside of the ever common S3 Buckets.
usage: CloudScraper.py [-h] [-v] [-p Processes] [-d DEPTH] [-u URL] [-l TARGETLIST]
optional arguments:
-h, --help show this help message and exit
-u URL Target Scope
-d DEPTH Max Depth of links Default: 5
-l TARGETLIST Location of text file of Line Delimited targets
-v Verbose Verbose output
-p Processes Number of processes to be executed in parallel. Default: 2
example: python3 CloudScraper.py -u https://rottentomatoes.com
To add keywords, simply add to the list in the parser function.
Sharing is caring! Pull requests welcome, things like adding support for more detections, multithreading etc are highly desired :)
So Bryce Kunz actually made a tool to do something similar but it used scrapy and I wanted to build something myself that didn't depend on Python2 or any scraping modules such as scrapy. I did end up using BeautifulSoup to parse for href links for spidering only. Hence, CloudScraper was born. The benefit of using raw regex's instead of parsing for href links, is that many times, these are not included in href links, they can be buried in JS or other various locations. CloudScraper grabs the entire page and uses a regex to look for links. This also has its flaws such as grabbing too much or too little but at least we know we are covering our bases :)
错误示例 Traceback (most recent call last): cloudscraper.exceptions.CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) vers
代码访问网站403,使用Python cloudscraper模块报错Detected a Cloudflare version 2 Captcha challenge 使用ScrapingAnt替代Cloudflare,打开网站需要魔法上网,注册完成免费套餐每个月送1W点数,查询一次十点,代码如下: import requests import urllib.pa