Website Directory Scanner

Exposed directory indices are a common sight on the internet. In certain cases, these are unintentional and may contain sensitive information or indicate exploitable scripts or services. In general, directory listing should be disabled.

Removing Directory Listing

For nearly all HTTP servers, creating an index.html file will disable directory listing in a specific directory. However, to entirely disable directory listing, the steps will depend on your specific server. Steps for two common HTTP servers are provided below.

Apache

Create a .htaccess file in the directory with the following contents:

Options -Indexes

or, edit your Apache configuration file (generally found in /etc/httpd/conf/httpd.conf) and change Options Indexes FollowSymLinks to Options FollowSymLinks.

Nginx

Directory listing is disabled by default in Nginx and is controlled by the autoindex setting. In available_sites, make sure no server or location has autoindex on in it.

Dependencies

Install dependencies using the following command:

pip3 install -r requirements.txt

Usage

usage: main.py [-h] -u BASE_URL [-d]
optional arguments:
  -h, --help                        show this help message and exit
  -u BASE_URL, --base-url BASE_URL  the starting URL
  -d, --only-directories            display only directories (exclude files)

Example Usage:

python main.py -u https://srikavin.me/a/

Output (asciinema)