urlhunter is a recon tool that allows searching on URLs that are exposed via shortener services such as bit.ly and goo.gl. The project is written in Go.
How?
A group named URLTeam (kudos to them) are brute forcing the URL shortener services and publishing matched results on a daily basis. urlhunter downloads their collections and lets you analyze them.Installation
From Binary
You can download the pre-built binaries from the releases page and run. For example:tar xzvf urlhunter_0.1.0_Linux_amd64.tar.gz./urlhunter --helpFrom Source
- Install Go on your system
- Run: go install github.com/utkusen/urlhunter@latest
Usage
urlhunter requires 3 parameters to run: -keywords, -date.For example: urlhunter --keywords keywords.txt --date 2020-11-20
--keywords
You need to specify the txt file that contains keywords to search on URLs. Keywords must be written line by line. You have three different ways to specify keywords:Single Keyword: urlhunter will search the given keyword as a substring. For example:
acme.com keyword will both match https://acme.com/blabla and https://another.com/?referrer=acme.com
Multiple Keywords: urlhunter will search the given keywords with an AND logic. Which means, a URL must include all the provided keywords. Keywords must be separated with , character. For example:
acme.com,admin will match https://acme.com/secret/adminpanel but won't match https://acme.com/somethingelse
Regex Values: urlhunter will search for the given regex value. In the keyword file, the line that contains a regular expression formula must start with regex string. The format is: regex REGEXFORMULA. For example:
regex 1\d{10} will match https://example.com/index.php?id=12938454312 but won't match https://example.com/index.php?id=abc223
--date
urlhunter downloads the archive files of the given date(s). You have three different ways to specify the date:Latest: urlhunter will download the latest archive. -date latest
Single Date: urlhunter will download the archive of the given date. Date format is YYYY-MM-DD.
For example: -date 2020-11-20
Year: urlhunter will download the archive of the given year. Year format is YYYY.
For example: -date 2024
Date Range: urlhunter will download all the archives between given start and end dates.
For example: -date 2020-11-10:2020-11-20
--output
You can specify the output file with -o parameter. For example -o out.txt--rm
You can specify the -rm parameter to remove the downloaded archive folders after processing. This is useful to save disk space.Demonstration Video
Example Use Cases
urlhunter might be useful for cyber intelligence and bug bounty purposes. For example:docs.google.com/a/acme.com drive.google.com/a/acme.com keywords allow you to find public Google Docs&Drive share links of Acme company.
acme.com,password_reset_token keyword may allow you to find the working password reset tokens of acme.com