Uncovering Hidden Treasures: Mastering Wayback URLs for Bug Bounty Hunting

Introduction

Following our previous discussions on reconnaissance techniques and automated security scanning, this write-up explores another crucial tool for bug bounty hunters: Wayback URLs. The Wayback Machine archives historical versions of web pages, and waybackurls helps extract these stored URLs. This can reveal forgotten or vulnerable endpoints, making it a valuable asset in the reconnaissance phase. In this continuation, we'll cover installation, usage, and best practices for incorporating Wayback URLs into your bug bounty workflow.

1. Installing Wayback URLs

Before we dive into usage, let’s install the tool:

 go install -v github.com/tomnomnom/waybackurls@latest

Verify installation by running:

waybackurls -h

2. Extracting URLs Using Waybackurls

To fetch archived URLs for a target domain:

cat live.txt | waybackurls

To extract URLs from a list of live domains:

cat live.txt | waybackurls > wayback.txt

This will save all the discovered URLs into wayback.txt, which can then be analyzed for vulnerabilities.

3. Filtering for Sensitive Information

Archived URLs may contain exposed sensitive files, API endpoints, or credentials. You can use grep to filter them:

grep -E "@gmail|xml|json|config" wayback.txt

For JWT tokens:

grep "eyJ" wayback.txt | tee jwt_tokens.txt

Decode these tokens using jwt.io to check for sensitive information.

4. Hunting for Vulnerabilities

Finding Old API Endpoints

Attackers often exploit deprecated endpoints. Use grep to search for API-related URLs:

grep -E "api|v1|v2" wayback.txt

Searching for Backup Files

Developers sometimes leave backup files that can expose sensitive configurations:

grep -E ".bak|.old|.swp|.backup" wayback.txt

Detecting Exposed Credentials

Look for .env or .config files that might contain API keys or database credentials:

grep -i "password|apikey|secret" wayback.txt

5. Automating with Other Tools

Using HTTPX for Valid URLs

To check which URLs are still active:

cat wayback.txt | httpx -silent -status-code

Scanning with Nuclei

Combine Wayback URLs with nuclei to check for vulnerabilities:

cat wayback.txt | nuclei -t ~/nuclei-templates/

Using GF Patterns

Use gf patterns to find potential vulnerabilities:

cat wayback.txt | gf sqli

6. Best Practices & Limitations

• Beware of False Positives: Not all archived URLs are still valid.
• Respect Rate Limits: Avoid hammering sites with too many requests.
• Use Additional Recon Methods: Wayback URLs should complement, not replace, other enumeration techniques.
• Check Robots.txt: Sometimes, disallowed URLs in robots.txt can indicate hidden endpoints.

Conclusion

Building on our previous topics, Wayback URLs provide another reconnaissance technique that can help uncover forgotten vulnerabilities. By integrating waybackurls with other tools like httpx, nuclei, and grep, you can automate and refine your bug bounty reconnaissance process. Stay tuned for more advanced techniques in our bug bounty series. Happy hunting! 🚀