How to write a script to check for insecure HTTP headers

Question

I want to analyze a website’s HTTP security headers to check for misconfigurations that might expose vulnerabilities. I’m looking for guidance on:

• What HTTP headers (e.g., CSP, HSTS, X-Frame-Options) should be checked.
• How to write a Python or Bash script to automate header analysis.
• How to interpret results and recommend fixes.
  A sample script or recommendations on tools like curl, requests, or securityheaders.com would be useful.

Answer 1

Ensuring the security of your web application involves analyzing HTTP response headers to identify potential vulnerabilities. This guide provides an overview of essential security headers, guidance on scripting automated checks using Python, and recommendations for interpreting results and implementing fixes.

1. Essential HTTP Security Headers to Check

Implementing the following HTTP security headers can significantly enhance your website's security posture:

• Content-Security-Policy (CSP): Restricts sources from which content can be loaded, mitigating cross-site scripting (XSS) and data injection attacks.

• Strict-Transport-Security (HSTS): Enforces secure (HTTPS) connections to the server, protecting against protocol downgrade attacks.

• X-Frame-Options: Prevents the website from being embedded in frames or iframes, defending against clickjacking attacks.

• X-Content-Type-Options: Prevents browsers from MIME-sniffing a response away from the declared content type, reducing the risk of executing malicious files.

• X-XSS-Protection: Enables the browser's cross-site scripting filter to prevent the execution of malicious scripts.

• Referrer-Policy: Controls how much referrer information is included with requests, enhancing user privacy.

• Permissions-Policy: Manages access to browser features and APIs, such as geolocation and camera, reducing potential abuse.

2. Automating Header Analysis with Python

To automate the analysis of these headers, you can write a Python script utilizing the requests library. Below is a sample script that checks the presence and configuration of these headers:

import requests

# Define the security headers and their descriptions
SECURITY_HEADERS = {
    "Content-Security-Policy": "Defines approved sources of content.",
    "Strict-Transport-Security": "Enforces secure (HTTP over SSL/TLS) connections to the server.",
    "X-Content-Type-Options": "Prevents MIME type sniffing.",
    "X-Frame-Options": "Controls whether the site can be framed.",
    "X-XSS-Protection": "Enables cross-site scripting filters.",
    "Referrer-Policy": "Controls how much referrer information is included with requests.",
    "Permissions-Policy": "Manages browser features and APIs permissions."
}

def analyze_headers(url):
    try:
        response = requests.get(url)
        headers = response.headers

        for header, description in SECURITY_HEADERS.items():
            if header in headers:
                print(f"[+] {header} is present: {headers[header]}")
            else:
                print(f"[-] {header} is missing. {description}")
    except requests.RequestException as e:
        print(f"Error fetching {url}: {e}")

if __name__ == "__main__":
    url = input("Enter the URL to analyze: ")
    analyze_headers(url)

3. Interpreting Results and Recommending Fixes

After running the script, interpret the results as follows:

• Present Headers: If a header is present, verify its configuration aligns with security best practices.

• Missing Headers: If a header is missing, implement it to enhance security.

Recommendations for Implementing Headers:

• Content-Security-Policy: Define a policy that restricts sources for scripts, styles, and other content. For example:

Content-Security-Policy: default-src 'self'; script-src 'self' https://trusted.cdn.com; object-src 'none';

• Strict-Transport-Security: Enforce HTTPS by adding:

Strict-Transport-Security: max-age=31536000; includeSubDomains; preload

• X-Frame-Options: Prevent clickjacking by setting:

X-Frame-Options: DENY

• X-Content-Type-Options: Prevent MIME type sniffing with:

X-Content-Type-Options: nosniff

• X-XSS-Protection: Enable XSS filtering:

X-XSS-Protection: 1; mode=block

• Referrer-Policy: Control referrer information with:

Referrer-Policy: no-referrer-when-downgrade

• Permissions-Policy: Manage feature permissions:

Permissions-Policy: geolocation=(), camera=()

4. Utilizing Existing Tools

Several tools can assist in analyzing HTTP security headers:

• securityheaders.com: An online service that analyzes HTTP response headers and provides a security rating.

• Shcheck: A Python-based tool that scans security headers of any website.

• CheckMyHeaders: A Python command-line tool to analyze HTTP security headers.

By implementing and regularly auditing these headers, you can significantly enhance your web application's security against common vulnerabilities.