XPROXY

Python Requests ＆ curl_cffi との連携

Learn how to route Python Requests and TLS-impersonated curl_cffi scrapers through XProxy 4G/5G mobile proxies with automated IP rotation triggers on HTTP 429/403 rate limits.

Why use XProxy with Python Requests & curl_cffi?

Python Requests and curl_cffi (which mimics Chrome/Firefox TLS fingerprints at the socket level) are the most popular libraries for fast, lightweight web scraping. Routing your Python scripts through XProxy self-hosted 4G/5G mobile proxies lets you bypass Cloudflare TLS fingerprinting and IP rate limits with unlimited data throughput.

事前準備

  • Python 3.8+ with requests and curl_cffi installed (pip install requests curl_cffi).
  • XProxy server running with active 4G/5G USB dongles or Android phones.
  • Proxy credentials (IP, Port, Username, Password, Rotation API Link) from XProxy Dashboard.

1. Standard Python Requests Example

Below is a complete script demonstrating automatic XProxy IP rotation when receiving HTTP 429 or 403 status codes:

python_requests_xproxy.py
import time
import requests

# --- XProxy 設定手順 ---
PROXIES = {
    "http": "http://username:[email protected]:5001",
    "https": "http://username:[email protected]:5001",
}
ROTATION_API_URL = "http://192.168.6.7:8081/api/v1/rotate_ip/position/1"

def rotate_xproxy_ip():
    """Trigger 4G/5G cellular reconnection on XProxy."""
    print("🔄 Triggering XProxy IP rotation...")
    try:
        res = requests.get(ROTATION_API_URL, timeout=10)
        if res.status_code == 200:
            print("✅ IP rotation requested. Waiting 6 seconds for 4G reconnect...")
            time.sleep(6)
            return True
    except Exception as e:
        print(f"❌ Rotation error: {e}")
    return False

def scrape_url(url):
    for attempt in range(3):
        try:
            response = requests.get(url, proxies=PROXIES, timeout=15)
            
            # Handle rate limits or anti-bot blocks
            if response.status_code in (429, 403):
                print(f"⚠️ Encountered HTTP {response.status_code}. Rotating IP...")
                rotate_xproxy_ip()
                continue
                
            print(f"✅ Success ({response.status_code}): {response.text[:100]}")
            return response
        except requests.exceptions.RequestException as e:
            print(f"❌ Request error: {e}. Retrying with new IP...")
            rotate_xproxy_ip()

if __name__ == "__main__":
    scrape_url("https://api.ipify.org?format=json")

2. TLS Impersonation with `curl_cffi`

To bypass Cloudflare & Akamai TLS fingerprint checks while using XProxy 4G/5G mobile IPs, use curl_cffi to spoof real Chrome browser TLS signatures:

curl_cffi_xproxy.py
from curl_cffi import requests

# Configure SOCKS5 or HTTP proxy with Chrome TLS impersonation
proxy_url = "socks5://username:[email protected]:5001"

response = requests.get(
    "https://tls.browserleaks.com/json",
    proxies={"http": proxy_url, "https": proxy_url},
    impersonate="chrome120",  # Impersonate Chrome 120 TLS fingerprint
    timeout=15
)

print("Status Code:", response.status_code)
print("TLS Response:", response.json())

Frequently Asked Questions

  • Q: Should I use HTTP or SOCKS5 with Python Requests?
    A: Standard requests works great with HTTP proxies. For SOCKS5 support in Python Requests, install requests[socks] (e.g. socks5h://user:[email protected]:5001).