Integração com Python Requests e curl_cffi
Aprenda a rotear os scrapers de Python Requests e curl_cffi com impersonação de TLS pelos proxies móveis 4G/5G da XProxy, com gatilhos automáticos de rotação de IP em limites de taxa HTTP 429/403.
Por que usar a XProxy com Python Requests e curl_cffi?
Python Requests e curl_cffi (que imita as impressões digitais TLS do Chrome/Firefox no nível de socket) são as bibliotecas mais populares para web scraping rápido e leve. Ao rotear seus scripts Python pelos proxies móveis 4G/5G auto-hospedados da XProxy, você contorna a impressão digital TLS do Cloudflare e os limites de taxa de IP com vazão de dados ilimitada.
Pré-requisitos
- Python 3.8+ com
requestsecurl_cffiinstalados (pip install requests curl_cffi). - Servidor XProxy em execução com dongles USB 4G/5G ou celulares Android ativos.
- Credenciais de proxy (IP, Porta, Usuário, Senha, Link da API de Rotação) do Painel da XProxy.
1. Exemplo padrão de Python Requests
Abaixo está um script completo que demonstra a rotação automática de IP da XProxy ao receber os códigos de status HTTP 429 ou 403:
import time
import requests
# --- XProxy Configuration ---
PROXIES = {
"http": "http://username:[email protected]:5001",
"https": "http://username:[email protected]:5001",
}
ROTATION_API_URL = "http://192.168.6.7:8081/api/v1/rotate_ip/position/1"
def rotate_xproxy_ip():
"""Trigger 4G/5G cellular reconnection on XProxy."""
print("🔄 Triggering XProxy IP rotation...")
try:
res = requests.get(ROTATION_API_URL, timeout=10)
if res.status_code == 200:
print("✅ IP rotation requested. Waiting 6 seconds for 4G reconnect...")
time.sleep(6)
return True
except Exception as e:
print(f"❌ Rotation error: {e}")
return False
def scrape_url(url):
for attempt in range(3):
try:
response = requests.get(url, proxies=PROXIES, timeout=15)
# Handle rate limits or anti-bot blocks
if response.status_code in (429, 403):
print(f"⚠️ Encountered HTTP {response.status_code}. Rotating IP...")
rotate_xproxy_ip()
continue
print(f"✅ Success ({response.status_code}): {response.text[:100]}")
return response
except requests.exceptions.RequestException as e:
print(f"❌ Request error: {e}. Retrying with new IP...")
rotate_xproxy_ip()
if __name__ == "__main__":
scrape_url("https://api.ipify.org?format=json")2. Impersonação de TLS com `curl_cffi`
Para contornar as verificações de impressão digital TLS do Cloudflare e da Akamai ao usar IPs móveis 4G/5G da XProxy, use curl_cffi para falsificar as assinaturas TLS reais do navegador Chrome:
from curl_cffi import requests
# Configure SOCKS5 or HTTP proxy with Chrome TLS impersonation
proxy_url = "socks5://username:[email protected]:5001"
response = requests.get(
"https://tls.browserleaks.com/json",
proxies={"http": proxy_url, "https": proxy_url},
impersonate="chrome120", # Impersonate Chrome 120 TLS fingerprint
timeout=15
)
print("Status Code:", response.status_code)
print("TLS Response:", response.json())Perguntas Frequentes
- P: Devo usar HTTP ou SOCKS5 com Python Requests?
R: Orequestspadrão funciona muito bem com proxies HTTP. Para suporte a SOCKS5 no Python Requests, instalerequests[socks](por exemplo,socks5h://user:[email protected]:5001).
Why High-Quality SOCKS5 UDP & Mobile Proxies Matter for Python Requests & Scrapers
Using cheap datacenter proxies or basic HTTP proxies in antidetect profiles leads to instant account suspensions on Facebook, Google, TikTok, and Amazon. Here is why XProxy's self-hosted 4G/5G mobile proxies deliver superior anonymity and trust:
- SOCKS5 with Full UDP Support: Unlike standard HTTP proxies that drop UDP packets, XProxy SOCKS5 proxies fully support UDP datagrams. This ensures WebRTC, DNS queries, and real-time audio/video sockets route securely through your proxy without revealing your real IP address.
- Instant REST API IP Rotation: Trigger cellular IP rotation on demand via simple HTTP GET requests (e.g.
http://192.168.6.7:8081/api/v1/rotate_ip/position/1). Reconnect 4G/5G USB modems in 3 to 8 seconds with zero proxy dropouts. - 100% WebRTC & DNS Leak Protection: Combined with XProxy's local LAN gateway and TCP OS Spoofing, your browser profiles maintain pristine anonymity scores on BrowserLeaks, Whoer, Pixelscan, and IPhey.
- Carrier-Grade NAT (CGNAT) Trust: Mobile carrier IPs are shared by thousands of active smartphone users. Major platforms never ban mobile IP ranges, granting your accounts maximum trust.
