Troubleshoot empty or partial scraping responses from JavaScript-rendered pages. Covers wait parameters, AJAX handling, and Zenrows fixes.
Sometimes, your requests may return empty or incomplete content. This is typically due to how modern websites load content dynamically using JavaScript or AJAX (XHR) calls. By default, Zenrows returns the HTML content once the DOM is initially loaded, which might not include content populated by JavaScript.This guide provides diagnostic steps, common scenarios, and parameter configurations to help you resolve these issues and get complete, usable responses.
These troubleshooting steps are applicable across all Zenrows features, including screenshot, outputs, json_response, and others.
Why Does This Happen?
Even if a page visually “loads,” its content might still be missing because:
The content is rendered by JavaScript after the DOM loads.
The actual data is fetched via XHR (AJAX) requests.
The page relies on user interaction or delays to render content.
Anti-bot measures may serve minimal or obfuscated content.
Quick Decision Tree
ERROR ↓Is js_render enabled? ├─ No → Enable `js_render` └─ Yes ↓ Is content dynamic (loaded after a delay or via JS)? ├─ Yes → Use `wait` or `wait_for` └─ No ↓ Is content loaded via XHR? ├─ Yes → Use `json_response` └─ No → Enable screenshot to visually debug ↓ Does the page require user interaction? ├─ Yes → Use `js_instructions` └─ No → Content may be static or blocked
Uses residential proxies for protected access to restricted content
Site blocks datacenter IPs or has bot detection
Boolean
Recommended Steps
1. Enable JavaScript Rendering
If the target website relies on JavaScript to display content, enable js_render. This ensures that Zenrows runs JavaScript in a headless browser to render the page fully.
wait_for will cause the request to fail if the selector is not found. Use browser DevTools to find reliable CSS selectors on your target site.
3. Use json_response for XHR-based Pages
If the data you’re trying to scrape is loaded via JavaScript through AJAX calls, you can use the json_response parameter to access the response of those internal XHR requests.
Combine json_response with wait or wait_for to ensure all background requests have completed before data is captured.
4. Use Screenshot for Debugging
To understand what Zenrows actually “sees”, enable the screenshot parameter. This can help determine whether content is missing due to rendering delays or protections.
This feature provides a rendered image of the page after all scripts have run and delays have passed.
5. Use js_instructions for Interactive Pages
If the page requires user interaction to load the content (like clicking a button, scrolling, or inputting text), use the js_instructions parameter. This allows you to simulate actions in the browser before the content is captured.Example:
When using js_instructions, each instruction can have an optional wait_for step. If the selector is not found, Zenrows will attempt the next instruction after a short delay.
Mixing Steps for Advanced Debugging
You can combine multiple steps for complex scenarios. For example:
Use js_instructions to simulate user interactions, followed by json_response to capture XHR data.
Enable screenshot alongside wait_for to visually confirm dynamic content.