Treat every failure as information
A resilient scraper distinguishes an unavailable source from an empty result, a validation problem, or a temporary access failure. Clear classification makes the next action predictable.
Bounded retries can then focus on transient failures while permanent contract problems surface quickly for investigation.
Validate before delivery
Schema checks and content-level assertions catch responses that look successful but do not contain the requested record. This protects downstream systems from silent quality regressions.
