Scrapy:How to print request referrer

python, scrapy

Solution

HTTP `Referer` field is set up by HTTP client in request headers, not in response headers, as this header tells server where did client come from to current page.

It would be rather weird to receive http `Referer` header in response.

But when talking about `scrapy`, there's a reference to `Request` object on which the `Response` was generated, in response's `request` field, so the next call result:

response.request.headers.get('Referer', None)

can contain `Referer` header if it was set when making request.

Problem

Original source