"Mozilla/5.0 (Macintosh Intel Mac OS X 10_8_0) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/.0 Safari/536.3", # for more user agent strings,you can find it in # the default user_agent_list composes chrome,I E,firefox,Mozilla,opera,netscape You will have to insert the new components in the settings.py: DOWNLOADER_MIDDLEWARES = '.format(('User-Agent'), request), Modify DOWNLOADER_MIDDLEWARES as follows.
An approach using Scrapy will make use of two components, RandomProxy and RotateUserAgentMiddleware.
0 kommentar(er)
