2026-09-17 06:16:41 [scrapy.utils.log] INFO: Scrapy 2.18.0 started (bot: grabbers) 2026-09-17 06:16:41 [scrapy.utils.log] INFO: Versions: {'lxml': '6.1.3', 'libxml2': '2.14.6', 'cssselect': '1.5.0', 'parsel': '1.11.0', 'w3lib': '2.4.1', 'Twisted': '26.4.0', 'Python': '3.14.7 (main, Sep 1 2026, 14:18:09) [Clang 22.1.3 ]', 'pyOpenSSL': '26.4.0 (OpenSSL 4.0.2 25 Aug 2026)', 'cryptography': '50.0.1', 'Platform': 'Linux-6.18.41-94.142.amzn2023.x86_64-x86_64-with-glibc2.35'} 2026-09-17 06:16:41 [scrapy.addons] INFO: Enabled addons: [] 2026-09-17 06:16:41 [py.warnings] WARNING: /layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/extensions/feedexport.py:479: ScrapyDeprecationWarning: The `FEED_URI` and `FEED_FORMAT` settings have been deprecated in favor of the `FEEDS` setting. Please see the `FEEDS` setting docs for more details exporter = cls(crawler) 2026-09-17 06:16:41 [scrapy.middleware] INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.logcount.LogCount', 'scrapy.extensions.memusage.MemoryUsage', 'scrapy.extensions.logstats.LogStats', 'grabbers.extensions.custom_fields.ExportCustomFieldsExtension', 'grabbers.extensions.kol_roles.ExportKolRolesExtension', 'grabbers.extensions.exporters.DefaultScrapedItemsExporter'] 2026-09-17 06:16:41 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'grabbers', 'FEED_EXPORT_FIELDS': ['External Id', 'Parent External Id', 'Name', 'Plain text: Parent Name', 'Start date', 'Start time', 'End date', 'End time', 'Location', 'Plain text: Event Link', 'Plain text: Event Link (Excel)', 'Plain text: Session ID', 'Plain text: Session description', 'Filter: Session type', 'Filter: Presentation topic', 'Plain text: Presentation number', 'Plain text: Abstract', 'Is all day', 'Plain text: Speakers', 'Plain text: Speakers Countries'], 'FEED_FORMAT': 'xlsx', 'FEED_URI': 's3://usummit-prod-grabbers-bucket/ESMO Congress 2025 ' 'Posters/ESMO Congress 2025 Posters-2538_2026-09-17_06-16-33.xlsx', 'LOG_FILE': '/etc/scrapyd/dbs/logs/grabbers/ESMO Congress 2025 ' 'Posters/54b62750b25f11f196267ffafbfb6dbc.log', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'grabbers.spiders', 'SPIDER_MODULES': ['grabbers.spiders'], 'TELNETCONSOLE_ENABLED': False, 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ' '(KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36'} 2026-09-17 06:16:42 [scrapy.middleware] INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware', 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'grabbers.middlewares.WafSolverMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scrapy.downloadermiddlewares.stats.DownloaderStats'] 2026-09-17 06:16:42 [scrapy.middleware] INFO: Enabled spider middlewares: ['scrapy.spidermiddlewares.start.StartSpiderMiddleware', 'scrapy.spidermiddlewares.httperror.HttpErrorMiddleware', 'grabbers.middlewares.SilentValidationMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware', 'scrapy.spidermiddlewares.metacopy.MetaCopyDetectionMiddleware'] 2026-09-17 06:16:42 [scrapy.middleware] INFO: Enabled item pipelines: ['grabbers.pipelines.CleanTextPipeline', 'grabbers.pipelines.DateTimeFormatPipeline', 'grabbers.pipelines.BoolFormatPipeline', 'grabbers.pipelines.ValidateEventNamePipeline', 'grabbers.pipelines.CountryCodePipeline', 'grabbers.pipelines.HyperlinkedEventLinkPipeline'] 2026-09-17 06:16:42 [scrapy.core.engine] INFO: Spider opened 2026-09-17 06:16:42 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2026-09-17 06:16:43 [grabbers.middlewares.waf_solver] INFO: AWS WAF challenge detected and solved for domain cslide.ctimeetingtech.com. 2026-09-17 06:17:42 [scrapy.extensions.logstats] INFO: Crawled 79 pages (at 79 pages/min), scraped 0 items (at 0 items/min) 2026-09-17 06:18:42 [scrapy.extensions.logstats] INFO: Crawled 182 pages (at 103 pages/min), scraped 0 items (at 0 items/min) 2026-09-17 06:19:42 [scrapy.extensions.logstats] INFO: Crawled 259 pages (at 77 pages/min), scraped 0 items (at 0 items/min) 2026-09-17 06:19:53 [scrapy.core.scraper] ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/utils/defer.py", line 373, in aiter_errback yield await anext(it) ^^^^^^^^^^^^^^^ File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/utils/python.py", line 366, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/utils/python.py", line 347, in _async_chain async for o in as_async_generator(it): yield o File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/utils/asyncgen.py", line 18, in as_async_generator async for r in it: yield r File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/utils/python.py", line 366, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/utils/python.py", line 347, in _async_chain async for o in as_async_generator(it): yield o File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/utils/asyncgen.py", line 18, in as_async_generator async for r in it: yield r File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 111, in _evaluate_iterable exception_result: MutableAsyncChain[_T] = self._process_spider_exception( ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ response, ex, exception_processor_index ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception raise exception File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 108, in _evaluate_iterable async for r in iterable: yield r File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/utils/decorators.py", line 152, in asyncgen_inner async for item in func(*args, **kwargs): yield item File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/spidermiddlewares/base.py", line 67, in process_spider_output_async async for o in result: if (o := self._get_processed(o, response)) is not None: yield o File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 111, in _evaluate_iterable exception_result: MutableAsyncChain[_T] = self._process_spider_exception( ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ response, ex, exception_processor_index ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception raise exception File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 108, in _evaluate_iterable async for r in iterable: yield r File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/utils/decorators.py", line 152, in asyncgen_inner async for item in func(*args, **kwargs): yield item File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/spidermiddlewares/base.py", line 67, in process_spider_output_async async for o in result: if (o := self._get_processed(o, response)) is not None: yield o File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 111, in _evaluate_iterable exception_result: MutableAsyncChain[_T] = self._process_spider_exception( ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ response, ex, exception_processor_index ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception raise exception File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 108, in _evaluate_iterable async for r in iterable: yield r File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/utils/decorators.py", line 152, in asyncgen_inner async for item in func(*args, **kwargs): yield item File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/spidermiddlewares/base.py", line 67, in process_spider_output_async async for o in result: if (o := self._get_processed(o, response)) is not None: yield o File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 111, in _evaluate_iterable exception_result: MutableAsyncChain[_T] = self._process_spider_exception( ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ response, ex, exception_processor_index ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception raise exception File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 108, in _evaluate_iterable async for r in iterable: yield r File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/utils/decorators.py", line 152, in asyncgen_inner async for item in func(*args, **kwargs): yield item File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/spidermiddlewares/depth.py", line 97, in process_spider_output_async async for o in super().process_spider_output_async(response, result): yield o File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/utils/decorators.py", line 152, in asyncgen_inner async for item in func(*args, **kwargs): yield item File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/spidermiddlewares/base.py", line 67, in process_spider_output_async async for o in result: if (o := self._get_processed(o, response)) is not None: yield o File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 111, in _evaluate_iterable exception_result: MutableAsyncChain[_T] = self._process_spider_exception( ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ response, ex, exception_processor_index ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception raise exception File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 108, in _evaluate_iterable async for r in iterable: yield r File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/utils/decorators.py", line 152, in asyncgen_inner async for item in func(*args, **kwargs): yield item File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/spidermiddlewares/base.py", line 67, in process_spider_output_async async for o in result: if (o := self._get_processed(o, response)) is not None: yield o File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 111, in _evaluate_iterable exception_result: MutableAsyncChain[_T] = self._process_spider_exception( ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ response, ex, exception_processor_index ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception raise exception File "/layers/paketo-buildpacks_python-package-managers-run/uv-env/venv/lib/python3.14/site-packages/scrapy/core/spidermw.py", line 108, in _evaluate_iterable async for r in iterable: yield r File "/workspace/source/app/grabbers/spiders/bases/ctimeetingtech/posters_grabber.py", line 41, in parse_list_page yield self.event_class(poster_data) ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ TypeError: BaseModel.__init__() takes 1 positional argument but 2 were given 2026-09-17 06:19:53 [scrapy.core.engine] INFO: Closing spider (finished) 2026-09-17 06:19:54 [scrapy.extensions.feedexport] INFO: Stored xlsx feed (0 items) in: s3://usummit-prod-grabbers-bucket/ESMO Congress 2025 Posters/ESMO Congress 2025 Posters-2538_2026-09-17_06-16-33.xlsx 2026-09-17 06:19:54 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/request_bytes': 217875, 'downloader/request_count': 273, 'downloader/request_method_count/GET': 273, 'downloader/response_bytes': 874180, 'downloader/response_count': 273, 'downloader/response_status_count/200': 272, 'downloader/response_status_count/202': 1, 'elapsed_time_seconds': 191.37126471893862, 'feedexport/success_count/S3FeedStorage': 1, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2026, 9, 17, 6, 19, 53, 844600, tzinfo=datetime.timezone.utc), 'httpcompression/response_bytes': 6593459, 'httpcompression/response_count': 272, 'items_per_minute': 0.0, 'log_count/ERROR': 1, 'log_count/INFO': 6, 'memusage/max': 197345280, 'memusage/startup': 184373248, 'response_received_count': 272, 'responses_per_minute': 85.44502617801048, 'scheduler/dequeued': 1, 'scheduler/dequeued/memory': 1, 'scheduler/enqueued': 1, 'scheduler/enqueued/memory': 1, 'spider_exceptions/TypeError': 1, 'spider_exceptions/count': 1, 'start_time': datetime.datetime(2026, 9, 17, 6, 16, 42, 473333, tzinfo=datetime.timezone.utc)} 2026-09-17 06:19:54 [scrapy.core.engine] INFO: Spider closed (finished)