Scrape data from a table with scrapy

Question

Scrape data from a table with scrapy. The table html is like: ```

MON, NOV 11	Item		Item	Item
Item	Item	Item		Item
Item	Item	Item		Item

``` Entire list is avialable by ``` items = hxs.select('//table[@class="tablehd"]//td//text()').extract() ``` How would you split them to each item and then assign data td1 - td5ta

Accepted Answer

Not sure what exactly do you want to see in your items, but here's an example and I hope this is it:

```
class MyItem(Item):
    value = Field()

class MySpider(BaseSpider):
    ...

def parse(self, response):
        hxs = HtmlXPathSelector(response)
        items = hxs.select('//table[@class="tablehd"]/td')

for item in items:
            my_item = MyItem()
            my_item['value'] = item.select('.//text()').extract()
            yield my_item
```

Hope that helps.

Scrape data from a table with scrapy

Solution

Problem