Selecting with non-class tag in scrapy python
Multi tool use
up vote
1
down vote
favorite
I am trying to scrap title of a website but the problem it has no class and id.
Usually i use this to get title that has class:
titles = response.xpath('//a[@class="result-title hdrlnk"]/text()').extract()
Now I am trying to extract text, please see the screenshot, can you please fix it? [https://i.stack.imgur.com/k6aCN.png][1]
python xpath scrapy
asked Nov 10 at 15:58
newcoder
111
New contributor
newcoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
up vote
1
down vote
favorite
I am trying to scrap title of a website but the problem it has no class and id.
Usually i use this to get title that has class:
titles = response.xpath('//a[@class="result-title hdrlnk"]/text()').extract()
Now I am trying to extract text, please see the screenshot, can you please fix it? [https://i.stack.imgur.com/k6aCN.png][1]
python xpath scrapy
asked Nov 10 at 15:58
newcoder
111
New contributor
newcoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I am trying to scrap title of a website but the problem it has no class and id.
Usually i use this to get title that has class:
titles = response.xpath('//a[@class="result-title hdrlnk"]/text()').extract()
Now I am trying to extract text, please see the screenshot, can you please fix it? [https://i.stack.imgur.com/k6aCN.png][1]
python xpath scrapy
asked Nov 10 at 15:58
newcoder
111
New contributor
newcoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
I am trying to scrap title of a website but the problem it has no class and id.
Usually i use this to get title that has class:
titles = response.xpath('//a[@class="result-title hdrlnk"]/text()').extract()
Now I am trying to extract text, please see the screenshot, can you please fix it? [https://i.stack.imgur.com/k6aCN.png][1]
python xpath scrapy
python xpath scrapy
asked Nov 10 at 15:58
newcoder
111
New contributor
newcoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Nov 10 at 15:58
newcoder
111
New contributor
newcoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Nov 10 at 15:58
newcoder
111
New contributor
newcoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Nov 10 at 15:58
newcoder
111
asked Nov 10 at 15:58
newcoder
111
111
New contributor
newcoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
newcoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
newcoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
You may locate a specific node by any attribute (not only class and id) or its relative position with some others.
A few examples for the text in your screenshot:
response.xpath('//div[@class="job-title-text"]/a/text()')
response.xpath('//a[contains(@onclick,"clickJObTitle")]/text()')
response.xpath('//a[contains(@href,"jobdetails")]/text()')
response.css('div.job-title-text a::text')
response.css('a[onclick*=clickJObTitle]::text')
response.css('a[href*=jobdetails]::text')
See also:
- https://www.w3schools.com/xml/xpath_syntax.asp
- https://www.w3schools.com/cssref/css_selectors.asp
answered Nov 10 at 16:49
starrify
10.2k42240
Thank you so much
– newcoder
Nov 10 at 17:59
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
You may locate a specific node by any attribute (not only class and id) or its relative position with some others.
A few examples for the text in your screenshot:
response.xpath('//div[@class="job-title-text"]/a/text()')
response.xpath('//a[contains(@onclick,"clickJObTitle")]/text()')
response.xpath('//a[contains(@href,"jobdetails")]/text()')
response.css('div.job-title-text a::text')
response.css('a[onclick*=clickJObTitle]::text')
response.css('a[href*=jobdetails]::text')
See also:
- https://www.w3schools.com/xml/xpath_syntax.asp
- https://www.w3schools.com/cssref/css_selectors.asp
answered Nov 10 at 16:49
starrify
10.2k42240
Thank you so much
– newcoder
Nov 10 at 17:59
add a comment |
up vote
0
down vote
You may locate a specific node by any attribute (not only class and id) or its relative position with some others.
A few examples for the text in your screenshot:
response.xpath('//div[@class="job-title-text"]/a/text()')
response.xpath('//a[contains(@onclick,"clickJObTitle")]/text()')
response.xpath('//a[contains(@href,"jobdetails")]/text()')
response.css('div.job-title-text a::text')
response.css('a[onclick*=clickJObTitle]::text')
response.css('a[href*=jobdetails]::text')
See also:
- https://www.w3schools.com/xml/xpath_syntax.asp
- https://www.w3schools.com/cssref/css_selectors.asp
answered Nov 10 at 16:49
starrify
10.2k42240
Thank you so much
– newcoder
Nov 10 at 17:59
add a comment |
up vote
0
down vote
up vote
0
down vote
You may locate a specific node by any attribute (not only class and id) or its relative position with some others.
A few examples for the text in your screenshot:
response.xpath('//div[@class="job-title-text"]/a/text()')
response.xpath('//a[contains(@onclick,"clickJObTitle")]/text()')
response.xpath('//a[contains(@href,"jobdetails")]/text()')
response.css('div.job-title-text a::text')
response.css('a[onclick*=clickJObTitle]::text')
response.css('a[href*=jobdetails]::text')
See also:
- https://www.w3schools.com/xml/xpath_syntax.asp
- https://www.w3schools.com/cssref/css_selectors.asp
answered Nov 10 at 16:49
starrify
10.2k42240
You may locate a specific node by any attribute (not only class and id) or its relative position with some others.
A few examples for the text in your screenshot:
response.xpath('//div[@class="job-title-text"]/a/text()')
response.xpath('//a[contains(@onclick,"clickJObTitle")]/text()')
response.xpath('//a[contains(@href,"jobdetails")]/text()')
response.css('div.job-title-text a::text')
response.css('a[onclick*=clickJObTitle]::text')
response.css('a[href*=jobdetails]::text')
See also:
- https://www.w3schools.com/xml/xpath_syntax.asp
- https://www.w3schools.com/cssref/css_selectors.asp
answered Nov 10 at 16:49
starrify
10.2k42240
answered Nov 10 at 16:49
starrify
10.2k42240
answered Nov 10 at 16:49
starrify
10.2k42240
answered Nov 10 at 16:49
starrify
10.2k42240
10.2k42240
Thank you so much
– newcoder
Nov 10 at 17:59
add a comment |
Thank you so much
– newcoder
Nov 10 at 17:59
Thank you so much
– newcoder
Nov 10 at 17:59
Thank you so much
– newcoder
Nov 10 at 17:59
add a comment |
newcoder is a new contributor. Be nice, and check out our Code of Conduct.
newcoder is a new contributor. Be nice, and check out our Code of Conduct.
newcoder is a new contributor. Be nice, and check out our Code of Conduct.
newcoder is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53240713%2fselecting-with-non-class-tag-in-scrapy-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Xo6ggK0qeUR5NbxcUFbZNrFKkJY,3eDZ,YUH9DU3j4l0SFt,1UPAh2fv6ux,rM,3 u5ITPzaQK,N6gWymTwF6n3GB
