Web Scraping Country name form google search?

python

#1

Hi, Friends

I am new in Python and trying to do web scraping using python.
I have list of cities. Now I want to scrape country name according to city name using google search or any other method.
Can anyone have idea how can I do in python?

i got one code but it gives me all text of city which I search from google:

import urllib
from bs4 import BeautifulSoup
import requests
import webbrowser

text = 'Central London'
text = urllib.parse.quote_plus(text)

url = 'https://google.com/search?q=' + text

response = requests.get(url)

soup = BeautifulSoup(response.text, 'lxml')
for g in soup.find_all(class_='g'):
    print(g.text)
    print('-----')

output:

Central London - Wikipediahttps://en.wikipedia.org/wiki/Central_LondonCachedSimilarCentral London is the innermost part of London, in the United Kingdom, spanning 
several boroughs. Over time, a number of definitions have been used to define 
the scope of central London for statistics, urban planning and local government. 
Its characteristics are understood to include a high density built environment, 
high ...Characteristics - Definitions - London Plan
-----
Central London - visitlondon.comhttps://www.visitlondon.com/things-to-do/london-areas/centralCachedStep into the beating heart of London and lose yourself in its exciting mix of 
tradition, culture, commerce and innovation. Discover historical treasures at the 
British Museum, follow in the footsteps of the famous at Westminster Abbey or get 
a great aerial view from the London Eye.

I just want to country name
How can I do that? all help would be appreciate.

Thanks in Advance


#2

Instead of searching in google, can you try some website where it will give country name if we give city.

That would be easier to scrap


#3

Hi,

If you are looking for countries for capitals, you should use names of cities like london along with query. For example use ‘London country’ as your text and you will get the country name in 1st search result. Further, you can store it in a data frame or list.

Regards
Ankit


#4

@ankit2106
@jprakash0205

Thank you for reply

No I am not looking for only capital of countries.
I almost done but just small issue in scraping.
my code is follows:

import urllib
from bs4 import BeautifulSoup
import requests
import webbrowser

text = 'where is London'

text = urllib.parse.quote_plus(text)

url = 'https://www.google.com/search?q='+ text
print(url)

response = requests.get(url)

soup = BeautifulSoup(response.text, 'lxml')

country = soup.find('h3', class_ = 'r').a.text
print(country) 

I get city name and country name both but now I am not getting how can I get only country name


#5

It must be returning a string as a query result. You could use str.split to get the country name.


#6

Yes I used set() command for keywords


#7

You could modify the code a bit.

country=soup.find(‘h3’, class_ = ‘r’).get_text(’\n’)

You will get the output as
London
United Kingdom

you can extract the country name by converting the string to list.


#8

@A.Malathi

Thank you so much. It works