Using data from a website in R

r
json

#1

Hey everybody!
I am an avid reader of the Analytics Vidhya articles and owe a lot of my knowledge to this amazing forum!
I am currently doing an internship and am in a fix.
I have the time stamp and latitude longitude data of a truck and there are a few websites which will provide me with the weather of the particular place at the said time. However I have no idea of how to use this data from my data frame for the website and how to use the data obtained from the website in my data frame.
I would be extremely grateful if someone could help me out with this, Thanks in advance!


#2

@k.shivashish7
Hi there,

This is a very open ended question. I would like to know what are you trying to do with data and what do you actually want to do ? for eg: Do you want to know how to download the data from the website using techniques like scraping etc.

Thanks,
Neeraj


#3

Hey,
I have 21597 observations for the latitude, longitude and the timestamp.
What I want to do is preferably input the lat, long and timestamp through my code to the website and download the output from the website to create another column in my data frame, on which I have to apply a bit of text analytics.
Hope this was helpful, thank you for your reply.


#4

@k.shivashish7
This will require some scripting to automate the process of filling in the information on the website. Rest all is manageable. Give me tonight if no on else answers. I will also probably learn scripting.

Neeraj


#5

@k.shivashish7 - You can use the RODBC package to access the whole database and according you can write a programme which automatically extract the weather by providing the latitude and longitude .

Hope this helps!

Regards,
Hinduja


#6

@k.shivashish7
Hi, I looked into your problem.

There are two approaches you can take.

1- Use websites like openweathermap which provide their own APIs to access their data. It is one of the easiest method to access. You get data in some very popular formats like json,xml etc which are easy to parse.

2- Using selenium IDE or selenium 2.0 depending upon your need. Use IDE when you want this to work for once else I would prefer using Selenium 2.0 to program the entire thing for future reference and use. The programming is done in languages like python and java. So with a bit of googling it will be an achievable task in minimum time with mazimum benefits.

Hope this helps !


#7

If I understand your question well, you want to plot the co-ordinates on to the website(?) I think the best solution for you is using GGMAP and SHINY packages.


#8

Hey ,
Sorry I couldn’t reply earlier, was with my parents.
I did the task, using the first approach in Python.

Thank you so much for the help.
I’m a beginner at Data, thanks for helping me out.