Computer Resource Requirement for Data Science Competitions


#1

Hi Folks.

In R, I usually get out of memory errors while running a linear model on a big dataset or random forest with 5000 trees. I usually work on my laptop on windows 7 32 bit OS, 1 CPU, 4G RAM.
Ány idea what is the minimum recommended computer configuration for working on dataset provided in Data Science Competitions? Is there any cost effective solution to this than getting a new upgraded laptop/computer?

Thanks in advance.


#2

@Rajiv_Iyer,

Kunal has written a article on resource requirement to work on data science competitions.

Regards,
Sunil


#3

@Rajiv_Iyer

As Sunil has mentioned, the article should help.

If you are serious, the best option is to upgrade to a new machine as recommended in the article. A new machine with quad core 3rd / 4th gen chip and 16 GB RAM would cost around 50 - 60k INR.

If you can not upgrade, then try maxing out the RAM. If the model is 64 bit compatible, move to a 64 bit system. Also, I would move to Linux as well.

Regards,
Kunal


#4

@kunal @Sunil
Thanks for your suggestions and help. I have better clarity after going through the article. I initially considered going for cloud services. But as you said that does not give the same kind of availability & freedom as a personal machine can. And eventually in the long run a personal machine turns out to be cost effective as well.