How to load 9.3 GB data in R studio with 32 GB RAM size

memory_allocation
r

#1

When I load 9.3 gb data in R studio.It shows error…says can’t make vector of size 2048 mb. The system I am using is 64 bit windows with 32 GB RAM with R 64 bit. The limit was supposed to be up to 4 gb vector size.
How can I download my dataset in R?
One more thing, when I used another system of 64 bit…data was loaded in R studio and I did my operations.
But the next day, when I tried to load the same data set.then This system again gave me the error that it can not make vector of size 2048 mb.
Hows this possible?


#2

Hi Ved ,

This a common problem that everyone faces while dealing with large data sets in R.
The problem with “R” is it doesnt allocate the entire RAM to R . for 32 Bit system memory allocation is 4 GB and 64 Bit system 8GB so 9.3GB is more than 8GB so definitely it will fail !

however you can increase your memory size by memory.limit(2010241024*1024) for allocating 20GB to your R

If you want to X-GB memory memory.limit(X10241024*1024)

To check the memory allocated to R pls use memory.size()

I believe this a windows system


#3

Hi Jeeban,

Thanks for your response…Could you please tell me how we can increase memory allocated to single address space?
I have checked 32GB RAM is available for data, 4gb space should be there if I am using 64 bit system with 64 bit R. If I can increase memory for single process then it will definitely run the data set.
I have checked these…
memory.size()
[1] 8700.91

memory.limit()
[1] 32668
memory.size(NA)
[1] 32668
memory.size(T)
[1] 9241.62

This is the exact error- "Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, :
could not allocate memory (2048 Mb) in C function ‘R_AllocStringBuffer’ "


#4

Pls use memory.limit(2010241024*1024) to allocate 20GB of RAM
Hope ur problem will be resolved.


#5

Hi Jeeban,

Problem is still there. The problem is that R is not able to allocate memory more than 2048mb to single address space. Memory.limit is for whole memory that R can use for processing.

If anybody else know solution for this problem…please let me know here.


#6

HI, i just ran the code to expand my memory space i dont know if it will work because i have not testing it, i have a project that i want to run with 70,000 obs and 784 variables
memory.limit(2010241024*1024)