Hi all,

I want to process a large dataset typically in matrix format in R. I am facing memory issue “Error: cannot allocate vector of size 446.8 Mb”

How can i handle this?

Regards,

Karthikeyan P

Hi all,

I want to process a large dataset typically in matrix format in R. I am facing memory issue “Error: cannot allocate vector of size 446.8 Mb”

How can i handle this?

Regards,

Karthikeyan P

How to resolve error in allocation of memory while applying linear regression?

Hi @karthe1,

This is probably because your C drive or where you have R installed is low on memory.R does give memory issues but not on 446.8Mb(otherwise it won’t be really that popular, :P).

One more reason can be that the matrix dimensions are too huge for R to handle.I had faced this issue sometime back while trying matrix rotations by 10,15 degrees.R gave the same error to me.

Then I did two rotations of 5 each and it worked.I guess you have to try something similar.

So either reduce the size of the matrix or do multiple small operations.

Hope this helps!!

Hi @shuvayan,

Thanks for the reply. I dont know the concept of rotation of matrix. What is meant by that? I know taking transpose of a matrix. But what will rotating 5 times will have an effect on the memory size?

Regards.

Karthikeyan P

You can look at this thread for some additional methods:

http://discuss.analyticsvidhya.com/t/what-are-the-ways-to-handle-huge-data-in-r/141

also see this package

http://www.inside-r.org/packages/cran/bigmemory/docs/read.big.matrix

Karthikeyan,

You can also try something like “ff” or “data.table” packages which are meant to handle big data. Most of the function still work in these packages.

Hope this helps.

Tavish