You have a new project at work/grad school, how do you plan the project's execution?


#1

Ok so, I’m an Economist in Central Banking. My job is to -literally - extract data (not info) from whatever datasets made available to us (Stock Exchanges, financial derivatives’ operations, Trade Chambers, and so forth). We filter these huge datasets so we get data that is well defined and can be analyzed by Academics and industry researchers.

This is the tricky part: I receive several datasets, in many file-types, shapes and sizes. My question is: How should I go about it? What steps should I take?

I want to develop a methodology, a series of steps to be taken whenever trying to process a new data frame, from a raw dataset.


#2

Hi @ocarreon

I get a sense that you probably want to automate your work. But can you please specify your language of preference and what exactly is the purpose that you want to fulfilby developing this “methodology”?


#3

Thanks for replying @sauravkaushik8. My question is more general than that.

I work with R and Excel, the former is my tool for large data sets manipulation, the latter is mandatory (format, shapes, presentation, blabla).

What I want to work on is developing a general method, some step by step thing, that helps me go through whatever new data-based project. My problem these days is: whatever changes in the size or file types from one project to another makes me go back and start from scratch. Not so versatile, huh.

I’ve been advise in another forum to find a Tutor. I figure this is the kind of stuff one cannot learn from books.