Big Data Framework from Storage to Processing 20,000 stock prices on 20,000 trade setups

What would be the ideal big data framework for downloading the end of day prices only of over 20,000 global stock prices, storing, and then running over 20,000 proprietary trade setups everyday. We would need a cloud infrastructure to spin up nodes so this can be completed fast because on a desktop computer this would take over a month to complete.

© Copyright 2013-2019 Analytics Vidhya