Historical Stock Data?

[Requiem]

Does anybody know where I can find some high granularity stock data for months, years, or preferably decades? I'm looking specifically at the granularity, I'd like minute or even second capture intervals.

The highest granularity I've been able to find in 30 minutes on google is daily, which isn't useful for what I have in mind.

Any ideas?

Edit: Before somebody asks for what stocks, for anything. For all stocks in the world, ideally. I'm looking to do some serious data mining.

[The Ancient]

For what?

[r3probate]

I don't think you could get data that fine without paying for a Bloomberg terminal or other professional trading platform, rinthea or prescient might know better.

[prescient63]

It sounds like you want something like compustat data and it isn't free. I am curious as to what you want to use it for. The only real reasons I see for having it are either academic or HFT, and HFT requires colocation, sub millisecond feeds, and a host of other infrastructure hurdles to overcome.

[Requiem]

I want to use it to develop an HFT algorithm, yes.

[prescient63]

You could enroll in a course at a college where they have access to compustat data.

Could try these guys as they seem pretty cheap Tick Data - Pricing

I know very little about HFT trading but I would think that you will additionally need market depth info as bid/ask spreads don't tell you too much.

IE 10 share buy on one side and a 3 share sell on the other.

While I'm thiking about it you're really going to be handicapped against firms that have access to flash orders. I'm not trying to be a dick but HFT trading requires significant capital investments because you're up against GS, Citadel, Renaissance, etc...... Plus a good chunk of profits are probably coming from liquidity rebates.

[Requiem]

That's not a bad idea. I've requested some quotes, but I have a feeling it's going to be big numbers.

[Requiem]

Originally Posted by prescient63

While I'm thiking about it you're really going to be handicapped against firms that have access to flash orders. I'm not trying to be a dick but HFT trading requires significant capital investments because you're up against GS, Citadel, Renaissance, etc...... Plus a good chunk of profits are probably coming from liquidity rebates.

HFT may not be a good characterization of what I want to do. I'm not looking to compete on that level, I just think that I can generate an algorithm that trades on fairly small intervals that makes me some money for little effort. I'm good at that kind of work. Rather than approach it from the empirical/theoretical side of things and come up with something by hand, I intend to borrow from graph theory and generate a big data structure that I can send agent swarms through to distill it into some PSO/SDS metrics that I can then derive an algorithm from. I have lots of idle cores that could be doing that sort of thing. If I can get a whole decade of data to derive an algorithm from, and then run simulations on with a positive result, the risk of giving it some money and letting it do it's thing seems minimal.

[Lyenae]

You just want to create an automated trading system then. That's what you want to look up/research. As for quotes, you will probably have to pay for having 'intraday' data past a certain time frame.

[Tyen]

First.

Membership Levels

[prescient63]

That bar data only goes back 7 months though. If you really want to do something like this you need to get your hands on compustat data or the equivalent.

Additionally, I don't think I would bother going back decades looking for patterns. I'm hardly an expert but I imagine that trading has changed enough in the last 5-10 years to make everything before that useless. I would think that 5 years of data is enough, and your data set will be huge if you're getting tick data. But like I said wtf do I know?

I would actually like to try something like this though I just don't want to spend any money on it. I had a little automated arbitrage model for trading ETFs that would probably work but hasn't been fully tested. Plus, I wouldn't want to put the amount of cash at risk that I would need to (I'm thinking min 100k to try out my hair brained idea and 1mil would be better). However it would be fun to play with if I could find a place to just do the fake executions for me and they had some sort of api I could interact with.

Edit: And, no I can't back test because I don't know of a source of time series swap curve data needed as an input.

[FulorianC]

I would tend to agree. The demographics and the technology behind trading has transitioned so immensely that any purely technical information you would glean from data that fine would be worthless past about 1995.

[rinthea]

You'll have to buy it if you want decent data. You get what you pay for too, lots of data is bad and big firms doing this sort of trading have their own proprietary data sets. You could buy the tick data (data of each trade) fairly cheaply from an emini contract and that would be enough to start. It seems to me more practical just using one contract of tick data, then a decades worth a longer time frame data, from the perspective that markets change and what happened 10 years ago is not necessarily relevant to whats going to happen for a few minutes next week.

But, data mining and curve fitting will get you know where. Execution, commissions and speed are most important. Competing with the big guys that keep their servers in the same building as the exchange, with massive amounts of capital and resources sounds like suicide to me.

Also, there is a huge difference between simulated trading and real trading, from a practical point of actually getting fills, slippage and costs etc.

"I just think that I can generate an algorithm that trades on fairly small intervals that makes me some money for little effort." This sounds to me a little naive. If only it was that easy! Good luck though.

There's a lot of nerds over the Wilmott | Serving The Quantitative Finance Community | Home forums which is you may find interesting/relevant.

[Hachima]

Test your trades with TradeStation's high quality, intra-day, historical data

You can get 6 months of tick data, and much longer for non tick data.

It's probably not exactly what you want but maybe it will interest you.

[Gauss]

Originally Posted by prescient63

While I'm thiking about it you're really going to be handicapped against firms that have access to flash orders. I'm not trying to be a dick but HFT trading requires significant capital investments because you're up against GS, Citadel, Renaissance, etc...... Plus a good chunk of profits are probably coming from liquidity rebates.

Didn't the SEC just propose to close that loophole with flash orders?

Rather than approach it from the empirical/theoretical side of things and come up with something by hand, I intend to borrow from graph theory and generate a big data structure that I can send agent swarms through to distill it into some PSO/SDS metrics that I can then derive an algorithm from. I have lots of idle cores that could be doing that sort of thing. If I can get a whole decade of data to derive an algorithm from, and then run simulations on with a positive result, the risk of giving it some money and letting it do it's thing seems minimal.

Best of luck to you. I know that some guys here in the CS department that have made some very nice money using algorithms to do trading. Of course, they refuse to say how much :X