Xts Quantmod Workshop

Working with xts and quantmodLeveraging R with xts and quantmod for quantitative trading Jeffrey A. Ryan [email protected] R/Finance 2009 Workshop Presented on April 24, 2009 R/Finance 2009: Applied Finance with R University of Illinois at Chicago xts: extensible time series Most everything in trading involves a time series Regular data (positions data, P&L) Irregular data (market data, book data, trades) R has many ways to manage this... xts: extensible time series Data Classes xts: extensible time series fts Data Classes matrix mts tframe data.frame timeSeries ts zoo irts vectors zooreg .table its data. xts: extensible time series fts Data Classes matrix mts tframe data.table its data.frame timeSeries ts zoo irts vectors Time Classes zooreg xts: extensible time series fts Data Classes matrix mts tframe data.table its data.frame timeSeries chron yearmon ts zoo irts vectors Time Classes zooreg character numeric timeDate POSIXct Date POSIXlt yearqtr xts: extensible time series fts Data Classes matrix mts tframe ts irts Sometimes, choice is timeSeries bad for package zooreg vectors developers and interoperability character chron POSIXct Time Classes Date numeric POSIXlt yearmon timeDate yearqtr data.table its data.frame zoo xts: extensible time series The “solution” ? . xts: extensible time series add one more class of course... . . 2007) Avid user of zoo • Natural R-like interface • Flexible and complete methods • S3! I still wanted a few features for trading.xts: extensible time series Motivation (c.. • Additional series metadata • Require time-based indexing • Conversion/reconversion tools . xts: extensible time series Significant design requirements for xts: • Preserve zoo behavior • Utilize time-based indexing • Allow for arbitrary attributes to be cleanly attached • ISO 8601 subsetting by time strings • Lossless conversion utilities to hide messy details . . index .CLASS.indexFORMAT.indexTZ. .xts: extensible time series What’s inside an xts object? Index + Matrix + Attr Internally the storage is always a numeric vector! Can be of any class supported by matrix Hidden attributes AND user attributes . .indexCLASS. index . .indexCLASS.CLASS.indexFORMAT. . .xts: extensible time series What’s inside an xts object? Index + Matrix + Attr Internally the storage is always a numeric vector! Can be of any class supported by matrix Hidden attributes AND user attributes .indexTZ. indexFORMAT. . .CLASS. .xts: extensible time series What’s inside an xts object? Index + Matrix + Attr Internally the storage is always a numeric vector! Can be of any class supported by matrix Hidden attributes AND user attributes .indexTZ. index .indexCLASS. .indexTZ. index .indexFORMAT. .CLASS.xts: extensible time series What’s inside an xts object? Index + Matrix + Attr Internally the storage is always a numeric vector! Can be of any class supported by matrix Hidden attributes AND user attributes . .indexCLASS. . index Important! index must be a time-based class .indexCLASS.indexTZ.xts: extensible time series What’s inside an xts object? Index + Matrix + Attr Internally the storage is always a numeric vector! Can be of any class supported by matrix Hidden attributes AND user attributes .CLASS. .indexFORMAT. . xts: extensible time series Index as numeric? That isn’t “time-based”!! • Internally all index values are represented in POSIX time (seconds since the epoch) • Coercion happens at object creation or upon index replacement • index() converts back to user-level class • .indexFORMAT and .indexTZ attributes • Rationale? Simplicity for C level code. removal of multiple conversion in most instances.indexCLASS. .index() access the raw seconds in the index • . more consistent behavior • All details are hidden from the user . org/ISO_8601 .g.wikipedia. CCYYMMDDHHMMSS • Reduced accuracy forms are valid: e.s] • Fixed number of digits • Separators can be omitted e.g. CCYY-MM • Fractional decimal time is supported • Intervals can be expressed e.g. 2000-05/2001-04 source: http://en.xts: extensible time series Time-based indexing in xts (ISO 8601) • Date and time organized from most significant to least significant: CCYY-MM-DD HH:MM:SS[. xts: extensible time series Create an xts object > library(xts) Loading required package: zoo xts now requires a valid TZ variable to be set your current TZ:America/Chicago > x <.1268833 2009-03-27 -0.6945146 2009-03-28 -0.2152293 2009-04-01 0.1938840 2009-03-30 0.Date()+1:10) >x [.xts(rnorm(10). Sys.2368576 2009-03-31 -1.2812633 2009-03-26 0.3936148 2009-03-29 -0.1] 2009-03-24 0.8100493 2009-04-02 1.3554788 2009-03-25 1.4152439 Load xts package . 1268833 2009-03-27 -0.xts(rnorm(10).6945146 2009-03-28 -0.2812633 2009-03-26 0.3936148 2009-03-29 -0.4152439 .2152293 2009-04-01 0.3554788 2009-03-25 1.xts: extensible time series Create an xts object > library(xts) Loading required package: zoo xts now requires a valid TZ variable to be set your current TZ:America/Chicago Create an xts object > x <.1] 2009-03-24 0.1938840 2009-03-30 0.8100493 2009-04-02 1. Sys.2368576 2009-03-31 -1.Date()+1:10) >x [. 2152293 2009-04-01 0.2368576 2009-03-31 -1.1] 2009-03-28 -0.8100493 > x["20090328/20090401"] [.Date("2009-04-01") ] [.2152293 2009-04-01 0.1938840 2009-03-30 0.xts: extensible time series Indexing by time Index using standard tools (still works) > x[ index(x) >= as.3936148 2009-03-29 -0.2368576 2009-03-31 -1.1] 2009-03-28 -0.8100493 .3936148 2009-03-29 -0.Date("2009-03-28") & index(x) <= + as.1938840 2009-03-30 0. 1] 2009-03-28 -0.Date("2009-03-28") & index(x) <= + as.1] 2009-03-28 -0.xts: extensible time series Indexing by time > x[ index(x) >= as.3936148 2009-03-29 -0.2368576 2009-03-31 -1.2368576 2009-03-31 -1.8100493 Index via ISO-style with xts .2152293 2009-04-01 0.Date("2009-04-01") ] [.1938840 2009-03-30 0.3936148 2009-03-29 -0.1938840 2009-03-30 0.2152293 2009-04-01 0.8100493 > x["20090328/20090401"] [. 2812633 2009-03-26 0.3936148 2009-03-29 -0.2368576 2009-03-31 -1.8100493 2009-04-02 1.3554788 2009-03-25 1.1268833 2009-03-27 -0.2152293 > x["2009"] [.4152439 All of 2009 .1] 2009-04-01 0.2368576 2009-03-31 -1.1938840 2009-03-30 0.6945146 2009-03-28 -0.1] 2009-03-24 0.xts: extensible time series Indexing by time > x["20090301/200903"] [.1268833 2009-03-27 -0.6945146 2009-03-28 -0.1] 2009-03-24 0.3936148 2009-03-29 -0.3554788 2009-03-25 1.1938840 2009-03-30 0.4152439 > x["200904"] [.8100493 2009-04-02 1.2152293 2009-04-01 0.2812633 2009-03-26 0. 1268833 2009-03-27 -0.2812633 2009-03-26 0.1268833 2009-03-27 -0.2368576 2009-03-31 -1.8100493 2009-04-02 1.6945146 2009-03-28 -0.2368576 2009-03-31 -1.6945146 2009-03-28 -0.1] 2009-04-01 0.4152439 All of April 2009 .3554788 2009-03-25 1.3936148 2009-03-29 -0.2152293 2009-04-01 0.1938840 2009-03-30 0.1938840 2009-03-30 0.2812633 2009-03-26 0.1] 2009-03-24 0.3936148 2009-03-29 -0.3554788 2009-03-25 1.2152293 > x["2009"] [.4152439 > x["200904"] [.xts: extensible time series Indexing by time > x["20090301/200903"] [.8100493 2009-04-02 1.1] 2009-03-24 0. 1268833 2009-03-27 -0.2812633 2009-03-26 0.6945146 2009-03-28 -0.2152293 > x["2009"] [.3554788 2009-03-25 1.2368576 2009-03-31 -1.4152439 > x["200904"] [.xts: extensible time series Indexing by time > x["20090301/200903"] [.2152293 2009-04-01 0.1] 2009-03-24 0.4152439 From the first March to the end of March .3554788 2009-03-25 1.2812633 2009-03-26 0.8100493 2009-04-02 1.3936148 2009-03-29 -0.1938840 2009-03-30 0.2368576 2009-03-31 -1.1268833 2009-03-27 -0.1] 2009-03-24 0.6945146 2009-03-28 -0.8100493 2009-04-02 1.1938840 2009-03-30 0.3936148 2009-03-29 -0.1] 2009-04-01 0. 832 .006 0..time(x10m['20090323']) user system elapsed 0.POSIXct] TZ: America/Chicago xts Attributes: NULL > str(x100k) # 100 thousand observations An 'xts' object from 2009-03-23 16:19:00 to 2009-03-24 20:05:39 containing: Data: int [1:100000. 1] 1 2 3 4 5 6 7 8 9 10 . 1] 1 2 3 4 5 6 7 8 9 10 .001 0. Indexed by objects of class: [POSIXt.POSIXct('2009-03-23 23:59:58')]) user system elapsed 1.time(x100k['20090323']) user system elapsed 0.xts: extensible time series All subsetting is via a binary search algorithm.POSIXct] TZ: America/Chicago xts Attributes: NULL > system.006 > system.457 1.006 0.372 2.. F-A-S-T! > str(x10m) # 10 million observations An 'xts' object from 2009-03-23 16:19:00 to 2009-07-17 10:05:39 containing: Data: int [1:10000000.001 0.006 > system.time(x10m[index(x10m) >= as..POSIXct('2009-03-23 16:19:00') & index(x10m) <= as. Indexed by objects of class: [POSIXt.. F-A-S-T! > str(x10m) # 10 million observations An 'xts' object from 2009-03-23 16:19:00 to 2009-07-17 10:05:39 containing: Data: int [1:10000000.006 > system.POSIXct] TZ: America/Chicago xts Attributes: NULL > str(x100k) # 100 thousand observations An 'xts' object from 2009-03-23 16:19:00 to 2009-03-24 20:05:39 containing: Data: int [1:100000.time(x10m[index(x10m) >= as.006 0.006 > system.POSIXct('2009-03-23 16:19:00') & index(x10m) <= as.001 0. 1] 1 2 3 4 5 6 7 8 9 10 .time(x100k['20090323']) user system elapsed 0... Indexed by objects of class: [POSIXt.832 .POSIXct('2009-03-23 23:59:58')]) user system elapsed 1.457 1. Indexed by objects of class: [POSIXt.372 2.001 0.006 0. 1] 1 2 3 4 5 6 7 8 9 10 .xts: extensible time series All subsetting is via a binary search algorithm.time(x10m['20090323']) user system elapsed 0..POSIXct] TZ: America/Chicago xts Attributes: NULL > system.. 372 2.POSIXct('2009-03-23 23:59:58')]) user system elapsed 1.. Indexed by objects of class: [POSIXt.006 0.POSIXct] TZ: America/Chicago xts Attributes: NULL > str(x100k) # 100 thousand observations An 'xts' object from 2009-03-23 16:19:00 to 2009-03-24 20:05:39 containing: Data: int [1:100000.006 Identical speed! > system..time(x100k['20090323']) user system elapsed 0.time(x10m['20090323']) user system elapsed 0.001 0. 1] 1 2 3 4 5 6 7 8 9 10 .POSIXct('2009-03-23 16:19:00') & index(x10m) <= as.. Indexed by objects of class: [POSIXt.POSIXct] TZ: America/Chicago xts Attributes: NULL > system.006 0.001 0. F-A-S-T! > str(x10m) # 10 million observations An 'xts' object from 2009-03-23 16:19:00 to 2009-07-17 10:05:39 containing: Data: int [1:10000000..time(x10m[index(x10m) >= as.006 > system.832 .457 1.xts: extensible time series All subsetting is via a binary search algorithm. 1] 1 2 3 4 5 6 7 8 9 10 . 006 > system.372 2..POSIXct] TZ: America/Chicago xts Attributes: NULL > str(x100k) # 100 thousand observations An 'xts' object from 2009-03-23 16:19:00 to 2009-03-24 20:05:39 containing: Data: int [1:100000. 1] 1 2 3 4 5 6 7 8 9 10 .832 .006 > system. Indexed by objects of class: [POSIXt.457 1.time(x100k['20090323']) user system elapsed 0.POSIXct('2009-03-23 16:19:00') & index(x10m) <= as.POSIXct] TZ: America/Chicago xts Attributes: NULL > system...POSIXct('2009-03-23 23:59:58')]) user system elapsed 1. F-A-S-T! > str(x10m) # 10 million observations An 'xts' object from 2009-03-23 16:19:00 to 2009-07-17 10:05:39 containing: Data: int [1:10000000.time(x10m['20090323']) user system elapsed 0.001 0.xts: extensible time series All subsetting is via a binary search algorithm. 1] 1 2 3 4 5 6 7 8 9 10 .006 0.. Indexed by objects of class: [POSIXt.006 0.001 0.time(x10m[index(x10m) >= as. . xts now has 3000+ lines of C . Prime C candidate • Implemented optimized sort-merge-join in C with custom algorithm • Additional C based routines followed.xts to C dramatically decreased subsetting costs • Highest cost basic operation in R was merge.xts: extensible time series xts + C • Moving [.. period.apply.xts.period.xts: extensible time series Additional xts tools to. reclass. Reclass . try. endpoints. timeBasedRange. xts: extensible time series ...in development . xts: extensible time series Binary .xd files Representation of xts objects on disk Seekable for disk-based subsetting Future time-series database structure XTS (disk) xts (memory) xtsDB . runCov. etc.apply runSum. (moving from TTR) .xts: extensible time series Parallel processing period. runSD. runSD. runCov. etc. Multiple index support dimensional attributes .xts: extensible time series Parallel processing period.apply runSum. runSD. runCov.xts: extensible time series Parallel processing period. etc. Multiple index support dimensional attributes Tighter zoo integration Backport C code into zoo .apply runSum. AAPL 105 100 95 90 85 80 Dec 01 2008 Dec 15 2008 Dec 29 2008 Jan 12 2009 Jan 26 2009 [2008−12−01/2009−03−23] Feb 09 2009 Feb 23 2009 Mar 09 2009 Mar 23 2009 quantmod . quantmod quantmod was envisioned to be a rapid prototyping environment in R to facilitate quantitative modeling. and trading . testing. quantmod Data. Visualization. Modeling. quantmod Data. Visualization. Modeling. Trading requires lots of different types of data, from many different sources. quantmod aims to hide the details of the data source, to make using data a priority quantmod Data. Visualization. Modeling. Trading requires lots of different types of data, from many different sources. quantmod aims to hide the details of the data source, to make using data a priority getSymbols quantmod Data. getSymbols csv SQLite Interactive Brokers Rdata google FRED MySQL yahoo oanda . Visualization. Modeling. Modeling. getSymbols getSymbols is the top level function that dispatches to custom methods based on user direction setSymbolLookup getSymbolLookup saveSymbolLookup loadSymbolLookup .quantmod Data. Visualization. Modeling.quantmod Data. getSymbols getSymbols behave like base::load by assigning objects into the user’s workspace (. Visualization.GlobalEnv) . it is redundant to have to manually assign each. Also facilitates multiple requests. Modeling. Visualization. . getSymbols getSymbols behave like base::load by assigning objects into the user’s workspace (.quantmod Data.GlobalEnv) Rationale: when dealing with potentially dozens of symbols interactively. Modeling. getSymbols behave like base::load by assigning objects into the user’s workspace (.GlobalEnv) getSymbols (devel) now returns all symbols in an environment! loadSymbols will be available to directly replace the previous getSymbols behavior U D E T A D P Visualization. getSymbols .quantmod Data. getSymbols getSymbols(“AAPL”) getSymbols(“AAPL.quantmod Data.SBUX”) getSymbols(“USD/EUR”.src= “oanda”) . Modeling. Visualization. getFX. getOptionChain .src= “oanda”) Additional data wrappers: getDividends. getMetals.quantmod Data. Visualization. getSplits. Modeling. getQuote. getSymbols getSymbols(“AAPL”) getSymbols(“AAPL.SBUX”) getSymbols(“USD/EUR”. quantmod Data. highly customizable financial charting in R . Visualization. Modeling. Interactive. Visualization. ^gspc Modeling. price 700 2009−02−02 750 800 850 2009−02−14 2009−02−27 Time 2009−03−11 2009−03−23 Basic OHLC chart from tseries .quantmod Data. quantmod Data. GSPC 850 Visualization. subset='200902/'. [2009−02−02/2009−03−23] 800 750 700 Feb 02 2009 Feb 09 2009 Feb 17 2009 Feb 23 2009 Mar 02 2009 Mar 09 2009 Mar 16 2009 Mar 23 2009 candleChart(GSPC. TA=NULL) . Modeling. theme='white'. Modeling.quantmod Data. Visualization. Requirements • Fast rendering (base plotting tools) • Interactive and scriptable • Work with all timeseries classes • Minimal commands • Highly customizable • Full technical analysis support (via TTR) . 200 Jan 03 2007 Apr 02 2007 Jul 02 2007 Oct 01 2007 Jan 02 2008 Apr 01 2008 Jul 01 2008 Oct 01 2008 Jan 02 2009 Mar 23 2009 > chartSeries(IBM) . [2007−01−03/2009−03−23] Volume (millions): 12. Visualization.quantmod Data.407.71 Modeling. The Basics IBM 130 120 110 100 90 80 70 30 25 20 15 10 5 Last 98. 200 Jan 03 2007 Apr 02 2007 Jul 02 2007 Oct 01 2007 Jan 02 2008 Apr 01 2008 Jul 01 2008 Oct 01 2008 Jan 02 2009 Mar 23 2009 > addBBands() .71 Bollinger Bands (20.220 Modeling. The Basics IBM 130 120 110 100 90 80 70 30 25 20 15 10 5 Last 98.670/83. [2007−01−03/2009−03−23] Volume (millions): 12.2) [Upper/Lower]: 95.quantmod Data.407. Visualization. 71 Bollinger Bands (20.50.9): MACD: 1.407.26.200 Moving Average Convergence Divergence (12.670/83. The Basics Modeling.220 Visualization.392 Jan 03 2007 Mar 01 2007 May 01 2007 Jul 02 2007 Sep 04 2007 Nov 01 2007 Jan 02 2008 Mar 03 2008 May 01 2008 Jul 01 2008 Sep 02 2008 Nov 03 2008 Jan 02 2009 Mar 02 2009 > addMACD(32.quantmod Data. [2007−01−03/2009−03−23] Volume (millions): 12.12) .418 Signal: 0. IBM 130 120 110 100 90 80 70 30 25 20 15 10 5 5 0 −5 Last 98.2) [Upper/Lower]: 95. 392 Jan 02 2009 Jan 12 2009 Jan 20 2009 Jan 26 2009 Feb 02 2009 Feb 09 2009 Feb 17 2009 Feb 23 2009 Mar 02 2009 Mar 09 2009 Mar 16 2009 Mar 23 2009 > reChart(subset= “2009”. type= “candles”) .418 Signal: 0. [2009−01−02/2009−03−23] 90 85 25 20 15 10 3 2 1 0 −1 −2 −3 Volume (millions): 12.670/83. IBM Last 98.quantmod Data.407.theme= “white”. The Basics Modeling.71 Bollinger Bands (20.2) [Upper/Lower]: 95.200 Moving Average Convergence Divergence (12.26.9): MACD: 1.220 95 Visualization. 392 Jan 02 2009 Jan 20 2009 Feb 02 2009 Feb 17 2009 Mar 02 2009 Mar 16 2009 .66 Inside chartSeries [2007−01−03/2009−03−23] chartSeries chob (chart object) 120 100 80 120 100 80 60 40 20 Volume (millions): 23. Visualization.quantmod Data. AAPL 200 180 160 140 Last 107.670/83.900 Jan 03 2007 May 01 2007 Sep 04 2007 Jan 02 2008 May 01 2008 Sep 02 2008 Jan 02 2009 IBM [2009−01−02/2009−03−23] Last 98.799.71 Bollinger Bands (20.9): MACD: 1.418 Signal: 0.2) [Upper/Lower]: 95.220 95 addTA chobTA 90 85 3 2 1 0 −1 −2 −3 Moving Average Convergence Divergence (12.26. Modeling. AAPL 200 180 160 140 Last 107.670/83.799.418 Signal: 0.26. Visualization.66 Inside chartSeries [2007−01−03/2009−03−23] chartSeries chob (chart object) 120 100 80 120 100 80 60 40 20 Volume (millions): 23.9): MACD: 1.quantmod Data.2) [Upper/Lower]: 95.900 Jan 03 2007 May 01 2007 Sep 04 2007 Jan 02 2008 May 01 2008 Sep 02 2008 Jan 02 2009 IBM [2009−01−02/2009−03−23] Last 98.392 Jan 02 2009 Jan 20 2009 Feb 02 2009 Feb 17 2009 Mar 02 2009 Mar 16 2009 .220 95 addTA chobTA 90 85 3 2 1 0 −1 −2 −3 Moving Average Convergence Divergence (12.71 Bollinger Bands (20. Modeling. 2) [Upper/Lower]: 95.66 Inside chartSeries [2007−01−03/2009−03−23] chartSeries chob (chart object) 120 100 80 120 100 80 60 40 20 Volume (millions): 23.392 Jan 02 2009 Jan 20 2009 Feb 02 2009 Feb 17 2009 Mar 02 2009 Mar 16 2009 .900 Jan 03 2007 May 01 2007 Sep 04 2007 Jan 02 2008 May 01 2008 Sep 02 2008 Jan 02 2009 IBM [2009−01−02/2009−03−23] Last 98. Modeling.quantmod Data. Visualization.418 Signal: 0.220 95 addTA chobTA 90 85 3 2 1 0 −1 −2 −3 Moving Average Convergence Divergence (12.71 Bollinger Bands (20.799.26.670/83. AAPL 200 180 160 140 Last 107.9): MACD: 1. 670/83.2) [Upper/Lower]: 95.chob .392 Jan 02 2009 Jan 20 2009 Feb 02 2009 Feb 17 2009 Mar 02 2009 Mar 16 2009 Drawn by chartSeries.9): MACD: 1. AAPL 200 180 160 140 Last 107.220 95 addTA chobTA 90 85 3 2 1 0 −1 −2 −3 Moving Average Convergence Divergence (12.418 Signal: 0.71 Bollinger Bands (20.26.quantmod Data.900 Jan 03 2007 May 01 2007 Sep 04 2007 Jan 02 2008 May 01 2008 Sep 02 2008 Jan 02 2009 IBM [2009−01−02/2009−03−23] Last 98. Modeling. Visualization.799.66 Inside chartSeries [2007−01−03/2009−03−23] chartSeries chob (chart object) 120 100 80 120 100 80 60 40 20 Volume (millions): 23. Extending chartSeries . Visualization.quantmod Data. Modeling. quantmod Data. Visualization. GMMA Guppy Multiple Moving Average (with newTA) . Modeling. 12. + function(xx) EMA(x.45.c(3.60) + x <. Modeling.quantmod Data.c(30.slowMA). Visualization.10.35.function(x) { + fastMA <.sapply(c(fastMA.8. > # create a function that returns our GMMA > GMMA <.15) + slowMA <.xx)) + return(x) +} > .5.50.40. 10.15) + slowMA <.40.45. + function(xx) EMA(x.50.60) + x <. + col=c(rep(3.c(3.6)).function(x) { + fastMA <.5.c(30. Visualization.8. > # create a function that returns our GMMA > GMMA <. + rep(”#333333”.sapply(c(fastMA. + preFUN=Cl. + legend=”GMMA”) > class(addGuppy) [1] “function” .35.6). Modeling.12.slowMA).xx)) + return(x) +} > > # create an addGuppy function with newTA > addGuppy <.quantmod Data.newTA(FUN=GMMA. + col=c(rep(3. candleChart(AAPL).40.15) + slowMA <.slowMA).10.12.xx)) + return(x) +} > .5.8.sapply(c(fastMA. Visualization.60) + x <.6)). Modeling.35. + preFUN=Cl.50.newTA(FUN=GMMA. + function(xx) EMA(x.45. + legend=”GMMA”) > class(addGuppy) [1] “function” > # create a function that returns our GMMA > GMMA <.quantmod Data. + rep(”#333333”.c(3.c(30. addGuppy() > # create an addGuppy function with newTA > addGuppy <.function(x) { + fastMA <.6). 900 50 : 105.991. [2008−08−01/2008−12−01] 140 120 100 80 Volume (millions): 32.566 35 : 98.625 8 : 90.93 160 Visualization.726 45 : 102.695 5 : 90.556 10 : 90.117 15 : 91.241 Aug 01 2008 Aug 11 2008 Aug 18 2008 Aug 25 2008 Sep 02 2008 Sep 15 2008 Sep 22 2008 Sep 29 2008 Oct 06 2008 Oct 13 2008 Oct 20 2008 Oct 27 2008 Nov 03 2008 Nov 10 2008 Nov 17 2008 Nov 24 2008 .593 40 : 100.quantmod Data. AAPL 180 Last 88.066 60 : 109. Modeling.700 80 60 40 20 180 160 140 120 100 80 GMMA () : 3 : 90.771 12 : 91.774 30 : 96. col=c(rep(“blue”. Modeling.241 Aug 01 2008 Aug 11 2008 Aug 18 2008 Aug 25 2008 Sep 02 2008 Sep 15 2008 Sep 22 2008 Sep 29 2008 Oct 06 2008 Oct 13 2008 Oct 20 2008 Oct 27 2008 Nov 03 2008 Nov 10 2008 Nov 17 2008 Nov 24 2008 .900 50 : 105.93 160 Visualization. AAPL 180 Last 88.991.771 12 : 91.695 5 : 90.quantmod Data.6))) GMMA () : 3 : 90. rep(“black”.774 30 : 96.593 40 : 100.6).556 10 : 90.700 80 60 40 20 180 160 140 120 100 80 addGuppy(on=-1.726 45 : 102. [2008−08−01/2008−12−01] 140 120 100 80 Volume (millions): 32.625 8 : 90.566 35 : 98.066 60 : 109.117 15 : 91. 695 5 : 90.6))) GMMA () : 3 : 90.900 50 : 105.556 10 : 90.771 12 : 91.117 15 : 91.quantmod Data.566 35 : 98.625 8 : 90. Modeling.991.726 45 : 102. [2008−08−01/2008−12−01] on arg selects panel 140 120 100 80 Volume (millions): 32. rep(“black”. AAPL 180 Last 88. col=c(rep(“blue”.700 80 60 40 20 180 160 140 120 100 80 addGuppy(on=-1.593 40 : 100.066 60 : 109.93 160 Visualization.774 30 : 96.6).241 Aug 01 2008 Aug 11 2008 Aug 18 2008 Aug 25 2008 Sep 02 2008 Sep 15 2008 Sep 22 2008 Sep 29 2008 Oct 06 2008 Oct 13 2008 Oct 20 2008 Oct 27 2008 Nov 03 2008 Nov 10 2008 Nov 17 2008 Nov 24 2008 . 300 : 107. AAPL Last 125. Modeling.342 : 105.545 : 119.719.quantmod Data.541 : 121.4 : 123.937 120 Visualization.361 : 110.680 : 118.600 .447 : 108.371 : 120.348 : 112.429 : 122.796 : 109. [2009−01−02/2009−04−23] 110 100 90 80 60 50 40 30 20 Jan 02 2009 Jan 20 2009 Feb 02 2009 Feb 17 2009 Mar 02 2009 Mar 16 2009 Mar 30 2009 Apr 13 2009 Apr 23 2009 Volume (millions): 33. Visualization. 5% 4% 3% 2% 1% Jan 02 2008 0% 30yr 20yr 10yr Feb 01 2008 Mar 03 2008 Apr 01 2008 May 01 2008 Jun 02 2008 Jul 01 2008 Aug 01 2008 Sep 02 2008 Oct 01 2008 Nov 03 2008 Dec 01 2008 Dec 31 2008 1yr 6mo 3mo 1mo 2yr 3yr 5yr 7yr . chartSeries3d Yield Curve 2008 −−− Daily Modeling.quantmod Data. . Modeling. Visualization.quantmod Data. etc. rotChart. reChart. chartSeries3d chartSeries functionality to 3d/persp style graphics automatic time axis annotation “interactive”. 0 2.25 [2008−01−02/2008−12−31] 1.0 Last 0.0 T[.0 2. "3mo"] 3.5 2. chartSeries + chartSeries3d T[.0 2.0 2.76 [2008−01−02/2008−12−31] 3.55 [2008−01−02/2008−12−31] 4.5 3.5 1.0 1.0 Jan 02 2008 Apr 01 2008 Jun 02 Aug 01 Oct 01 2008 2008 2008 Dec 31 2008 Jan 02 2008 Apr 01 2008 Jun 02 Aug 01 Oct 01 2008 2008 2008 Dec 31 2008 Jan 02 2008 Apr 01 2008 Jun 02 Aug 01 Oct 01 2008 2008 2008 Dec 31 2008 1.0 0.0 Jan 02 2008 Apr 01 2008 Jun 02 Aug 01 Oct 01 2008 2008 2008 Dec 31 2008 Yield Curve 2008 −−− Daily 5% 4% 3% 2% 1% Jan 02 2008 Feb 01 2008 0% 30yr Mar 03 2008 20yr Apr 01 2008 10yr May 01 2008 7yr Jun 02 2008 5yr Jul 01 2008 3yr Aug 01 2008 2yr Sep 02 2008 1yr Oct 01 2008 6mo Nov 03 2008 Dec 01 2008 3mo Dec 31 2008 1mo . Visualization.0 T[.11 [2008−01−02/2008−12−31] 3.5 2. Modeling.5 0. "5yr"] Last 1.0 3.quantmod Data.5 T[.5 1.5 2. "2yr"] Last 0. "10yr"] Last 2.5 3.5 2. 0 2.0 3.5 2.5 2.0 Jan 02 2008 Apr 01 2008 Jun 02 Aug 01 Oct 01 2008 2008 2008 Dec 31 2008 Jan 02 2008 Apr 01 2008 Jun 02 Aug 01 Oct 01 2008 2008 2008 Dec 31 2008 Jan 02 2008 Apr 01 2008 Jun 02 Aug 01 Oct 01 2008 2008 2008 Dec 31 2008 Jun 02 Aug 01 Oct 01 2008 2008 2008 Dec 31 2008 Yield Curve 2008 −−− Daily options (risk and volatility) limit order book data Jan 02 2008 Feb 01 2008 Mar 03 2008 Apr 01 2008 10yr May 01 2008 7yr Jun 02 2008 5yr Jul 01 2008 3yr Aug 01 2008 2yr Sep 02 2008 1yr Oct 01 2008 6mo Nov 03 2008 Dec 01 2008 3mo Dec 31 2008 1mo 5% 4% 3% 2% 1% 0% 30yr 20yr . "2yr"] Last 0.5 3.5 1. T[.0 Jan 02 2008 Apr 01 2008 Last 0.5 2. "3mo"] 3.5 T[. Visualization.5 1. "5yr"] Last 1.76 [2008−01−02/2008−12−31] 3.0 2.5 2.0 2.25 [2008−01−02/2008−12−31] 1.0 0. "10yr"] Last 2.0 1.0 T[. Modeling.quantmod Data.5 3.0 T[.55 [2008−01−02/2008−12−31] 4.5 0.11 [2008−01−02/2008−12−31] 3.0 2.0 1. quantmod attachSymbols New functionality to extend upon getSymbols . quantmod attachSymbols New functionality to extend upon getSymbols Create a demand based database system using getSymbols that allows for implicit loading of an entire universe of symbols . GlobalEnv" "package:quantmod" "package:Defaults" [4] "package:xts" "package:zoo" "package:stats" [7] "package:graphics" "package:grDevices" "package:utils" [10] "package:datasets" "package:methods" "Autoloads" [13] "package:base" .quantmod attachSymbols Example: All US Equity symbols on demand. > search() [1] ". pos=2.") Contains symbols and method . prefix="E.quantmod attachSymbols Example: All US Equity symbols on demand.GlobalEnv" "package:quantmod" "package:Defaults" [4] "package:xts" "package:zoo" "package:stats" [7] "package:graphics" "package:grDevices" "package:utils" [10] "package:datasets" "package:methods" "Autoloads" [13] "package:base" > attachSymbols(DB=DDB_Yahoo(). > search() [1] ". quantmod attachSymbols Example: All US Equity symbols on demand.") > search() [1] ". prefix="E.GlobalEnv" "package:quantmod" "package:Defaults" [4] "package:xts" "package:zoo" "package:stats" [7] "package:graphics" "package:grDevices" "package:utils" [10] "package:datasets" "package:methods" "Autoloads" [13] "package:base" > attachSymbols(DB=DDB_Yahoo(). > search() [1] ". pos=2.GlobalEnv" "DDB:Yahoo" [4] "package:Defaults" "package:xts" [7] "package:stats" "package:graphics" [10] "package:utils" "package:datasets" [13] "Autoloads" "package:base" "package:quantmod" "package:zoo" "package:grDevices" "package:methods" . AA" "E.GlobalEnv" "DDB:Yahoo" [4] "package:Defaults" "package:xts" [7] "package:stats" "package:graphics" [10] "package:utils" "package:datasets" [13] "Autoloads" "package:base" "package:quantmod" "package:zoo" "package:grDevices" "package:methods" > str(ls("DDB:Yahoo")) chr [1:7406] "E..GlobalEnv" "package:quantmod" "package:Defaults" [4] "package:xts" "package:zoo" "package:stats" [7] "package:graphics" "package:grDevices" "package:utils" [10] "package:datasets" "package:methods" "Autoloads" [13] "package:base" > attachSymbols(DB=DDB_Yahoo().") > search() [1] ".AAII" .AAC" "E. . prefix="E.AACC" "E..AAI" "E.quantmod attachSymbols Example: All US Equity symbols on demand.A" "E. > search() [1] ". pos=2. quantmod attachSymbols Example: All US Equity symbols on demand.. > search() [1] "..AAII" .GlobalEnv" "DDB:Yahoo" [4] "package:Defaults" "package:xts" [7] "package:stats" "package:graphics" [10] "package:utils" "package:datasets" [13] "Autoloads" "package:base" "package:quantmod" "package:zoo" "package:grDevices" "package:methods" > str(ls("DDB:Yahoo")) chr [1:7406] "E.AAI" "E.AA" "E.AAC" "E. 7406 symbols are availablepos=2.AACC" "E.GlobalEnv" "package:quantmod" "package:Defaults" [4] "package:xts" "package:zoo" "package:stats" [7] "package:graphics" "package:grDevices" "package:utils" [10] "package:datasets" "package:methods" "Autoloads" [13] "package:base" > attachSymbols(DB=DDB_Yahoo().A" "E.") > search() [1] ". . prefix="E. "dimnames")=List of 2 .Open" "A.High" "A. 1:6] 35 34...3 34.Low" "A.Close" .. > str(E.POSIXct] TZ: America/ Chicago xts Attributes: List of 2 $ src : chr "yahoo" $ updated: POSIXct[1:1].3 34 34.quantmod attachSymbols Example: All US Equity symbols on demand.1 .$ : chr [1:6] "A. format: "2009-03-24 10:59:14" .. .A) An 'xts' object from 2007-01-03 to 2009-03-23 containing: Data: num [1:559..$ : NULL .attr(*. Indexed by objects of class: [POSIXt.. Close" . 1:6] 35 34. > str(E.attr(*..Low" "A.$ : chr [1:6] "A..Open" "A.POSIXct] TZ: America/ Chicago xts Attributes: List of 2 $ src : chr "yahoo" $ updated: POSIXct[1:1].High" "A. format: "2009-03-24 10:59:14" First access loads data .. .$ : NULL .quantmod attachSymbols Example: All US Equity symbols on demand..A) An 'xts' object from 2007-01-03 to 2009-03-23 containing: Data: num [1:559..3 34 34..1 . Indexed by objects of class: [POSIXt. "dimnames")=List of 2 .3 34. 032 0.quantmod attachSymbols Example: All US Equity symbols on demand. > system.AKAM) user system elapsed 0.267 download from Yahoo .time(E.004 0. 004 0.AKAM) user system elapsed 0.time(E.AKAM) user system elapsed 0 0 0 subsequent calls from cache . > system.032 0.267 > system.quantmod attachSymbols Example: All US Equity symbols on demand.time(E. objects are cached to disk. makeActiveBinding Memory after first access objects remain in memory delayedAssign .quantmod attachSymbols Two Methods to Cache Disk after first access. quantmod attachSymbols Custom DDB methods DDB object Binding Function Attach symbols . quantmod attachSymbols Custom DDB methods example: DDB:Yahoo > DDB_Yahoo() > # creates DDB object of all US Equity Symbols . attr(*.. "class")= chr "DDB" . ..quantmod attachSymbols Custom DDB methods example: DDB:Yahoo > DDB_Yahoo() > # creates DDB object of all US Equity Symbols > str(quantmod:::DDB_Yahoo()) List of 3 $ name: chr "DDB:Yahoo" $ src : chr "yahoo" $ db : chr [1:7358] "AACC" "AAME" "AANB" "AAON" . quantmod attachSymbols Custom DDB methods example: DDB:Yahoo > attachSymbols() > # “binds” symbols to functions to load/reload . . ..quantmod attachSymbols Custom DDB methods example: DDB:Yahoo > attachSymbols() > # “binds” symbols to functions to load/reload A few details. binding .quantmod attachSymbols attachSymbols attachSymbols.yahoo create. prefix = NULL. "DDB")) stop("DB must be of class 'DDB'") do.cache. file. DB$src.yahoo create.").quantmod attachSymbols attachSymbols attachSymbols.cache = mem.dir = cache. file. mem. pos = 2. cache.cache = FALSE.call(paste("attachSymbols". prefix = prefix. mem. pos = pos. postfix = NULL.binding > attachSymbols function (DB = DDB_Yahoo().cache = TRUE.dir = tempdir()) { if (!inherits(DB. postfix = postfix.cache. list(DB = DB.cache = file.dir)) } <environment: namespace:quantmod> . cache. sep = ". postfix = postfix.dir = tempdir()) { if (!inherits(DB. prefix = prefix. postfix = NULL.cache = file. cache. sep = ". mem.yahoo create. "DDB")) stop("DB must be of class 'DDB'") do. prefix = NULL. list(DB = DB. mem. DB$src.call(paste("attachSymbols".binding > attachSymbols function (DB = DDB_Yahoo().cache.cache. file.quantmod attachSymbols attachSymbols attachSymbols. cache.cache = FALSE.dir)) } <environment: namespace:quantmod> .cache = mem.").cache = TRUE.dir = cache. pos = 2. pos = pos. file. .binding > quantmod:::attachSymbols. create. postfix.function(x) paste(prefix. lsym = lsym.quantmod attachSymbols attachSymbols attachSymbols.character(x). name = DB$name) rsym <. pos = pos.. x.cache. mem. envir = DB$name)) } <environment: namespace:quantmod> .binding.cache = file.yahoo function (DB.dir. sep = "") invisible(sapply(DB$db. "-".) { attach(NULL.cache = mem. as. file. file. postfix.yahoo create. cache. cache.dir.cache.cache.function(x) gsub("_". rsym = rsym. . mem. pos. prefix.dir = cache.cache. perl = TRUE) lsym <. ..environment(envir) delayedAssign(lsym(s). makeActiveBinding(lsym(s). env = envir) get(lsym(s). { assign(lsym(s).quantmod attachSymbols attachSymbols attachSymbols.cache = FALSE.dir = tempdir(). rsym.binding function (s. file.cache) { envir <.environment(envir)) } if (mem.cache) { .. getSymbols(rsym(s).. if(file.assign = FALSE).env = envir) } } <environment: namespace:quantmod> . mem.. as. lsym. cache.cache = TRUE. assign..as. env = envir) }. envir) { . f.binding > quantmod:::create. auto.yahoo create. . cache..yahoo create. auto. f.environment(envir)) } if (mem.environment(envir) delayedAssign(lsym(s).as. rsym.binding function (s. getSymbols(rsym(s).assign = FALSE). if(file..dir = tempdir(). { assign(lsym(s). makeActiveBinding(lsym(s).cache) { . mem.. as.cache = FALSE.env = envir) } } <environment: namespace:quantmod> ..cache) { envir <. lsym.cache = TRUE.binding > quantmod:::create. assign. env = envir) }.quantmod attachSymbols attachSymbols attachSymbols. envir) { . . env = envir) get(lsym(s).. file.. in one session Simple mechanisms to create and manage .quantmod attachSymbols Custom DDB uses Auto-loading data based on source Multiple sources in unique environments .leverages getSymbols infrastructure attachSymbols(DTN_DDB()) attachSymbols(Bloomberg_bonds_DDB()) attachSymbols(OPRA_DDB()) . quantmod Future Work Integration of trading/testing with blotter & PerformanceAnalytics package More data methods. easier to extend specifyModel .tradeModel work .buildModel . [email protected] www.More Information www.quantmod.quantmod.com Presented by Jeffrey A.com/RFinance2009 . Ryan jeffrey.insightalgo.com www.