《利用python做数据分析》第十章：时间序列分析

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

//anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
warnings.warn(‘Matplotlib is building the font cache using fc-list. This may take a moment.’)

from pandas import Series,DataFrame

#### Time Seiries Analysis
****
> build-in package
time datetime calendar

from datetime import datetime

now = datetime.now()

now

datetime.datetime(2016, 2, 1, 11, 11, 8, 934671)

> ** display time right now **

now.year,now.month,now.day

(2016, 2, 1)

datetime以毫秒形势存储��和⌚️，**datetime.datedelta**表示两个datetime对象之间的时间差

delta = datetime(2011,1,7) - datetime(2008,6,24,8,15)

显示的前一个是天数，后一个是秒钟
—-
delta.days
delta.seconds

delta

datetime.timedelta(926, 56700)

### 可以给datetime对象加上或者减去一个或者多个timedelta，会产生一个新对象

from datetime import timedelta

start = datetime(2011, 1, 7)

start + timedelta(12)

datetime.datetime(2011, 1, 19, 0, 0)

start - timedelta(12) * 4

datetime.datetime(2010, 11, 20, 0, 0)

> 可见timedelta是以天为单位

#### datetime模块中的数据类型
—–
- date | 以公历形式存储日历日期（年、月、日）
- time | 将时间存储为时、分、秒、毫秒
- datetime | 存储时间和日期
- timedelta| 比阿诗两个datetime值之间的差（日， 秒， 毫秒）

## str transformed to datetime
use ** str ** or ** strftime(invoke a formed str) ** ,datetime object and pandas.Timestamp can be formulated to string

stamp = datetime(2011, 1, 3)

str(stamp)

‘2011-01-03 00:00:00’

stamp.strftime('%Y-%m-%d')

‘2011-01-03’

stamp.strftime('%Y-%m')

‘2011-01’

value = '2011-01-03'

datetime.strptime(value, '%Y-%m-%d')

datetime.datetime(2011, 1, 3, 0, 0)

datestrs = ['7/6/2011','8/6/2011']

[datetime.strptime(x, '%m/%d/%Y') for x in datestrs]

[datetime.datetime(2011, 7, 6, 0, 0), datetime.datetime(2011, 8, 6, 0, 0)]

datetime.striptime 是通过已知格式进行日期解析的最佳方式，但每次都要编写格式定义
- 使用dateutil中的parser.parse来实现

from dateutil.parser import parse

parse('2011-01-03')

datetime.datetime(2011, 1, 3, 0, 0)

parse的解析能力很强，几乎可以解析一切格式

parse('Jan 31,1997 10:45 PM')

datetime.datetime(1997, 1, 31, 22, 45)

parse('6/30/2011', dayfirst=True)

datetime.datetime(2011, 6, 30, 0, 0)

datestrs

[‘7/6/2011’, ‘8/6/2011’]

# pd.to_datetime()

pd.to_datetime(datestrs)

DatetimeIndex([‘2011-07-06’, ‘2011-08-06’], dtype=’datetime64[ns]’, freq=None)

dates = [datetime(2011, 1, 2),datetime(2011,1,5),datetime(2011,1,7),
datetime(2011,1,8),datetime(2011,1,10),datetime(2011,1,12)]

ts = Series(np.random.randn(6), index=dates)

ts

2011-01-02 0.573974
2011-01-05 -0.337112
2011-01-07 -1.650845
2011-01-08 0.450012
2011-01-10 -1.253801
2011-01-12 -0.402997
dtype: float64

type(ts)

pandas.core.series.Series

ts.index

DatetimeIndex([‘2011-01-02’, ‘2011-01-05’, ‘2011-01-07’, ‘2011-01-08’,
‘2011-01-10’, ‘2011-01-12’],
dtype=’datetime64[ns]’, freq=None)

ts + ts[::2]

2011-01-02 1.147949
2011-01-05 NaN
2011-01-07 -3.301690
2011-01-08 NaN
2011-01-10 -2.507602
2011-01-12 NaN
dtype: float64

ts[::2]

2011-01-02 0.573974
2011-01-07 -1.650845
2011-01-10 -1.253801
dtype: float64

## 索引、选取、子集构造

ts['1/10/2011']

-1.2538008746706757

传入可以解释为日期的字符，就可以代替索引

ts['20110110']

-1.2538008746706757

longer_ts=Series(np.random.randn(1000),index=pd.date_range('20000101',periods=1000))

longer_ts

2000-01-01 -1.025498
2000-01-02 -0.913267
2000-01-03 0.240895
2000-01-04 -1.475368
2000-01-05 -1.675558
2000-01-06 1.020005
2000-01-07 0.638097
2000-01-08 0.503482
2000-01-09 -0.541771
2000-01-10 -1.107036
2000-01-11 0.797612
2000-01-12 1.691745
2000-01-13 1.889323
2000-01-14 -0.852126
2000-01-15 -0.987578
2000-01-16 0.558084
2000-01-17 -0.842907
2000-01-18 1.932399
2000-01-19 -1.126650
2000-01-20 -0.529707
2000-01-21 0.116756
2000-01-22 -0.012790
2000-01-23 0.501330
2000-01-24 0.346976
2000-01-25 -0.880443
2000-01-26 -0.229017
2000-01-27 0.926648
2000-01-28 0.894491
2000-01-29 -0.573260
2000-01-30 -1.712945
…
2002-08-28 -0.751376
2002-08-29 -1.731035
2002-08-30 -0.150107
2002-08-31 -0.621332
2002-09-01 0.449311

14596
2002-09-02 0.873422
2002-09-03 1.496143
2002-09-04 -0.581023
2002-09-05 2.882920
2002-09-06 -0.347482
2002-09-07 0.165490
2002-09-08 -0.475642
2002-09-09 0.191958
2002-09-10 0.801963
2002-09-11 -1.603021
2002-09-12 1.114401
2002-09-13 0.994800
2002-09-14 -0.974208
2002-09-15 2.096747
2002-09-16 -0.252620
2002-09-17 -0.279536
2002-09-18 -0.059076
2002-09-19 -0.497615
2002-09-20 -0.009895
2002-09-21 1.813504
2002-09-22 0.863885
2002-09-23 1.330777
2002-09-24 -0.394473
2002-09-25 -1.163973
2002-09-26 -0.986664
Freq: D, dtype: float64

longer_ts['2002']

2002-01-01 -1.249172
2002-01-02 -1.368829
2002-01-03 0.097135
2002-01-04 -0.972259
2002-01-05 -0.640629
2002-01-06 0.619072
2002-01-07 1.625769
2002-01-08 -0.893140
2002-01-09 0.113725
2002-01-10 0.446898
2002-01-11 -0.382041
2002-01-12 -1.667311
2002-01-13 -0.307464
2002-01-14 0.623383
2002-01-15 -0.211188
2002-01-16 -1.166355
2002-01-17 0.399710
2002-01-18 -0.171451
2002-01-19 -1.591578
2002-01-20 -0.367654
2002-01-21 0.985778
2002-01-22 0.125848
2002-01-23 1.366708
2002-01-24 0.449383
2002-01-25 0.211848
2002-01-26 -1.033201
2002-01-27 0.668416
2002-01-28 0.402693
2002-01-29 -0.730690
2002-01-30 1.666659
…
2002-08-28 -0.751376
2002-08-29 -1.731035
2002-08-30 -0.150107
2002-08-31 -0.621332
2002-09-01 0.449311
2002-09-02 0.873422
2002-09-03 1.496143
2002-09-04 -0.581023
2002-09-05 2.882920
2002-09-06 -0.347482
2002-09-07 0.165490
2002-09-08 -0.475642
2002-09-09 0.191958
2002-09-10 0.801963
2002-09-11 -1.603021
2002-09-12 1.114401
2002-09-13 0.994800
2002-09-14 -0.974208
2002-09-15 2.096747
2002-09-16 -0.252620
2002-09-17 -0.279536
2002-09-18 -0.059076
2002-09-19 -0.497615
2002-09-20 -0.009895
2002-09-21 1.813504
2002-09-22 0.863885
2002-09-23 1.330777
2002-09-24 -0.394473
2002-09-25 -1.163973
2002-09-26 -0.986664
Freq: D, dtype: float64

longer_ts['2001/03']

2001-03-01 -0.130463
2001-03-02 -1.245341
2001-03-03 1.035173
2001-03-04 1.115275
2001-03-05 0.013602
2001-03-06 0.828075
2001-03-07 -0.802564
2001-03-08 2.067711
2001-03-09 2.158392
2001-03-10 1.348256
2001-03-11 1.282607
2001-03-12 -1.088485
2001-03-13 -0.882978
2001-03-14 -0.030872
2001-03-15 0.840561
2001-03-16 -0.061428
2001-03-17 0.170721
2001-03-18 0.895892
2001-03-19 -0.050714
2001-03-20 0.608656
2001-03-21 1.222177
2001-03-22 0.889833
2001-03-23 -0.932351
2001-03-24 0.163275
2001-03-25 0.001171
2001-03-26 0.969950
2001-03-27 -0.118747
2001-03-28 -0.840478
2001-03-29 -2.654215
2001-03-30 -0.351836
2001-03-31 -0.365322
Freq: D, dtype: float64

ts['20110101':'20110201']

2011-01-02 0.573974
2011-01-05 -0.337112
2011-01-07 -1.650845
2011-01-08 0.450012
2011-01-10 -1.253801
2011-01-12 -0.402997
dtype: float64

ts.truncate(after='20110109')

2011-01-02 0.573974
2011-01-05 -0.337112
2011-01-07 -1.650845
2011-01-08 0.450012
dtype: float64

dates = pd.date_range('20000101', periods=100, freq='W-WED')

dates

DatetimeIndex([‘2000-01-05’, ‘2000-01-12’, ‘2000-01-19’, ‘2000-01-26’,
‘2000-02-02’, ‘2000-02-09’, ‘2000-02-16’, ‘2000-02-23’,
‘2000-03-01’, ‘2000-03-08’, ‘2000-03-15’, ‘2000-03-22’,
‘2000-03-29’, ‘2000-04-05’, ‘2000-04-12’, ‘2000-04-19’,
‘2000-04-26’, ‘2000-05-03’, ‘2000-05-10’, ‘2000-05-17’,
‘2000-05-24’, ‘2000-05-31’, ‘2000-06-07’, ‘2000-06-14’,
‘2000-06-21’, ‘2000-06-28’, ‘2000-07-05’, ‘2000-07-12’,
‘2000-07-19’, ‘2000-07-26’, ‘2000-08-02’, ‘2000-08-09’,
‘2000-08-16’, ‘2000-08-23’, ‘2000-08-30’, ‘2000-09-06’,
‘2000-09-13’, ‘2000-09-20’, ‘2000-09-27’, ‘2000-10-04’,
‘2000-10-11’, ‘2000-10-18’, ‘2000-10-25’, ‘2000-11-01’,
‘2000-11-08’, ‘2000-11-15’, ‘2000-11-22’, ‘2000-11-29’,
‘2000-12-06’, ‘2000-12-13’, ‘2000-12-20’, ‘2000-12-27’,
‘2001-01-03’, ‘2001-01-10’, ‘2001-01-17’, ‘2001-01-24’,
‘2001-01-31’, ‘2001-02-07’, ‘2001-02-14’, ‘2001-02-21’,
‘2001-02-28’, ‘2001-03-07’, ‘2001-03-14’, ‘2001-03-21’,
‘2001-03-28’, ‘2001-04-04’, ‘2001-04-11’, ‘2001-04-18’,
‘2001-04-25’, ‘2001-05-02’, ‘2001-05-09’, ‘2001-05-16’,
‘2001-05-23’, ‘2001-05-30’, ‘2001-06-06’, ‘2001-06-13’,
‘2001-06-20’, ‘2001-06-27’, ‘2001-07-04’, ‘2001-07-11’,
‘2001-07-18’, ‘2001-07-25’, ‘2001-08-01’, ‘2001-08-08’,
‘2001-08-15’, ‘2001-08-22’, ‘2001-08-29’, ‘2001-09-05’,
‘2001-09-12’, ‘2001-09-19’, ‘2001-09-26’, ‘2001-10-03’,
‘2001-10-10’, ‘2001-10-17’, ‘2001-10-24’, ‘2001-10-31’,
‘2001-11-07’, ‘2001-11-14’, ‘2001-11-21’, ‘2001-11-28’],
dtype=’datetime64[ns]’, freq=’W-WED’)

long_df = DataFrame(np.random.randn(100,4),index=dates,columns=['Colorado','Texas','New York','Ohio'])

long_df.ix['5-2001']

	Colorado	Texas	New York	Ohio
2001-05-02	1.783070	1.090816	-1.035363	-0.089864
2001-05-09	-1.290700	1.311863	-0.596037	0.819694
2001-05-16	0.688693	-0.249644	-0.859212	0.879270
2001-05-23	-1.602660	1.211236	-1.028336	2.022514
2001-05-30	-0.705427	-0.189235	-0.710712	-2.397815

dates = pd.DatetimeIndex(['1/1/2000','1/2/2000',
'1/2/2000','1/2/2000',
'1/3/2000'])

dup_ts = Series(np.arange(5), index=dates)

dup_ts

2000-01-01 0
2000-01-02 1
2000-01-02 2
2000-01-02 3
2000-01-03 4
dtype: int64

通过检查索引的** is_unique ** 属性，判断是不是唯一

dup_ts.index.is_unique

False

对这个时间序列进行索引，要么产生标量值，要么产生切片，具体要看所选的
> **时间点是否重复**

none repeat(2000-1-3)

dup_ts['1/3/2000']

4

repeat (2000-1-2)

dup_ts['1/2/2000']

2000-01-02 1
2000-01-02 2
2000-01-02 3
dtype: int64

define whether it is reaptable or not

dup_ts.index.is_unique

False

# 对具有非唯一时间戳的数据聚合 #

> groupby(level=0)

level=0意味着索引唯一一层！！！

—-

grouped = dup_ts.groupby(level=0)

grouped.mean(),grouped.count()

(2000-01-01 0
2000-01-02 2
2000-01-03 4
dtype: int64, 2000-01-01 1
2000-01-02 3
2000-01-03 1
dtype: int64)

> 将时间序列转换成 **具有固定频率（每日）的时间序列**
- resample

ts.resample('D')

2011-01-02 0.573974
2011-01-03 NaN
2011-01-04 NaN
2011-01-05 -0.337112
2011-01-06 NaN
2011-01-07 -1.650845
2011-01-08 0.450012
2011-01-09 NaN
2011-01-10 -1.253801
2011-01-11 NaN
2011-01-12 -0.402997
Freq: D, dtype: float64

生成日期范围
- pandas.date_range
- 类型：DatetimeIndex

index = pd.date_range('4/1/2012','6/1/2012')

## base frequency
- 基础频率通常以一个字符串表示，M每月，H每小时
- 对于每个基础频率都有一个偏移量与之对应
- date offset

from pandas.tseries.offsets import Hour, Minute

hour = Hour()

hour

> 传入一个整数即可定义偏移量的倍数：

four_hours = Hour(4)

four_hours

pd.date_range('1/1/2000','1/3/2000 23:59',freq='4h')

DatetimeIndex([‘2000-01-01 00:00:00’, ‘2000-01-01 04:00:00’,
‘2000-01-01 08:00:00’, ‘2000-01-01 12:00:00’,
‘2000-01-01 16:00:00’, ‘2000-01-01 20:00:00’,
‘2000-01-02 00:00:00’, ‘2000-01-02 04:00:00’,
‘2000-01-02 08:00:00’, ‘2000-01-02 12:00:00’,
‘2000-01-02 16:00:00’, ‘2000-01-02 20:00:00’,
‘2000-01-03 00:00:00’, ‘2000-01-03 04:00:00’,
‘2000-01-03 08:00:00’, ‘2000-01-03 12:00:00’,
‘2000-01-03 16:00:00’, ‘2000-01-03 20:00:00’],
dtype=’datetime64[ns]’, freq=’4H’)

偏移量可以通过加法链接

Hour(2) + Minute(30)

pd.date_range('1/1/2000', periods=10, freq='1h30min')

DatetimeIndex([‘2000-01-01 00:00:00’, ‘2000-01-01 01:30:00’,
‘2000-01-01 03:00:00’, ‘2000-01-01 04:30:00’,
‘2000-01-01 06:00:00’, ‘2000-01-01 07:30:00’,
‘2000-01-01 09:00:00’, ‘2000-01-01 10:30:00’,
‘2000-01-01 12:00:00’, ‘2000-01-01 13:30:00’],
dtype=’datetime64[ns]’, freq=’90T’)

### WOM(week of month)

rng = pd.date_range('1/1/2012','9/1/2012',freq='WOM-3FRI')

pd.date_range('1/1/2012','9/1/2012',freq='W-FRI')

DatetimeIndex([‘2012-01-06’, ‘2012-01-13’, ‘2012-01-20’, ‘2012-01-27’,
‘2012-02-03’, ‘2012-02-10’, ‘2012-02-17’, ‘2012-02-24’,
‘2012-03-02’, ‘2012-03-09’, ‘2012-03-16’, ‘2012-03-23’,
‘2012-03-30’, ‘2012-04-06’, ‘2012-04-13’, ‘2012-04-20’,
‘2012-04-27’, ‘2012-05-04’, ‘2012-05-11’, ‘2012-05-18’,
‘2012-05-25’, ‘2012-06-01’, ‘2012-06-08’, ‘2012-06-15’,
‘2012-06-22’, ‘2012-06-29’, ‘2012-07-06’, ‘2012-07-13’,
‘2012-07-20’, ‘2012-07-27’, ‘2012-08-03’, ‘2012-08-10’,
‘2012-08-17’, ‘2012-08-24’, ‘2012-08-31’],
dtype=’datetime64[ns]’, freq=’W-FRI’)

> 时间表别名10-4 P314

### 移动（超前和滞后）数据
- 移动（shifting）指的是沿着时间轴将数据迁移或者后移
- Series & Dataframe都有一个shift方法单纯执行前移后移
- 保持索引不变

ts = Series(np.random.randn(4),index=pd.date_range('1/1/2000',periods=4,freq='M'))

ts

2000-01-31 -0.550830
2000-02-29 -1.297499
2000-03-31 1.178102
2000-04-30 1.359573
Freq: M, dtype: float64

ts.shift(-2)

2000-01-31 1.178102
2000-02-29 1.359573
2000-03-31 NaN
2000-04-30 NaN
Freq: M, dtype: float64

shift ususally used to calculate the pct change of a series

ts / ts.shift(1) - 1

2000-01-31 NaN
2000-02-29 1.355534
2000-03-31 -1.907979
2000-04-30 0.154037
Freq: M, dtype: float64

ts.pct_change()

2000-01-31 NaN
2000-02-29 1.355534
2000-03-31 -1.907979
2000-04-30 0.154037
Freq: M, dtype: float64

ts.shift(2, freq='M')

2000-03-31 -0.550830
2000-04-30 -1.297499
2000-05-31 1.178102
2000-06-30 1.359573
Freq: M, dtype: float64

ts.shift(3, freq='D')

2000-02-03 -0.550830
2000-03-03 -1.297499
2000-04-03 1.178102
2000-05-03 1.359573
dtype: float64

type(ts)

pandas.core.series.Series

ts.shift()

2000-01-31 NaN
2000-02-29 -0.550830
2000-03-31 -1.297499
2000-04-30 1.178102
Freq: M, dtype: float64

ts.shift(3)

2000-01-31 NaN
2000-02-29 NaN
2000-03-31 NaN
2000-04-30 -0.55083
Freq: M, dtype: float64

ts.shift(freq='D')

2000-02-01 -0.550830
2000-03-01 -1.297499
2000-04-01 1.178102
2000-05-01 1.359573
Freq: MS, dtype: float64

ts.shift(periods=2)

2000-01-31 NaN
2000-02-29 NaN
2000-03-31 -0.550830
2000-04-30 -1.297499
Freq: M, dtype: float64

freq means move the index by the frequence

from pandas.tseries.offsets import Day, MonthEnd

如果增加的是⚓️点偏移量（比如MonthEnd)，第一次增量会讲原来的日期向前滚动到适合规则的下一个日期
- 今天11月17号，MonthEnd就是这个月末11.31

now = datetime(2011, 11, 17)

now + 3*Day()

Timestamp(‘2011-11-20 00:00:00’)

now + MonthEnd()

Timestamp(‘2011-11-30 00:00:00’)

now + MonthEnd(2)

Timestamp(‘2011-12-31 00:00:00’)

offset = MonthEnd()

offset.rollforward(now)

Timestamp(‘2011-11-30 00:00:00’)

offset.rollback(now)

Timestamp(‘2011-10-31 00:00:00’)

巧妙的使用**groupby**和**⚓️点偏移量**

ts = Series(np.random.randn(20), index=pd.date_range('1/15/2000',periods=20,freq='4d'))

ts.groupby(offset.rollforward).mean()

2000-01-31 -0.223943
2000-02-29 -0.241283
2000-03-31 -0.080391
dtype: float64

更方便快捷的方法应该是用
> resample

ts.resample('M', how='mean')

2000-01-31 -0.223943
2000-02-29 -0.241283
2000-03-31 -0.080391
Freq: M, dtype: float64

# import pytz
—-
pytz是一个世界时区的库，时区名

import pytz

pytz.common_timezones[-5:]

[‘US/Eastern’, ‘US/Hawaii’, ‘US/Mountain’, ‘US/Pacific’, ‘UTC’]

tz = pytz.timezone('US/Eastern')

tz

### 本地化和转换

rng = pd.date_range('3/9/2012 9:30',periods=6, freq='D')
ts = Series(np.random.randn(len(rng)),index=rng)

del index

ts.index.tz

add a time zone set of the ts
- make it print

pd.date_range('3/9/2000 9:30',periods=10, freq='D',tz='UTC')

DatetimeIndex([‘2000-03-09 09:30:00+00:00’, ‘2000-03-10 09:30:00+00:00’,
‘2000-03-11 09:30:00+00:00’, ‘2000-03-12 09:30:00+00:00’,
‘2000-03-13 09:30:00+00:00’, ‘2000-03-14 09:30:00+00:00’,
‘2000-03-15 09:30:00+00:00’, ‘2000-03-16 09:30:00+00:00’,
‘2000-03-17 09:30:00+00:00’, ‘2000-03-18 09:30:00+00:00’],
dtype=’datetime64[ns, UTC]’, freq=’D’)

> The +00:00 means
- time zone

use *tz_localize* to localize the time zone

ts_utc = ts.tz_localize('UTC')

ts_utc

2012-03-09 09:30:00+00:00 -0.258702
2012-03-10 09:30:00+00:00 -1.019056
2012-03-11 09:30:00+00:00 1.044139
2012-03-12 09:30:00+00:00 0.826684
2012-03-13 09:30:00+00:00 0.998759
2012-03-14 09:30:00+00:00 -0.839695
Freq: D, dtype: float64

just have a try of crtl+v

ts_utc.index

DatetimeIndex([‘2012-03-09 09:30:00+00:00’, ‘2012-03-10 09:30:00+00:00’,
‘2012-03-11 09:30:00+00:00’, ‘2012-03-12 09:30:00+00:00’,
‘2012-03-13 09:30:00+00:00’, ‘2012-03-14 09:30:00+00:00’],
dtype=’datetime64[ns, UTC]’, freq=’D’)

convert localized time zone to another one use:
> *tz_convert*

ts_utc.tz_convert('US/Eastern')

2012-03-09 04:30:00-05:00 -0.258702
2012-03-10 04:30:00-05:00 -1.019056
2012-03-11 05:30:00-04:00 1.044139
2012-03-12 05:30:00-04:00 0.826684
2012-03-13 05:30:00-04:00 0.998759
2012-03-14 05:30:00-04:00 -0.839695
Freq: D, dtype: float64

*tz_localize* & *tz_convert* are also instance methods on *DatetimeIndex*

ts.index.tz_localize('Asia/Shanghai')

DatetimeIndex([‘2012-03-09 09:30:00+08:00’, ‘2012-03-10 09:30:00+08:00’,
‘2012-03-11 09:30:00+08:00’, ‘2012-03-12 09:30:00+08:00’,
‘2012-03-13 09:30:00+08:00’, ‘2012-03-14 09:30:00+08:00’],
dtype=’datetime64[ns, Asia/Shanghai]’, freq=’D’)

# operations with Time Zone
- awrae Timestamp Objects

Localized from naive to time zone-aware and converted from one time zone to another

stamp = pd.Timestamp('2011-03-12 4:00')
stamp_utc = stamp.tz_localize('utc')

stamp_utc.tz_convert('US/Eastern')

Timestamp(‘2011-03-11 23:00:00-0500’, tz=’US/Eastern’)

>Time zone-aware Timestamp objects internally store a UTC timestamp calue as nano-seconed since thr UNIX epoch(January 1,1970)
- this UTC value is invariant between time zone conversions

stamp_utc.value

1299902400000000000

stamp = pd.Timestamp('2012-03-12 01:30', tz='US/Eastern')

stamp

Timestamp(‘2012-03-12 01:30:00-0400’, tz=’US/Eastern’)

stamp + Hour()

Timestamp(‘2012-03-12 02:30:00-0400’, tz=’US/Eastern’)

# operations between different time zones

rng = pd.date_range('3/7/2012 9:30',periods=10, freq='B')

ts = Series(np.random.randn(len(rng)), index=rng)

ts

2012-03-07 09:30:00 0.315600
2012-03-08 09:30:00 0.616440
2012-03-09 09:30:00 -1.633940
2012-03-12 09:30:00 0.260501
2012-03-13 09:30:00 -0.394620
2012-03-14 09:30:00 -0.554103
2012-03-15 09:30:00 2.441851
2012-03-16 09:30:00 -3.473308
2012-03-19 09:30:00 -0.339365
2012-03-20 09:30:00 0.335510
Freq: B, dtype: float64

ts1 = ts[:7].tz_localize('Europe/London')

ts2 = ts1[2:].tz_convert('Europe/Moscow')

result = ts1 + ts2

>different time zone can be added up together freely

result.index

DatetimeIndex([‘2012-03-07 09:30:00+00:00’, ‘2012-03-08 09:30:00+00:00’,
‘2012-03-09 09:30:00+00:00’, ‘2012-03-12 09:30:00+00:00’,
‘2012-03-13 09:30:00+00:00’, ‘2012-03-14 09:30:00+00:00’,
‘2012-03-15 09:30:00+00:00’],
dtype=’datetime64[ns, UTC]’, freq=’B’)

## Periods and Periods Arithmetic

> Periods
- time spans
- days, months,quarters,years

p = pd.Period(2007, freq='A-DEC')

p

Period(‘2007’, ‘A-DEC’)

## Time Series Plotting

close_px_call = pd.read_csv('/Users/Houbowei/Desktop/SRP/books/pydata-book-master/pydata-book-master/ch09/stock_px.csv', parse_dates=True,index_col=0)

close_px = close_px_call[['AAPL','MSFT','XOM']]

close_px = close_px.resample('B',fill_method='ffill')

close_px

	AAPL	MSFT	XOM
2003-01-02	7.40	21.11	29.22
2003-01-03	7.45	21.14	29.24
2003-01-06	7.45	21.52	29.96
2003-01-07	7.43	21.93	28.95
2003-01-08	7.28	21.31	28.83
2003-01-09	7.34	21.93	29.44
2003-01-10	7.36	21.97	29.03
2003-01-13	7.32	22.16	28.91
2003-01-14	7.30	22.39	29.17
2003-01-15	7.22	22.11	28.77
2003-01-16	7.31	21.75	28.90
2003-01-17	7.05	20.22	28.60
2003-01-20	7.05	20.22	28.60
2003-01-21	7.01	20.17	27.94
2003-01-22	6.94	20.04	27.58
2003-01-23	7.09	20.54	27.52
2003-01-24	6.90	19.59	26.93
2003-01-27	7.07	19.32	26.21
2003-01-28	7.29	19.18	26.90
2003-01-29	7.47	19.61	27.88
2003-01-30	7.16	18.95	27.37
2003-01-31	7.18	18.65	28.13
2003-02-03	7.33	19.08	28.52
2003-02-04	7.30	18.59	28.52
2003-02-05	7.22	18.45	28.11
2003-02-06	7.22	18.63	27.87
2003-02-07	7.07	18.30	27.66
2003-02-10	7.18	18.62	27.87
2003-02-11	7.18	18.25	27.67
2003-02-12	7.20	18.25	27.12
…	…	…	…
2011-09-05	374.05	25.80	72.14
2011-09-06	379.74	25.51	71.15
2011-09-07	383.93	26.00	73.65
2011-09-08	384.14	26.22	72.82
2011-09-09	377.48	25.74	71.01
2011-09-12	379.94	25.89	71.84
2011-09-13	384.62	26.04	71.65
2011-09-14	389.30	26.50	72.64
2011-09-15	392.96	26.99	74.01
2011-09-16	400.50	27.12	74.55
2011-09-19	411.63	27.21	73.70
2011-09-20	413.45	26.98	74.01
2011-09-21	412.14	25.99	71.97
2011-09-22	401.82	25.06	69.24
2011-09-23	404.30	25.06	69.31
2011-09-26	403.17	25.44	71.72
2011-09-27	399.26	25.67	72.91
2011-09-28	397.01	25.58	72.07
2011-09-29	390.57	25.45	73.88
2011-09-30	381.32	24.89	72.63
2011-10-03	374.60	24.53	71.15
2011-10-04	372.50	25.34	72.83
2011-10-05	378.25	25.89	73.95
2011-10-06	377.37	26.34	73.89
2011-10-07	369.80	26.25	73.56
2011-10-10	388.81	26.94	76.28
2011-10-11	400.29	27.00	76.27
2011-10-12	402.19	26.96	77.16
2011-10-13	408.43	27.18	76.37
2011-10-14	422.00	27.27	78.11

2292 rows × 3 columns

close_px.resample?

close_px['AAPL'].plot()

close_px.ix['2009'].plot()

close_px['AAPL'].ix['01-2011':'03-2011'].plot()

apple_q = close_px['AAPL'].resample('Q-DEC', fill_method='ffill')

apple_q.ix['2009':].plot()

close_px.AAPL.plot()

close_px.plot()

apple_std250 = pd.rolling_std(close_px.AAPL, 250)

apple_std250.describe()

count 2043.000000
mean 20.604571
std 12.606813
min 1.335707
25% 9.121461
50% 22.231490
75% 32.411445
max 39.327273
Name: AAPL, dtype: float64

apple_std250.plot()

close_px.describe()

	AAPL	MSFT	XOM
count	2292.000000	2292.000000	2292.000000
mean	125.339895	23.953010	59.568473
std	107.218553	3.267322	16.731836
min	6.560000	14.330000	26.210000
25%	37.122500	21.690000	49.517500
50%	91.365000	24.000000	62.980000
75%	185.535000	26.280000	72.540000
max	422.000000	34.070000	87.480000

close_px_call.describe()

	AAPL	MSFT	XOM	SPX
count	2214.000000	2214.000000	2214.000000	2214.000000
mean	125.516147	23.945452	59.558744	1183.773311
std	107.394693	3.255198	16.725025	180.983466
min	6.560000	14.330000	26.210000	676.530000
25%	37.135000	21.700000	49.492500	1077.060000
50%	91.455000	24.000000	62.970000	1189.260000
75%	185.605000	26.280000	72.510000	1306.057500
max	422.000000	34.070000	87.480000	1565.150000

spx = close_px_call.SPX.pct_change()

spx

2003-01-02         NaN
2003-01-03   -0.000484
2003-01-06    0.022474
2003-01-07   -0.006545
2003-01-08   -0.014086
2003-01-09    0.019386
2003-01-10    0.000000
2003-01-13   -0.001412
2003-01-14    0.005830
2003-01-15   -0.014426
2003-01-16   -0.003942
2003-01-17   -0.014017
2003-01-21   -0.015702
2003-01-22   -0.010432
2003-01-23    0.010224
2003-01-24   -0.029233
2003-01-27   -0.016160
2003-01-28    0.013050
2003-01-29    0.006779
2003-01-30   -0.022849
2003-01-31    0.013130
2003-02-03    0.005399
2003-02-04   -0.014088
2003-02-05   -0.005435
2003-02-06   -0.006449
2003-02-07   -0.010094
2003-02-10    0.007569
2003-02-11   -0.008098
2003-02-12   -0.012687
2003-02-13   -0.001600
...
2011-09-02   -0.025282
2011-09-06   -0.007436
2011-09-07    0.028646
2011-09-08   -0.010612
2011-09-09   -0.026705
2011-09-12    0.006966
2011-09-13    0.009120
2011-09-14    0.013480
2011-09-15    0.017187
2011-09-16    0.005707
2011-09-19   -0.009803
2011-09-20   -0.001661
2011-09-21   -0.029390
2011-09-22   -0.031883
2011-09-23    0.006082
2011-09-26    0.023336
2011-09-27    0.010688
2011-09-28   -0.020691
2011-09-29    0.008114
2011-09-30   -0.024974
2011-10-03   -0.028451
2011-10-04    0.022488
2011-10-05    0.017866
2011-10-06    0.018304
2011-10-07   -0.008163
2011-10-10    0.034125
2011-10-11    0.000544
2011-10-12    0.009795
2011-10-13   -0.002974
2011-10-14    0.017380
Name: SPX, dtype: float64

returns = close_px.pct_change()

corr = pd.rolling_corr(returns.AAPL, spx, 125 , min_periods=100)

corr.plot()

<matplotlib.axes._subplots.AxesSubplot at 0x10bf49450>

corr = pd.rolling_corr(returns, spx, 125, min_periods=100).plot()