# Electricity flow trace input data

This directory contains the vSPD output data required for input to the tracing
routine.  The output is based on a future scenario using historic 2011-2013
market data.

The zip files provided by Concept Consulting were;  
```
  -rw-r--r-- 1   450M Jun  5 14:38 DH_branch.zip
  -rw-r--r-- 1   138M Jun  5 14:38 DH_node.zip
```
These contain all branch flow information and load/generation data that we
require for implementing the electricity flow trace. 

After unzipping, and to manage memory requirements, this data is split
into multiple monthly files.  For future reference, this was achieved 
using the \*nix stream editor called gawk.

For annuals you can do this:
```
gawk '{ print > "b_" substr($0,0,4) ".csv" }' DH_branch.csv
gawk '{ print > "n_" substr($0,0,4) ".csv" }' DH_node.csv
```
This results in six annual files of the form, b\_YYYY.csv and n\_YYYY.csv

For monthlys, as used in this analysis, we do this.
```
gawk '{ print > "b_" substr($0,0,4) substr($0,6,2) ".csv" }' DH_branch.csv
```
and end up with 36 files of the form; b\_YYYYMM.csv and b\_YYYYMM.csv

Monthlys have been zipped into the following files, for input into the trace.
```
-rw-rw-r-- 1   600M Jun  9 13:46 DH\_branch_monthly.zip
-rw-rw-r-- 1   162M Jun  9 13:44 DH\_node_monthly.zip
```
The n\_YYYYMM.csv files contain the branch flow information and look like
this (note these files do not contain the header below): 

```
head b_201101.csv
'datetime' , 'tp' , 'branch'    , 'from_mw' , 'dynamicloss (mw)' , 'fixedloss (mw)' , 'from_id_bus' , 'to_id_bus'
2011-01-01 , 1.0  , ABY\_T2.T2  , 4.63      , 0.04               , 0.02             , 545           , 546
2011-01-01 , 1.0  , ADD\_ISL1.1 , 18.01     , 0.03               , 0                , 526           , 542
2011-01-01 , 1.0  , ADD\_ISL1.2 , -14.88    , 0.01               , 0                , 519           , 542
2011-01-01 , 1.0  , ADD\_ISL1.3 , -3.09     , 0                  , 0                , 544           , 542
```
The n\_YYYYMM.csv files contain the generation and load/demand data and look like
this:
```
head n_201101.csv
'datetime' , 'tp' , 'node'  , 'bus' , 'allofact' , 'GENERATION' , 'LOAD' , 'bidMW'
2011-01-01 , 1.0  , ABY0111 , 545   , 1          , 0            , -4.641 , 0
2011-01-01 , 1.0  , ABY1101 , 546   , 1          , 0            , 0      , 0
2011-01-01 , 1.0  , ADD0111 , 518   , 1          , 0            , 16.477 , 0
2011-01-01 , 1.0  , ADD0661 , 519   , 1          , 0            , 46.782 , 0
2011-01-01 , 1.0  , ALB0331 , 20    , 1          , 0            , 43.434 , 0
```

DJH (updated 9/6/2015)