"FinalPrices" data...
   The source CD (LH6113HJ3102 3476 DO Label COMIT Prices) bore the following files:

 D:\Dispatch Prices\1998 Dispatch Prices - All nodes.zip
 D:\Dispatch Prices\1999 Dispatch Prices - All nodes.zip
 D:\Dispatch Prices\2000 Dispatch Prices - All nodes.zip
 D:\Dispatch Prices\2001 Dispatch Prices - All nodes.zip
 D:\Dispatch Prices\2002 Dispatch Prices - All nodes.zip
 D:\Dispatch Prices\2003 Dispatch Prices - All nodes.zip
 D:\Dispatch Prices\2004 Dispatch Prices - All nodes 01Jan-30 Apr.zip
 D:\Final Prices - All Nodes\1996 Final Prices - All Nodes.zip
 D:\Final Prices - All Nodes\1997 Final Prices - All Nodes.zip
 D:\Final Prices - All Nodes\1998 Final Prices - All Nodes.zip
 D:\Final Prices - All Nodes\1999 Final Prices - All Nodes.zip
 D:\Final Prices - All Nodes\2000 Final Prices - All Nodes.zip
 D:\Final Prices - All Nodes\2001 Final Prices - All Nodes.zip
 D:\Final Prices - All Nodes\2002 Final Prices - All nodes.zip
 D:\Final Prices - All Nodes\2003 Final Prices - All Nodes.zip
 D:\Final Prices - All Nodes\2004 Final Prices - All nodes - Jan-Apr.zip
 D:\Final Reserve Prices\1996 Final Reserve Prices - All Nodes.zip
 D:\Final Reserve Prices\1997 Final Reserve Prices - All Nodes.zip
 D:\Final Reserve Prices\1998 Final Reserve Prices - All Nodes.zip
 D:\Final Reserve Prices\1999 Final Reserve Prices - All Nodes.zip
 D:\Final Reserve Prices\2000 Final Reserve Prices - All Nodes.zip
 D:\Final Reserve Prices\2001 Final Reserve Prices - All Nodes.zip
 D:\Final Reserve Prices\2002 Final Reserve Prices - All Nodes.zip
 D:\Final Reserve Prices\2003 Final Reserve Prices - All Nodes.zip
 D:\Final Reserve Prices\2004 Final Reserve Prices - All Nodes Jan-Apr.zip
 D:\Forecast Prices\Forecast Prices 2000 - All nodes 03Jul-31Dec.zip
 D:\Forecast Prices\Forecast Prices 2001 - All nodes.zip
 D:\Forecast Prices\Forecast Prices 2002 - All nodes.zip
 D:\Forecast Prices\Forecast Prices 2003 - All nodes.zip
 D:\Forecast Prices\Forecast Prices 2004 - All nodes 01Jan-30Apr.zip
 D:\Forecast Prices\Reserve Forecast Prices 2000 - All nodes 03Jul-31Dec.zip
 D:\Forecast Prices\Reserve Forecast Prices 2001 - All nodes.zip
 D:\Forecast Prices\Reserve Forecast Prices 2002 - All nodes.zip
 D:\Forecast Prices\Reserve Forecast Prices 2003 - All nodes.zip
 D:\Forecast Prices\Reserve Forecast Prices 2004 - All nodes 01Jan-30Apr.zip

   The initial plan is to deal with the second collection, the D:\Final Prices - All Nodes\ group, which generated:

C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Apr_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Aug_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Dec_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Feb_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Jan_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Jul_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Jun_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Mar_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_May_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Nov_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Oct_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Apr2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Aug2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Dec2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Feb2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Jan2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Jul2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Jun2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Mar2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_May2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Nov2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Oct2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Sep2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Sep_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200201.csv.Z
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200202.csv.Z
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200203.csv.Z
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200204.csv.Z
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200205.csv.Z
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200206.csv.Z
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200207.csv.Z
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200208.csv.Z
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200209.csv.Z
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200210.csv.Z
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200211.csv.Z
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200212.csv.Z
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1996 Final Prices\Final_1996_10.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1996 Final Prices\Final_1996_11.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1996 Final Prices\Final_1996_12.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1997 Final Prices\Final_1997_01.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1997 Final Prices\Final_1997_02.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1997 Final Prices\Final_1997_03.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1997 Final Prices\Final_1997_04.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1997 Final Prices\Final_1997_05.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1997 Final Prices\Final_1997_06.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1997 Final Prices\Final_1997_07.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1997 Final Prices\Final_1997_08.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1997 Final Prices\Final_1997_09.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1997 Final Prices\Final_1997_10.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1997 Final Prices\Final_1997_11.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1997 Final Prices\Final_1997_12.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1998 Final Prices\Final_1998_01.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1998 Final Prices\Final_1998_02.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1998 Final Prices\Final_1998_03.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1998 Final Prices\Final_1998_04.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1998 Final Prices\Final_1998_05.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1998 Final Prices\Final_1998_06.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1998 Final Prices\Final_1998_07.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1998 Final Prices\Final_1998_08.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1998 Final Prices\Final_1998_09.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1998 Final Prices\Final_1998_10.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1998 Final Prices\Final_1998_11.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1998 Final Prices\Final_1998_12.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1999 Final Prices\Final_1999_01.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1999 Final Prices\Final_1999_02.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1999 Final Prices\Final_1999_03.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1999 Final Prices\Final_1999_04.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1999 Final Prices\Final_1999_05.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1999 Final Prices\Final_1999_06.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1999 Final Prices\Final_1999_07.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1999 Final Prices\Final_1999_08.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1999 Final Prices\Final_1999_09.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1999 Final Prices\Final_1999_10.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1999 Final Prices\Final_1999_11.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\1999 Final Prices\Final_1999_12.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\2000 Final Prices\Final_2000_01.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\2000 Final Prices\Final_2000_02.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\2000 Final Prices\Final_2000_03.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\2000 Final Prices\Final_2000_04.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\2000 Final Prices\Final_2000_05.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\2000 Final Prices\Final_2000_06.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\2000 Final Prices\Final_2000_07.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\2000 Final Prices\Final_2000_08.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\2000 Final Prices\Final_2000_09.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\2000 Final Prices\Final_2000_10.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\2000 Final Prices\Final_2000_11.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\2000 Final Prices\Final_2000_12.csv

   On dealing with the second-order file compression, and moving the content of the sub-directories
into the main, the horde became:

C:\Nicky\HH\FPCD\Final Prices - All Nodes\2004 Final Prices - All nodes - Apr.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final prices Feb 2004.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final prices Jan 2004.txt
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final prices Mar 2004.TXT
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1996_10.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1996_11.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1996_12.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1997_01.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1997_02.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1997_03.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1997_04.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1997_05.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1997_06.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1997_07.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1997_08.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1997_09.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1997_10.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1997_11.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1997_12.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1998_01.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1998_02.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1998_03.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1998_04.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1998_05.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1998_06.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1998_07.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1998_08.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1998_09.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1998_10.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1998_11.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1998_12.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1999_01.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1999_02.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1999_03.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1999_04.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1999_05.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1999_06.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1999_07.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1999_08.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1999_09.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1999_10.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1999_11.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_1999_12.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_2000_01.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_2000_02.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_2000_03.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_2000_04.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_2000_05.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_2000_06.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_2000_07.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_2000_08.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_2000_09.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_2000_10.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_2000_11.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_2000_12.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Apr_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Aug_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Dec_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Feb_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Jan_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Jul_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Jun_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Mar_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_May_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Nov_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Oct_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Apr2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Aug2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Dec2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Feb2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Jan2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Jul2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Jun2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Mar2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_May2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Nov2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Oct2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Prices_Sep2003.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\Final_Sep_2001.csv
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200201.csv.txt
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200202.csv.txt
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200203.csv.txt
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200204.csv.txt
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200205.csv.txt
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200206.csv.txt
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200207.csv.txt
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200208.csv.txt
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200209.csv.txt
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200210.csv.txt
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200211.csv.txt
C:\Nicky\HH\FPCD\Final Prices - All Nodes\fp200212.csv.txt

   The first annoyance was easily dealt with. Some file names contained spaces, and as this
causes trouble with the scanning of commands the file names were changed slightly rather
than escalate to the annoyances of quoting such file names, especially when such quotes
are not supplied by the likes of the "DIR" command. This re-naming also caused the files
to appear closer to their associates in a sorted list of names.
                Old                                         New
     2004 Final Prices - All nodes - Apr.csv   2004_Final_Prices-AllNodes-Apr.csv
     Final prices Feb 2004.csv                 Final_prices_Feb2004.txt
     Final prices Jan 2004.txt                 Final_prices_Jan2004.txt
     Final prices Mar 2004.TXT                 Final_prices_Mar2004.txt

   Thus the Feb, Jan, and Mar 2004 files joined the other files with month names.
This of course means that they're not in chronological order by month, and of course,
with the year last, they're not in chronological order by year either, by contrast to
other files such as Final_2000_06 et al. The obvious is only obvious to some, obviously.
   The first files inspected contained data such as
       ABY0111,01/10/1996,1,55.14,,F
       ABY0111,01/10/1996,2,43.95,,F
       ABY0111,01/10/1996,3,42.71,,F
   So processing proceeded on that basis. The suffix of ",,F" was unexplained: it was
declared a "tail" and any twitches were monitored. Supposedly the "F" signified the
"finality" of the "final price". Then suddenly, as an attempt was made on the whole
collection, file Final_Apr_2001.csv was found to present fresh ingenuity.
Observe the start:
       ABY0111,1/04/2001,1,F,59.34,Y,2/04/2001 15:10:31
       ABY0111,1/04/2001,2,F,59.2,Y,2/04/2001 15:10:31
       ABY0111,1/04/2001,3,F,58.81,Y,2/04/2001 15:10:31
   The fourth field, after the HH number in the third field, is no longer the HH value
but a code, followed by a floating-point number, then a different style of tail-end bumphf.
So... if the fourth field does not contain a number, skip it and proceed as before with
the fifth field. And in passing, note the date format miscegnation: the month number
has a leading zero, but the day number does not. In this file.
   The fifth field holds the value in files for 2001, 2003 and 2004 (January - March)
but the other files (eg for 2002) have it in the fourth field.

   The next fun development was to discover the "improper dates" part-way through certain files,
such as Final_Prices_Apr2003.csv whose dates were of the form 4/1/2003 then 4/2/2003, etc.
Clearly, the order for these files was Month/Day/Year, rather than Day/Month/Year as used
elsewhere. At first, it seemed possible to deduce the order by relying on the first date
in the file being for the first day of the month, but this won't work for January since
a day/month of "1/1" is no different from a month/day of "1/1". Observing the transition
from the first date to the second might work, but enough: a command "SwapDM" was added to
Gnash, that swaps the day and month parts of any dates it spots in a line and so revised
files were produced, with a name change while I was at it as well:
                  Input                   Output.
      SwapDM Final_Prices_Jan2003.csv  fp2003m01.csv
      SwapDM Final_Prices_Feb2003.csv  fp2003m02.csv
      SwapDM Final_Prices_Mar2003.csv  fp2003m03.csv
      SwapDM Final_Prices_Apr2003.csv  fp2003m04.csv
      SwapDM Final_Prices_May2003.csv  fp2003m05.csv
      SwapDM Final_Prices_Jun2003.csv  fp2003m06.csv
      SwapDM Final_Prices_Jul2003.csv  fp2003m07.csv
      SwapDM Final_Prices_Aug2003.csv  fp2003m08.csv
      SwapDM Final_Prices_Sep2003.csv  fp2003m09.csv
      SwapDM Final_Prices_Oct2003.csv  fp2003m10.csv
      SwapDM Final_Prices_Nov2003.csv  fp2003m11.csv
      SwapDM Final_Prices_Dec2003.csv  y2003m12.csv

   Similarly, the other files with the standard dates were also renamed to produce
a consistent sequence, as follows:

      Rename 2004_Final_Prices-AllNodes-Apr.csv  fp2004m04.csv
      Rename Final_1996_10.csv                   fp1996m10.csv
      Rename Final_1996_11.csv                   fp1996m11.csv
      Rename Final_1996_12.csv                   fp1996m12.csv
      Rename Final_1997_01.csv                   fp1997m01.csv
      Rename Final_1997_02.csv                   fp1997m02.csv
      Rename Final_1997_03.csv                   fp1997m03.csv
      Rename Final_1997_04.csv                   fp1997m04.csv
      Rename Final_1997_05.csv                   fp1997m05.csv
      Rename Final_1997_06.csv                   fp1997m06.csv
      Rename Final_1997_07.csv                   fp1997m07.csv
      Rename Final_1997_08.csv                   fp1997m08.csv
      Rename Final_1997_09.csv                   fp1997m09.csv
      Rename Final_1997_10.csv                   fp1997m10.csv
      Rename Final_1997_11.csv                   fp1997m11.csv
      Rename Final_1997_12.csv                   fp1997m12.csv
      Rename Final_1998_01.csv                   fp1998m01.csv
      Rename Final_1998_02.csv                   fp1998m02.csv
      Rename Final_1998_03.csv                   fp1998m03.csv
      Rename Final_1998_04.csv                   fp1998m04.csv
      Rename Final_1998_05.csv                   fp1998m05.csv
      Rename Final_1998_06.csv                   fp1998m06.csv
      Rename Final_1998_07.csv                   fp1998m07.csv
      Rename Final_1998_08.csv                   fp1998m08.csv
      Rename Final_1998_09.csv                   fp1998m09.csv
      Rename Final_1998_10.csv                   fp1998m10.csv
      Rename Final_1998_11.csv                   fp1998m11.csv
      Rename Final_1998_12.csv                   fp1998m12.csv
      Rename Final_1999_01.csv                   fp1999m01.csv
      Rename Final_1999_02.csv                   fp1999m02.csv
      Rename Final_1999_03.csv                   fp1999m03.csv
      Rename Final_1999_04.csv                   fp1999m04.csv
      Rename Final_1999_05.csv                   fp1999m05.csv
      Rename Final_1999_06.csv                   fp1999m06.csv
      Rename Final_1999_07.csv                   fp1999m07.csv
      Rename Final_1999_08.csv                   fp1999m08.csv
      Rename Final_1999_09.csv                   fp1999m09.csv
      Rename Final_1999_10.csv                   fp1999m10.csv
      Rename Final_1999_11.csv                   fp1999m11.csv
      Rename Final_1999_12.csv                   fp1999m12.csv
      Rename Final_2000_01.csv                   fp2000m01.csv
      Rename Final_2000_02.csv                   fp2000m02.csv
      Rename Final_2000_03.csv                   fp2000m03.csv
      Rename Final_2000_04.csv                   fp2000m04.csv
      Rename Final_2000_05.csv                   fp2000m05.csv
      Rename Final_2000_06.csv                   fp2000m06.csv
      Rename Final_2000_07.csv                   fp2000m07.csv
      Rename Final_2000_08.csv                   fp2000m08.csv
      Rename Final_2000_09.csv                   fp2000m09.csv
      Rename Final_2000_10.csv                   fp2000m10.csv
      Rename Final_2000_11.csv                   fp2000m11.csv
      Rename Final_2000_12.csv                   fp2000m12.csv
      Rename Final_Apr_2001.csv                  fp2001m04.csv
      Rename Final_Aug_2001.csv                  fp2001m08.csv
      Rename Final_Dec_2001.csv                  fp2001m12.csv
      Rename Final_Feb_2001.csv                  fp2001m02.csv
      Rename Final_Jan_2001.csv                  fp2001m01.csv
      Rename Final_Jul_2001.csv                  fp2001m07.csv
      Rename Final_Jun_2001.csv                  fp2001m06.csv
      Rename Final_Mar_2001.csv                  fp2001m03.csv
      Rename Final_May_2001.csv                  fp2001m05.csv
      Rename Final_Nov_2001.csv                  fp2001m11.csv
      Rename Final_Oct_2001.csv                  fp2001m10.csv
      Rename Final_prices_Feb2004.txt            fp2004m02.csv
      Rename Final_prices_Jan2004.txt            fp2004m01.csv
      Rename Final_prices_Mar2004.txt            fp2004m03.csv
      Rename Final_Sep_2001.csv                  fp2001m09.csv
      Rename fp200201.csv                        fp2002m01.csv
      Rename fp200202.csv                        fp2002m02.csv
      Rename fp200203.csv                        fp2002m03.csv
      Rename fp200204.csv                        fp2002m04.csv
      Rename fp200205.csv                        fp2002m05.csv
      Rename fp200206.csv                        fp2002m06.csv
      Rename fp200207.csv                        fp2002m07.csv
      Rename fp200208.csv                        fp2002m08.csv
      Rename fp200209.csv                        fp2002m09.csv
      Rename fp200210.csv                        fp2002m10.csv
      Rename fp200211.csv                        fp2002m11.csv
      Rename fp200212.csv                        fp2002m12.csv

   These names are boring, but adjacency is upheld. Further, they now appear in chronological order.
   However, the data within the files is not itself chronological. In file fp2001m04.csv there
appeared data for LFD1102 on 9/4/2001, this being its first appearance, but then there followed
data for 6/4/2001 for LFD1102, provoking Gnash to complain about shifting the base day back.
Since data for 5/4/2001 was followed by 9/4/2001, then 6/4/2001 and there was no data for 9/4/2001
between that for 8/4/2001 and 10/4/2001, the data for 9/4/2001 was shifted there.

   And still there is obstructionism!
   File fp2004m01.csv starts with
      PRICE_GIP_GXP_FULL,PRICE_DATE,PRICE_PERIOD,PRICE_TYPE,PRICE,PRICE_RUN_TIME
      ABY0111,01/01/2004,1,F,31.2,02/01/2004 11:13:43
   File fp2004m02.csv starts with
      PRICE_GIP_GXP_FULL,PRICE_DATE,PRICE_PERIOD,PRICE_TYPE,PRICE,PRICE_RUN_TIME
      APS0111,01/02/2004,30,F,47.41,02/02/2004 07:46:29
   File fp2004m03.csv starts with
      PRICE_GIP_GXP_FULL,PRICE_DATE,PRICE_PERIOD,PRICE_TYPE,PRICE,PRICE_RUN_TIME
      ABY0111,01/03/2004,1,F,0.05,02/03/2004 08:36:35
   But file fp2004m04.csv starts as do the earlier files:
      ABY0111,01/04/2004,1,19.96,02/04/2004 12:01:45,F

   Describing the data is an excellent protocol, but it works better if it is done
every time! Otherwise it is just a damn nusiance. The simplest response is to uphold
conformity by conforming to the style of the majority of data files, which lack these
initial decorations, nor are they particularly explanatory. But it would be a pity
to reject a token attempt at being helpful, so, since Gnash happens to employ a %
symbol to signify a comment, in this case the annoyance was abated by prefixing
the first line with a % character. This causes the line to be ignored, and so it will
be the second line that is inspected in an attempt to identify the format.
   However,in later developments for other data, yet further file formats were presented,
forcing a more elaborate inspection of file formats so that headers became helpful,
so these headers were uncommented to be recognised in the more complex scheme.
   And notice that the HH value field is number five, not four, but this annoyance is
already accommodated thanks to the 2001 interlude of such data.

   Now consider the first few lines of file fp2001m09.csv (file Final_Sep_2001.csv as was)
      ABY0111	1/09/2001	1	F	83.35	Y	3/09/2001 11:28:04
      ABY0111	1/09/2001	2	F	79.4	Y	3/09/2001 11:28:04
      ABY0111	1/09/2001	3	F	79.46	Y	3/09/2001 11:28:04
   As can be seen, despite the .csv suffix, there are no commas separating the values.
They are instead in nice neat fixed columns (though aligned left is not so good),
and such data are particularly easy to read, and it would have been much easier if
all the other data were in this form, but they aren't.
   And the "aligned left" should have been a giveaway: instead of commas separating
the fields, there are tab characters. So here is the sample with spaces instead.
      ABY0111   1/09/2001       1       F       83.35   Y       3/09/2001 11:28:04
      ABY0111   1/09/2001       2       F       79.4    Y       3/09/2001 11:28:04
      ABY0111   1/09/2001       3       F       79.46   Y       3/09/2001 11:28:04
   Converting the 2,099,520 tabs to commas was the work of moments.
      ABY0111,1/09/2001,1,F,83.35,Y,3/09/2001 11:28:04
      ABY0111,1/09/2001,2,F,79.4,Y,3/09/2001 11:28:04
      ABY0111,1/09/2001,3,F,79.46,Y,3/09/2001 11:28:04
   Conformity is good with computers.

   And still the quality of ingenuity is not strained! File fp2003m12.csv exemplifies
indecision. One might hope that these data would be rolled forth in some coherent sequence,
but no. Since Gnash implements random-access for its storage, the days need not be
presented in order, similarly, the HH values for a day need not arrive in sequence.
   But this was not enough. Data for a supplier may not arrive in one piece, so now,
the programme holds storage for one day for each supplier so that values may arrive
either in the reasonable order of all values for each supplier, one supplier followed
by the next, or, in the less-reasonable order of hh1 for each supplier, then hh2 for
each supplier, etc. When a datum for a supplier arrives that has a date other than that
of the data in hand, then the held data is saved, and preparations are made for a new day
for that supplier. The hope is that by then, all of its values would have been encountered.
   This also was too much to hope for, a drab constraint that presented yet a further
opportunity for an exercise in ingenuity. It was surmounted with ease. Accordingly,
programme Gnash was extended to acknowledge missing data by reserving a special
value to signify "not a value", or "bad". Thus, a day about to be stored may contain
"bad" values (because no datum had been supplied) for particular half-hours
and likewise, there may be already in storage some data for that day. If so, the two are
compared: for each half hour, should the stored datum be good, an incoming "bad"
value does not replace it, conversely, should the previously-stored value be "bad"
then the incoming value (hopefully good) replaces it. Only the "ZAP" command can
replace a good value by a bad value, in a different process. Subsequently, a feature:
routine Swallow juggles good and bad values according to Input.BadValuesPrevail.
   Thus, a day can be constructed piecemeal, as the incoming data directs,
and Gnash now maintains a version count for each day, though not yet for each half-hour. (ha ha; see below)
   And with this apparatus in place, it turned out that some revisions effected
no change, others changed afresh what had been previously changed.

   The flexibility enabled by all the redundant information in each record allows
disorganisation, and it was not avoided but revelled in. File fp2003m12.csv escalated
to a further level: some data does not have a code of "F" but "V", however such
"V" entries are all later revised by "F"-associated values (at least in this file),
not that a "Final" value is immune from assault... The sequence annotated below
demonstrates these little conceits. For legibility, I have aligned the columns.
And notice in passing the appearance here of the AM/PM style indications rather
than the twenty-four hour clock used previously. As well, the dates were originally
supplied as month/day/year, but this is now only a trivial obstruction.

      Line  Supplier  Date     HH    Value    Concoction timestamp.
    400612: BDE0111,28/12/2003,48,V, 42.36,Y,29/12/2003 10:13:35 AM    HH48 for BDE0111; code V, not F.
        13: BEN0161,28/12/2003, 1,V,   0  ,Y,29/12/2003 10:13:35 AM   <Now for BEN0161; code V.
       660: BEN0161,28/12/2003,48,V,   0  ,Y,29/12/2003 10:13:35 AM   <Last for BEN0161 (all were zero)
        61: BEN0162,28/12/2003, 1,V, 44.6 ,Y,29/12/2003 10:13:35 AM    Start on BEN0162, 28'th.
    411268: WVY0111,28/12/2003,48,V, 45.13,Y,29/12/2003 10:13:35 AM    Last supplier for the 28'th.
        69: ABY0111,29/12/2003, 2,F, 45.59,Y,30/12/2003  7:48:33 AM    First supplier for the 29'th. No HH1. Code F
    422457: WVY0111,29/12/2003,48,F, 32.9 ,Y,30/12/2003  7:48:33 AM    Last of a block, none with HH1.
        58: ABY0111,27/12/2003, 1,F, 45.09,Y,30/12/2003  9:31:07 AM    Back to the 27'th, but now with F codes.
    423273: BEN0161,27/12/2003,48,F,   0  ,Y,30/12/2003  9:31:07 AM    BEN0161 48'th value for the 27'th
        74: BEN0161,28/12/2003, 1,F,100000,Y,30/12/2003  9:31:07 AM <<<Interloper! BEN0161, 28'th, single value! Code F, not V.
        75: BEN0162,27/12/2003, 1,F, 44   ,Y,30/12/2003  9:31:07 AM    Resume the 27'th, full sets of 48.
    433884: WVY0111,27/12/2003,48,F, 44.82,Y,30/12/2003  9:31:07 AM    Last for the 27'th.
        85: ABY0111,29/12/2003, 1,F, 46.22,Y,30/12/2003 10:13:46 AM    Start on the 29'th, but only HH1!
        86: ADD0111,29/12/2003, 1,F, 47.5 ,Y,30/12/2003 10:13:46 AM    Another name, but only HH1 for it,
        87: ADD0661,29/12/2003, 1,F, 47.43,Y,30/12/2003 10:13:46 AM    And only HH1 for this one too.
    434119: WVY0111,29/12/2003, 1,F, 46.2 ,Y,30/12/2003 10:13:46 AM    A full set of suppliers, for HH1.
        20: ABY0111,28/12/2003, 1,F, 45.77,Y,30/12/2003 10:13:46 AM    Single values for the 28'th
    434354: WVY0111,28/12/2003, 1,F, 45.74,Y,30/12/2003 10:13:46 AM    For all suppliers.
        55: ABY0111,28/12/2003, 2,F, 45.48,Y,30/12/2003 10:13:46 AM    Now values for HH2..
        56: ABY0111,28/12/2003, 3,F, 45.39,Y,30/12/2003 10:13:46 AM    HH3, ... HH48 for this supplier.
    435106: BDE0111,28/12/2003,48,F, 42.36,Y,30/12/2003 10:13:46 AM    Last of those with HH2 to HH48.
         7: BEN0161,28/12/2003, 1,F,   0  ,Y,30/12/2003 10:13:46 AM   <Third appearance of BEN0161 for the 28'th.
       154: BEN0161,28/12/2003,48,F,   0  ,Y,30/12/2003 10:13:46 AM   <Full set of 48, code F.
        55: BEN0161,29/12/2003, 1,F,100000,Y,30/12/2003 10:13:46 AM <<<Interloper!
        56: BEN0162,28/12/2003, 2,F, 44.6 ,Y,30/12/2003 10:13:46 AM    Resume the 28'th for BEN0162, with HH2 to HH48
    445546: WVY0111,28/12/2003,48,F, 45.13,Y,30/12/2003 10:13:46 AM    For all remaining suppliers.
        47: ABY0111,30/12/2003, 1,V, 41.92,Y,31/12/2003  7:59:04 AM    On to the 30'th... Code V again.
        48: ABY0111,30/12/2003, 2,V, 40.05,Y,31/12/2003  7:59:04 AM    ...etc

   In response to this, any line lacking a "F" past the start of the name field will be skipped,
since at least in the data files so far encountered, all such were later zapped by an "F"-coded value.
   The main point is that although a code "F" signifies a "Final" value, this may not be the case,
it may be revised later as in the above example sequence. If once a tiger is seen, is not the jungle
then full of tigers?
   For this reason, a further escalation: when data is being supplied in this format, for each datum
the value of the timestamp is maintained in an auxiliary file. If no timestamp is supplied then
that timestamp value is "BAD". This is distinct from the case when there has not been any previous
value for a time of a day for a run; there is no timestamp datum. Thus do we follow the fine
distinctions between different aspects of nothingness. So, not only does each day have a version number,
now each datum has a version number in the form of the timestamp, and appropriate remarks are made.
This checking doubles the number of runs since every single value has its own time stamp, and further,
timestamp values are stored in double precision.,
   Oh, and despite the name "Final" prices, and the notion of prices being specified before the transaction,
it appears that the price can be revised after the fact, thus the timestamp can follow the time
of the transaction: in other words, a check against this is of little use.
   And the upshot was: of the 32,000,000 or so values in the "Final Prices" collection so far,
just nine values were revised, and their timestamps, in the order as encountered in the data files,
were proper. If however the non-"F" data were admitted then there would be some 250,000 revisions.
   This is separate from the occasions when a day is revised by the arrival of a value for another
half-hour, the other values being retained. This is tiresomely frequent.

   Alternatively, data can be omitted. Let no modality of annoyance be overlooked!
File fp2004m04.csv for no apparent reason omits data for hh36 (only) for all 236 of its names for 24/4/2004.
Then arrived file final_Apr04-Feb05.csv which re-supplies data for April 2004, and all values for the 24'th
are present. More than present. For no apparent reason, names HWB0322 and ROB1101 present a value
for hh. 36 on the 24'th of April, unsupported by values for other times. They begin properly on the 29'th.
Aside from such entertainments, since it supplies all the data that file fp2004m04.csv delivers and more,
file fp2004m04.csv is moved to directory BadData and the new file becomes fp2004m04-2005m02.csv

   File FinalEnergyPrices_200503.csv, renamed to fp2005m03.csv does not have its datum in field 4,
but in field 5; this is not new, just the usual standard of uncontrolled variation.
   File FinalEnergyPrices_200504.csv, renamed to fp2005m04.csv likewise. This was from CD 6b; the first try
was rejected due to its deviant format: column re-order, appearance of $-signs.
   The revised file returned to the previous format (with the value in field four), except that the
previously-used header starting "PRICE_GIP_GXP_FULL," etc. had been found boring and was replaced by
a new and presumably better text "Grid_Exit_Point," etc.


              Do I dare
              Disturb the universe?
              In a minute there is time
              For decisions and revisions which a minute will reverse.

                        T.S. Eliot, The Love Song of J. Alfred Prufrock

   More grist:

   FinalEnergyPrices_200505.csv becomes fp2005m05.csv  from CD 7. Field 5 has the datum. So much for reverting to field 4.
   FinalEnergyPrices_200506.csv         fp2005m06.csv          8. Field 5 has the datum.
   FinalEnergyPrices_200507.csv         fp2005m07.csv          9. Field 5 has the datum.
   FinalEnergyPrices_200508.csv         fp2005m08.csv         10. Field 5 has the datum.
   FinalEnergyPrices_200509.csv         fp2005m09.csv         11. Field 5 has the datum.
   FinalEnergyPrices_200510.csv         fp2005m10.csv         12. Field 5 has the datum. RFN1101 has only 24 values for 21/10/2005, RFN1102 has 24 on 17/10/2005
   FinalEnergyPrices_200511.csv         fp2005m11.csv         13. Field 5 has the datum.
   Putting the year number first became too boring to bear, so...
   FinalEnergyPrices - 122005.csv       fp2005m12.csv         14. Field 5 has the datum.
   FinalEnergyPrices - 012006.csv       fp2006m01.csv         15. Field 5 has the datum.
   FinalEnergyPrices - 022006.csv       fp2006m02.csv         16. Field 5 has the datum.
   FinalEnergyPrices - 032006.csv       fp2006m03.csv         17. Field 5 has the datum. BPT1101 has only 24 values for 20/3/2006.

   While floundering through all this bumf, not much attention has been given to the resulting runs
of data. One puzzle had been the mysterious appearance of code names having just two letters rather
than three: this is even less helpful than a three-letter code and since neither are documented,
chasing this bunny did not look promising. There are twenty-nine such codes: AN2201, AT1101, CH0111, DN0141
DN0331, DN1101, DN2201, ER0331, ER1101, GM0331, HO0331, LG0111, LG0331, NG0331, NG1101, NI0111, OT0111,
PE0331, PI0331, RA0111, RR0111, ST0331, TI0111, TI2201, TM0111, TM0331, TN0331, TO0331, TR0331.
   Then, when Brian Bull started looking at these data, he found five-month holes that would be filled
nicely were each such name prefixed with the letter "M" and combined with the other data for that name.
This is exactly what the alias table is for, and happily, no name conflicts resulted...

   FinalEnergyPrices - 042006.csv       fp2006m04.csv         18. Field 5 has the datum.
   FinalEnergyPrices - 052006.csv       fp2006m05.csv         19. Field 5 has the datum.
   FinalEnergyPrices - 062006.csv       fp2006m06.csv         20. Field 5 has the datum.
   FinalEnergyPrices - 072006.csv       fp2006m07.csv         21. Field 5 has the datum. This file was unreadable on FourCrunch, but readable on Brian's computer.
   FinalEnergyPrices - 082006.csv       fp2006m08.csv         22. Field 5 has the datum.
   FinalEnergyPrices - 092006.csv       fp2006m09.csv         23. Field 5 has the datum.
   FinalEnergyPrices - 102006.csv       fp2006m10.csv         24b.Field 5 has the datum.
   FinalEnergyPrices - 112006.csv       fp2006m11.csv         25. Field 5 has the datum.
   FinalEnergyPrices - 122006.csv       fp2006m12.csv         26. Field 5 has the datum. Under directory EC - February 2007
   FinalEnergyPrices - 012007.csv       fp2007m01.csv         27. Field 5 has the datum. Under directory EC - March 2007. Introduces Waipara 66KV, so now it must be distinguished from Waipara 33KV. No sign of other data for Waipara 66KV as yet.
   FinalEnergyPrices - 022007.csv       fp2007m02.csv         28. Field 5 has the datum. No directory, no more?
   FinalEnergyPrices - 032007.csv       fp2007m03.csv         29. Field 5 has the datum. Nope. Directory  EC - May 2007.
   FinalEnergyPrices_200704.csv         fp2007m04.csv         30. Field 5 has the datum. ATU1101 has only 24 values for 28/4/2007.
   FinalEnergyPrices - 052007.csv       fp2007m05.csv         31. Field 5 has the datum. CST0111 has only 24 values for 30/5/2007.
   FinalEnergyPrices - 062007.csv       fp2007m06.csv         32. Field 5 has the datum.
   FinalEnergyPrices - 072007.csv       fp2007m07.csv         33. Field 5 has the datum. The CD wrongly annotated "EC - AUG 07": should be September
   FinalEnergyPrices - 082007.csv       fp2007m08.csv         34. Field 5 has the datum. Annotated EC - Sept 2007, so a revised convention?
   FinalEnergyPrices_092007.csv         fp2007m09.csv         35. Field 5 has the datum. File name change, again.
   FinalEnergyPrices - 102007.csv       fp2007m10.csv         36. Field 5 has the datum. And back, again.
   FinalEnergyPrices - 112007.csv       fp2007m11.csv         37. Field 5 has the datum. Staying with it for now.
   FinalEnergyPrices - 122007.csv       fp2007m12.csv         38. Field 5 has the datum.
   FinalEnergyPrices - 012008.csv       fp2008m01.csv         39. Field 5 has the datum.
   FinalEnergyPrices_200802.csv         fp2008m02.csv         40. Field 5 has the datum. Bored with previous file names.
   FinalEnergyPrices - 032008.csv       fp2008m03.csv         41. Field 5 has the datum. And back again.
   FinalEnergyPrices - 042008.csv       fp2008m04.csv         42. Field 5 has the datum. MLN0664 has only 24 values for 11/4/2008.
   FinalEnergyPrices - 052008.csv       fp2008m05.csv         43. Field 5 has the datum. Mysterious irruption of HWA0332, starting with noon 20/5/2008 and all zero so far.
   FinalEnergyPrices - 062008.csv       fp2008m06.csv         44. Field 5 has the datum. MDN0142 has only 24 values for 30/6/2008.
   FinalEnergyPrices - 072008.csv       fp2008m07.csv         45. Field 5 has the datum.
   FinalEnergyPrices - 082008.csv       fp2008m08.csv         46. Field 5 has the datum.
   FinalEnergyPrices - 092008.csv       fp2008m09.csv         47. Field 5 has the datum.
   FinalEnergyPrices - 102008.csv       fp2008m10.csv         48. Field 5 has the datum. Irruption of BEN2202, starting with noon 20/10/2008 so only 24 half-hours.
   FinalEnergyPrices_-_112008.csv       fp2008m11.csv         49. Field 5 has the datum. Irruption of WWD1102, WWD1103. Commencement at noon 18/11/2008. Previous use of WWD = Wairoa Watercare Dam is forgotten in favour of "West WinD", somewhere.
   Dec final prices.csv                 fp2008m12.csv         Deleted. Special arrival via e-mail (Rick Liew wanted the data for all of 2008, before he quit), after the first four-part attempt was rejected (deviant format, also, part two was a repetition of part one). The second attempt (employing one big file) lost the time part of the timestamps, but had the same values as the first attempt offered and did not duplicate the 1-7 batch in what was called the 8-14 batch. Field 5 had the datum, and the header reverted to an earlier form.
   FinalEnergyPrices - 122008.csv       fp2008m12.csv         50. Field 5 has the datum. This version preferred as this version has time information at the end of each line as well as the date. The e-mailed version also had a completely different order of presentation, but all data values were the same.
   finalPrices_-_012009.zip became finalPrices_-_012009.TXT   Deleted. Same values (though different order) as next arrived on CD51 as FinalEnergyPrices_-_012009.csv though calculation times (presumably) were given to the second and a.m./p.m. not just to the minute.
   FinalEnergyPrices_-_012009.csv       fp2009m01.csv         51. Field 5 has the datum.
   Final_Prices_-_022009.csv                                  Deleted. Same values (though different order) as next arrived on CD52 as FinalEnergyPrices_-_022009.csv though calculation times (presumably) were given to the second and a.m./p.m. not just to the minute.
   FinalEnergyPrices_-_022009.csv       fp2009m02.csv         52. Field 5 has the datum.
   Final_Prices_-_032009.csv                                  Deleted, as before. No a.m./p.m. this time.
   FinalEnergyPrices - 032009.csv       fp2009m03.csv         53. Field 5 has the datum.
   Final Prices - 042009.csv                                  Deleted, as before.
   FinalEnergyPrices - 042009.csv       fp2009m04.csv         54. Field 5 has the datum.
   Final Prices - 052009.csv                                  Deleted, as before.
   FinalEnergyPrices - 052009.csv       fp2009m05.csv         55. Field 5 has the datum.
   Final Prices - 062009.csv                                  Deleted, as before.
   FinalEnergyPrices - 062009.csv       fp2009m06.csv         56  Field 5 has the datum.
   Final_Prices_-_072009.csv                                  Deleted, as before. Introduces 203 new series... Assimilated, under the expectation of another repeat of the above.
   FinalEnergyPrices - 072009.csv       fp2009m07.csv         57  Field 5 has the datum.
   Final Prices - Aug 2009.csv                                Deleted, as before. This time, CD58's fp2009m08.csv is identical (times are given to seconds in both)
   FinalEnergyPrices - 082009.csv       fp2009m08.csv         58  Field 5 has the datum.
   Final Prices - 092009.csv                                  Deleted, as before. This time on CD 59 there is no file FinalEnergyPrices - 092009.csv, it is called Final Prices - 092009.csv
   Final Prices - 092009.csv            fp2009m09.csv         59  Field 5 has the datum.
   Final Prices - 102009.csv                                  Deleted, as before. CD60's file is identical (same timestamp).
   Final Prices - 102009.csv            fp2009m10.csv         60  Field 5 has the datum.
   Final Prices - 112009.csv                                  Deleted, as before. CD60's file is identical (same timestamp).
   Final Prices - 112009.csv            fp2009m11.csv         61  Field 5 has the datum. This now seems established.
   Final Prices - 122009.csv            fp2009m12.csv         Assimilated, on expectation of later replacement.
   FinalEnergyPrices - 122009.csv                             62  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 012010.csv            fp2010m01.csv         Assimilated, and perhaps it too will be preferred.
   FinalEnergyPrices - 012010.csv                             63  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 022010.csv            fp2010m02.csv         Assimilated, possibly to be superceded next month.
   FinalEnergyPrices - 022010.csv                             64  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 032010.csv            fp2010m03.csv         Assimilated. For 31/3/2010, only the first 28 values for Mount Maunganui 11KV.
   FinalEnergyPrices - 032010.csv                             65  Has a heading, and the same values (but in day order, not name then day), and still lacks data for Mount Maunganui 11KV on 31/3/2010, but deleted as its timestamps are truncated.
   Final Prices - 042010.csv            fp2010m04.csv         Assimilated. It seems that Mount Maunganui 11KV has ceased.
   FinalEnergyPrices - 042010.csv                             66  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 052010.csv            fp2010m05.csv         Assimilated.
   FinalEnergyPrices - 052010.csv                             67  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 062010.csv            fp2010m06.csv         Assimilated. For 2/6/2010, only a half day for SI.Invercargill.220KV.Code2.Price.
   FinalEnergyPrices - 062010.csv                             68  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 072010.csv            fp2010m07.csv         Assimilated.
   FinalEnergyPrices - 072010.csv                             69  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 082010.csv            fp2010m08.csv         Assimilated.
   FinalEnergyPrices - 082010.csv                             70  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 092010.csv            fp2010m09.csv         Assimilated.
   FinalEnergyPrices - 092010.csv                             71  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 102010.csv            fp2010m10.csv         Assimilated.
   FinalEnergyPrices - 102010.csv                             72  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 112010.csv            fp2010m11.csv         Assimilated.
   FinalEnergyPrices - 112010.csv                             73  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 122010.csv            fp2010m12.csv         Assimilated.
   FinalEnergyPrices - 122010.csv                             74  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 012011.csv            fp2011m01.csv         Assimilated.
   FinalEnergyPrices - 012011.csv                             75  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 022011.csv            fp2011m02.csv         Assimilated.
   FinalEnergyPrices - 022011.csv                             76  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 032011.csv            Not supplied. Suppressed as being "confusing".
   Final Prices - 032011.zip            fp2011m03hic.csv      Arrived by e-mail to Brian Kirtlan, with the data for the 26'th omitted. (Name order then date while the standard supply by CD is by date then name order)
   FinalEnergyPrices - 032011.csv                             77  Has a heading, and the same values and the same omission of the 26'th (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 042011.csv            fp2011m04.csv         77  Assimilated.
   FinalEnergyPrices - 042011.csv                             78  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 052011.csv            fp2011m05.csv         78  Assimilated.
   FinalEnergyPrices - 052011.csv                             79  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 062011.csv            fp2011m06.csv         79  Assimilated.
   FinalEnergyPrices - 062011.csv                             80  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 072011.csv            fp2011m07.csv         80  Assimilated.
   FinalEnergyPrices - 072011.csv                             81  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 082011.csv            fp2011m08.csv         81  Assimilated.
   FinalEnergyPrices - 082011.csv                             82  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 092011.csv            fp2011m09.csv         82  Assimilated.
   FinalEnergyPrices - 092011.csv                             83  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 102011.csv            fp2011m10.csv         83  Assimilated.
   FinalEnergyPrices - 102011.csv                             84  Has a heading, and the same values (but in day order, not name then day), but deleted as its timestamps are truncated.
   Final Prices - 112011.csv            fp2011m11.csv         84  Now has a heading, and the times are truncated.
   Final Prices - 122011.csv            fp2011m12.csv         85  Now has a heading, and the times are truncated. FinalEnergyPrices - 112011.csv not supplied.
   Final Prices - 012012.csv            fp2012m01.csv         86  The heading vanishes, the times are not truncated, still in name order.
   Final Prices - 022012.csv            fp2012m02.csv         87  The heading returns, and the times are truncated.
   Final Prices - 032012.csv            fp2012m03.csv         88  The heading remains, and times continue to be truncated.
   Final Prices - 042012.csv            fp2012m04.csv         89  The heading remains, and times continue to be truncated.
   Final Prices - 052012.csv            fp2012m05.csv         90  The heading remains, and times continue to be truncated.
   Final Prices - 062012.csv            fp2012m06.csv         91  The heading remains, and times continue to be truncated.
   Final Prices - 072012.csv            fp2012m07.csv         92  The heading remains, and times continue to be truncated.
   Final Prices - 082012.csv            fp2012m08.csv         93  Return to no heading, and times not truncated.
   Final Prices - 092012.csv            fp2012m09.csv         94  Continue with no heading, and times not truncated.
   Final Prices - 102012.csv            fp2012m10.csv         95  Continue with no heading, and times not truncated.
   Final Prices - 112012.csv            fp2012m11.csv         96  Continue with no heading, and times not truncated. Irruption of NAP2202 data, exactly the same as NAP2201 data.
   Final Prices - 122012.csv            fp2012m12.csv         Via e-mail to Brian Kirtlan of file EA data request - data for Dec 2012.zip.
   Final Prices - 012013.csv            fp2013m01.csv         97  CD supply resumes. Continue with no heading, and times not truncated.
   Final Prices - 022013.csv            fp2013m02.csv         98  Continue with no heading, and times not truncated.
   Final Prices - 032013.csv            fp2013m03.csv         99  Continue with no heading, and times not truncated.
   FP_Price_20110326.csv                fp2011m03d26revised.csv   Arrived from Brian. A lot of reformatting required.
   FP_Price_20110326_the_missing_nodes.csv                        Additional names, omitted from the first supply.
========================================================================================================================

                                       "Final Reserve Prices"
   As is obviously a habit, these data arrived in files named with the usual permutations, as follows:
               Random name                      Boring name.
      Rename   FinalReservePrices_200503.csv    rp2005m03.csv
      Rename   FinalReservePrices_200504.csv    rp2005m04.csv
      Rename   final_reserve_Apr04-Feb05.csv    rp2004m04-200503.csv
      Rename   Reserve_1996_10.csv              rp1996m10.csv
      Rename   Reserve_1996_11.csv              rp1996m11.csv
      Rename   Reserve_1996_12.csv              rp1996m12.csv
      Rename   Reserve_1997_01.csv              rp1997m01.csv
      Rename   Reserve_1997_02.csv              rp1997m02.csv
      Rename   Reserve_1997_03.csv              rp1997m03.csv

   And as ever, holding to the same data arrangement is beyond the capacity of these suppliers as well.
   File C:\HH\Source\ReservePrices\data\RP200503.csv presented a header line,
   GIP_GXP_FULL,TO_CHAR(TRADING_DATE,'DD/MM/RR,TRADING_PERIOD,PRICE_6S,PRICE_60S,TO_CHAR(RUN_TIME,'DD/MM/RRRRH,RUN_TYPE
   Which is not particularly helpful and poorly formed as well, so it was prefixed with a % to cause it to be ignored,
especially since the next file introduced a different header.
   File C:\HH\Source\ReservePrices\data\RP200504.csv introduced a header "Grid_Exit_Point,Trading_Date,Trading_Period,Price_6s,Price_60s,Run_Time,Run_Type"
which is now recognised and so is inserted into the earlier files. So far, newer files have yet to introduce variations...

      Rename   FinalReservePrices_200505.csv    rp2005m05.csv   from CD 7.
               FinalReservePrices_200506.csv    rp2005m06.csv           8.
               FinalReservePrices_200507.csv    rp2005m07.csv           9.
               FinalReservePrices_200508.csv    rp2005m08.csv          10.
               FinalReservePrices_200509.csv    rp2005m09.csv          11.
               FinalReservePrices_200510.csv    rp2005m10.csv          12.
               FinalReservePrices_200511.csv    rp2005m11.csv          13.
               FinalReservePrices - 122005.csv  rp2005m12.csv          14.
               FinalReservePrices - 012006.csv  rp2006m01.csv          15.
               FinalReservePrices - 022006.csv  rp2006m02.csv          16.
               FinalReservePrices - 032006.csv  rp2006m03.csv          17.
               FinalReservePrices - 042006.csv  rp2006m04.csv          18.
               FinalReservePrices - 052006.csv  rp2006m05.csv          19.
               FinalReservePrices - 062006.csv  rp2006m06.csv          20.
               FinalReservePrices - 072006.csv  rp2006m07.csv          21.
               FinalReservePrices - 082006.csv  rp2006m08.csv          22.
               FinalReservePrices - 092006.csv  rp2006m09.csv          23.
               FinalReservePrices - 102006.csv  rp2006m10.csv          24b. (The first attempt at CD 24 was unreadable)
               FinalReservePrices - 112006.csv  rp2006m11.csv          25
               FinalReservePrices - 122006.csv  rp2006m12.csv          26 Under directory EC - February 2007
               FinalReservePrices - 012007.csv  rp2007m01.csv          27 Under directory EC - March 2007
               FinalReservePrices - 022007.csv  rp2007m02.csv          28 No directory, no more?
               FinalReservePrices - 032007.csv  rp2007m03.csv          29 Nope. Directory  EC - May 2007.
               FinalReservePrices_200704.csv    rp2007m04.csv          30 No directory is back.
               FinalReservePrices - 052007.csv  rp2007m05.csv          31 And continues! But, with a different file name style.
               FinalReservePrices - 062007.csv  rp2007m06.csv          32
               FinalReservePrices - 072007.csv  rp2007m07.csv          33 The CD wrongly annotated "EC - AUG 07": should be September
               FinalReservePrices - 082007.csv  rp2007m08.csv          34 Annotated EC - Sept 2007, so a revised convention?
               FinalReservePrices_092007.csv    rp2007m09.csv          35 File name change, again.
               FinalReservePrices - 102007.csv  rp2007m10.csv          36 And back, again.
               FinalReservePrices - 112007.csv  rp2007m11.csv          37 Staying with it for now.
               FinalReservePrices - 122007.csv  rp2007m12.csv          38.
               FinalReservePrices - 012008.csv  rp2008m01.csv          39.
               FinalReservePrices - 022008.csv  rp2008m02.csv          40.
               FinalReservePrices - 032008.csv  rp2008m03.csv          41.
               FinalReservePrices - 042008.csv  rp2008m04.csv          42.
               FinalReservePrices - 052008.csv  rp2008m05.csv          43.
               FinalReservePrices - 062008.csv  rp2008m06.csv          44.
               FinalReservePrices - 072008.csv  rp2008m07.csv          45.
               FinalReservePrices - 082008.csv  rp2008m08.csv          46.
               FinalReservePrices - 092008.csv  rp2008m09.csv          47.
               FinalReservePrices - 102008.csv  rp2008m10.csv          48.
               FinalReservePrices_-_112008.csv  rp2008m11.csv          49. Underlines...
               FinalReservePrices - 122008.csv  rp2008m12.csv          50. Back to spaces.
               FinalReservePrices - 012009.csv  rp2009m01.csv          51.
               FinalReservePrices_-_022009.csv  rp2009m02.csv          52. And underlines again.
               FinalReservePrices - 032009.csv  rp2009m03.csv          53. Back to spaces.
               FinalReservePrices - 042009.csv  rp2009m04.csv          54.
               FinalReservePrices - 052009.csv  rp2009m05.csv          55.
               FinalReservePrices - 062009.csv  rp2009m06.csv          56.
               FinalReservePrices - 072009.csv  rp2009m07.csv          57.
               FinalReservePrices - 082009.csv  rp2009m08.csv          58.
               FinalReservePrices - 092009.csv  rp2009m09.csv          59.
               FinalReservePrices - 102009.csv  rp2009m10.csv          60.
               FinalReservePrices -112009.csv   rp2009m11.csv          61. One space saved!
               FinalReservePrices - 122009.csv  rp2009m12.csv          62. Once only!
               FinalReservePrices - 012010.csv  rp2010m01.csv          63.
               FinalReservePrices - 022010.csv  rp2010m02.csv          64.
               FinalReservePrices - 032010.csv  rp2010m03.csv          65.
               FinalReservePrices - 042010.csv  rp2010m04.csv          66.
               FinalReservePrices - 052010.csv  rp2010m05.csv          67.
               FinalReservePrices - 062010.csv  rp2010m06.csv          68.
               FinalReservePrices - 072010 .csv rp2010m07.csv          69. Reversion to the early-style header. Fortunately, times with a.m./p.m. are accommodated by EATHMS.
               FinalReservePrices - 082010.csv  rp2010m08.csv          70. Back to the established style, and no space before the .csv.
               FinaReservePrices - 092010.csv   rp2010m09.csv          71. This time, omit a letter.
               FinalReservePrices - 102010.csv  rp2010m10.csv          72.
               FinalReservePrices - 112010.csv  rp2010m11.csv          73.
               FinalReservePrices - 122010.csv  rp2010m12.csv          74.
               FinalReservePrices - 012011.csv  rp2011m01.csv          75.
               FinalReservePrices - 022011.csv  rp2011m02.csv          76.
               FinalReservePrices - 032011.csv  rp2011m03hic.csv       77. Data for 26/3/2011 was suppressed as "confusing".
               FinalReservePrices - 042011.csv  rp2011m04.csv          78.
               FinalReservePrices - 052011.csv  rp2011m05.csv          79.
               FinalReservePrices - 062011.csv  rp2011m06.csv          80.
               FinalReservePrices - 072011.csv  rp2011m07.csv          81.
               FinalReservePrices - 082011.csv  rp2011m08-m09.csv      82. Suddenly, the two-months-behind data has been joined by one-month-behind data.
               FinalReservePrices - 102011.csv  rp2011m10.csv          83. Supplied with " around every datum (47632/2 in all), and a spurious first column with what appears to be a line count.
               FinalReservePrices - 112011.csv  rp2011m11.csv          84.
               FinalReservePrices - 122011.csv  rp2011m12.csv          85.
               FinalReservePrices - 012012.csv  rp2012m01.csv          86. Another 47632 " around every datum, and a spurious first column.
               FinalReservePrices - 022012.csv  rp2012m02.csv          87.
               FinalReservePrices - 032012.csv  rp2012m03.csv          88.
               FinalReservePrices - 042012.csv  rp2012m04.csv          89.
               FinalReservePrices - 052012.csv  rp2012m05.csv          90.
               FinalReservePrices - 062012.csv  rp2012m06.csv          91.
               FinalReservePrices - 072012.csv  rp2012m07.csv          92.
               FinalReservePrices - 082012.csv  rp2012m08.csv          93. No header supplied; copied from the previous month.
               FinalReservePrices - 092012.csv  rp2012m09.csv          94.
               FinalReservePrices - 102012.csv  rp2012m10.csv          95.
               FinalReservePrices - 112012.csv  rp2012m11.csv          96.
               FinalReservePrices - 122012.csv  rp2012m12.csv Second email Supplied with quotes around every field, and a spurious extra column that appeared to be a record number.
               FinalReservePrices - 012013.csv  rp2013m01.csv          97. No header supplied; copied from the previous month.
               FinalReservePrices - 022013.csv  rp2013m02.csv          98.
               FinalReservePrices - 032013.csv  rp2013m03.csv          99.
               ReservePrices_20110326.csv       rp2011m03d26revised.csv    Arrived from Brian.