printer friendly version

NGI Price Data Automation and Retrieval

Data from NGI is available in tab separated format for three data series. The three data series are from NGI's Daily Gas Price Index (daily series), NGI's Weekly Gas Price Index (weekly series) and NGI's Bidweek Survey (bidweek series).

Format of Data Files:

The data files are in tab separated format.

For the daily series the fields are:

• Pointcode – an alpha numeric code for the delivery location. The delivery location, as it appears in the publication, is specified in the “pricing point” field(see below). The pointcode will not change for a particular location over time whereas the “pricing point” may change as pipelines change ownership or are re branded (e.g. Arkla became Noram, became Reliant, and is now CenterPoint)
• issue date – the issue date of NGI's Daily Gas Price Index in which this pricing information appears.
• trade date – the trade date gas was surveyed.
• region – the region section that the pricing point appears in as published in NGI's Daily Gas Price Index
• pricing point – the name of the location as it appears in NGI's Daily Gas Price Index. Note that this name may change over time and should not be used as a key field in your application. The Pointcode above does not change and would be more appropriate for a key field.
• low – the low of trades included in our survey.
• high – the high of trades included in our survey.
• Average – the weighted average of trades.
• Volume – the sum of the volume of the deals included in our survey, expressed in thousand MMBtus/day.
• Deals – a count of the transaction reports included in our survey.
• Delivery Start Date – the first day the gas in nominated to flow.
• Delivery End Date – the last day the gas in nominated to flow.

For the weekly series the fields are:

• Pointcode – same as the daily series
• Issue Date – The publication date of NGI's Weekly Gas Price Index. This will always be a Monday.
• Survey Start – the first date in which trading occurred for the survey period. This will be the first business day of the preceding week.
• Survey End – the last date in which trading occurred for the survey period. This will be the last business day of the preceding week.
• region – same as the daily series
• pricing point – same as the daily series
• Bidweek – this is a flag to let you know if this is bidweek series data or weekly series data. For the weekly series this will always be “N”
• low – the low of trades included in our survey.
• high – the high of trades included in our survey.
• Average – the average
• Volume – the sum of the volume of the deals included in our survey, expressed in thousand MMBtus/day.
• Deals – a count of the transaction reports included in our survey.

For the bidweek series the fields are :

• Pointcode – same as the weekly series
• Issue Date – The publication date of NGI's Bidweek Survey. This will always be the first calendar day of the month and corresponds to the delivery month for the survey.
• Survey Start – The first trade date surveyed.
• Survey End – The last trade date surveyed.
• Bidweek – this is a flag to indicate if this is bidweek series data and will always be “Y”.
• region – same as the weekly series
• pricing point – same as the weekly series
• low – the observed low*
• high – the observed high*
• Average – the average*
• Volume – the sum of the volume of the deals included in our survey, expressed in thousand MMBtus/day.
• Deals – a count of the transaction reports included in our survey.

Availability:

With each issue of the corresponding publication NGI publishes a flat, tab separated file containing the Natural Gas price data from that publication. The daily series is available no later than 3 a.m. EDT each business day, where a business day is defined as any day the IntercontinentalExchange is trading physical natural gas. The weekly series is available every Monday by 3 a.m. EDT, regardless if that Monday happens to be a holiday. The bidweek series is available no later than the afternoon of the first business day of the month.

The current files for the three data series are available on our web site:

Daily series: http://intelligencepress.com/subscribers/daily/delimited/20130618td.txt
Weekly series: http://intelligencepress.com/subscribers/weekly/delimited/20130617wt.txt
Bidweek series: http://intelligencepress.com/subscribers/weekly/delimited/20130601bw.txt

Please note that ongoing access to these files is a premium service and that you will need to have a license agreement with Intelligence Press in place. Limited access for evaluation and testing purposes is permitted.

Ftp access to these files is also available for an additional fee.

Automating Download of Data Files:

If you wish to schedule automated downloads of data files there are a number of ways you could accomplish that. One way would be to automate an http get of the files in question. To help you do that there are three special urls that will always return data from the most recent issue of the corresponding series. Those urls are :

http://intelligencepress.com/subscribers/daily/delimited/daily_series.txt
http://intelligencepress.com/subscribers/weekly/delimited/weekly_series.txt
http://intelligencepress.com/subscribers/weekly/delimited/bidweek_series.txt

You could schedule a pull of those three files every morning for processing (using a tool like wget or libwww).

Another approach would be to calculate the current date in a script and retrieve the appropriate file names. The advantage of doing this would be that you could know that you had retrieved the data for a given issue date. For example scheduling something like this perl script to run from cron every day will automate the retrieval of the appropriate files and notify you if information was missing from a certain date (which would happen in the case of holidays).

A third approach would be to mirror the directories with something like :

wget -N -L -np -r --http-user=username --http-passwd=password -nH \
http://intelligencepress.com/subscribers/weekly/delimited/

If you decide to mirror the directories, please do not do so more than once per day without first talking to us about your schedule for doing that.

Corrections and Error Resolution:

Occasionally a correction will be made to the prices in either the daily, weekly, or bidweek series. These corrections can automatically be applied to your data sets by loading and processing the errata files for each of those series.

http://intelligencepress.com/subscribers/daily/delimited/corrections_td.txt
http://intelligencepress.com/subscribers/weekly/delimited/corrections_wt.txt
http://intelligencepress.com/subscribers/weekly/delimited/corrections_bw.txt

Historical Data:

A dump of historical data is available as a series of delimited files, one for each pointcode. Please contact us for more information.

Notes: * In the bidweek series for some pricing points NGI has used editorial judgement based on confirmed bids or asks, or relationships to other market locations in determining the published price; these points are marked with an asterisk in the printed or web version of NGI's Bidweek survey.