Retail Power Prices in Texas: Analyzing the Whole Market in Context
This is the second article with a critique and analysis of Texas retail power prices in response to the recent Wall Street Journal article "Texas Electric Bills Were $28 Billion Higher Under Deregulation." In the first article, we discussed the issues associated with using a ‘weighted average’ measure of central tendency to represent a highly skewed distribution of residential power prices. In this article, we cover the last two questions posed previously.
- The WSJ article only covers residential prices. What has transpired in the (roughly 2/3) of the market for commercial and industrial customers? Why is that not part of this story?
- How can I place this $28 Billion in context? What are some other insights that can be gained from this Data?
Code, Formatting, Conventions
As before, all of the data and code supporting this article can be found on github1. If you spot an error, or have an extension, I will gladly take pull requests. All of the code is in python is, and we make heavy use of pandas multiindexing capabilities as well as matplotlib.
Why just focus on the residential market?
Recall the headline graph[2] of the article, reproduced below:
Pricey Power
The roughly 60% of Texans who must choose a retail electricity provider consistently pay more than customers in the state who buy their power from traditional utilities.

So that’s a weighted average of residential prices only. We discussed the statistical fallacy of using the weighted average to represent the typical consumer in a highly skewed price distribution as part of the last article. Here. we’re going to talk about cherry-picking. The residential segment is certainly not the majority of the retail market, nor are the other market segments irrelevant to the concerns of the average consumer. The Energy Information Agency breaks the marketplace into residential, commercial, industrial and transportation components. The transportation sector is relatively smaller and specialized, so we’ll focus here on the first three sectors, which have recorded sales volumes as follows[3] (note shorthand DeReg
for Retail Providers, Reg
for Traditional Utilities).

Of note:
- Although the general rule of thumb in the industry is 1/3 of the sales volumes in each segment, you can observe that, for the ‘traditional utilities’ (Munis, Coops, and Regulated Investor-Owned Utilities), the residential segment dominates their service load.
- For the ‘retail providers’, load is growing faster, and commercial and industrial load is a a greater share of ‘retail providers’ load compared to the load served by ‘traditional utilities’.
Why should the consumer care about commercial and industrial pricing?
We list three main reasons:
- Local commercial Electricity prices are a component of every bill the consumer pays for local goods and services, e.g. the dry cleaner, the dentist office, the grocery, government services.
- Competitive commercial and industrial electricity prices are an essential component to local economic health, job growth, and quality of services.
- Finally, if one sets out to compare price differences between a ‘traditional’ utility and a ‘deregulated’ market, it’s important to include all segments, as the regulated model has a tendency to cross-subsidize. In the absence of market forces (and often, transparency), regulated electricity pricing becomes, at least in part, a political matter. Considering only one market segment will miss potential effects of cross-subsidies.
What happens if we analyze the whole market, not just residential?
Analysis: We repeat our graph from the last article, featuring weighted average and median provider prices, for the other two segments; therefore we have 3 graphs in this panel, with a fixed window size of ¢7/kwh, but with different endpoints selected to reflect the range of data. This visualization design yields the following [3]:

Our commentary begins by noting that residential prices are generally higher than commercial, which in turn are generally higher than industrial, which we would expect, due to differing:
- Volatility, shape and weather sensitivity of customer load.
- Administrative, settlement and billing costs.
- Costs of consumer or business credit (which costs are particularly important for fixed-price contracts).
If we examine the relationship of weighted average to median provider prices, and we also note the relationship of retail provider prices compared to ‘traditional utility’, we observe the following.
- The only price series where the retail provider price doesn’t cross below the ‘traditional utility’ price in the timespan of the dataset is the weighted average price of the residential segment. In all other segments, whether comparing median suppliers or weighted-average price, retail provider prices have trended lower than ‘traditional utilities’ over time, as have median provider prices in the residential segment.
- Focusing on the retail provider space only [‘DeReg’ graph labels] the only segment where we observe a significant divergence between weighted average price and median supplier price is the residential sector – for the other two sectors, both measures track much more closely.
- We see significant price skew among the ‘traditional utility’ commercial customers, but that price skew points in the opposite direction compared to the ‘retail provider’ residential price distribution. So while the customer of the median residential retail provider pays significantly less than weighted average, the customer of the median commercial traditional utility pays significantly more.
We examined the price distribution of residential customers in the last article; here, we’ll dig into the price distribution for commercial customers. Lets look at the distribution for 2010, during the period when we see deregulated commercial prices start to decline sharply[3]:

For retail providers (top graph) we see the two dominant legacy providers (TXU and Reliant), again with prices well above the median or weighted average price. For regulated entities, we see the lower prices featured by a handful of large utilities (e.g. City of San Antonio, Southwestern Public Service, City of Austin, Entergy Texas). But the customer of the median supplier among these 144 utilities isn’t doing nearly as well in terms of price: to cite two examples, Taylor, TX [35 miles from Austin], averages ¢11.28 while Austin Energy customers pay ¢8.98. The City of Liberty receives power from the Entergy Texas system, but its price is ¢11.15 vs ¢6.94 for Entergy’s own customers. Once again, in comparing these two regulatory regimes, the use of weighted average for comparison yields a distorted result, and leaves out many customers whose prices vary significantly from the weighted average.
But this graph raises a good question – look at the top graph: what’s going on with the weighted average for retail providers? The large number of high-priced customers of the legacy providers doesn’t seem to be increasing the weighted average as much as we might expect. The answer, of course, is that the weighted average is by volume, and this graph looks at pricing by customer. But commercial customers vary much more widely in volume compared to residential; so, if we look at prices by volume, we can see what is happening[3]:

Comparing this graph to the previous:
- In the retail provider space, we see that a relatively smaller group of high-volume customers switched to non-legacy providers, thus making the volume-weighted average and median provider price relatively close together, even though legacy providers maintained a large number of smaller customers.
- In the traditional utility space, we see that the higher-priced utilities have a relatively large number of smaller customers.
The point of this exercise is to reinforce the message in The Price of Power, Part 1; the outcome of the analysis of price differences between retail providers and traditional utilities is highly dependent on the measure of central tendency chosen. We have discussed and demonstrated the impact of using the volume-weighted average price and median provider price here, but there are other potential measures; one could weight the average by customer count instead of volume, for example, or use the median price by customer (as opposed to the median-price provider). We argue here that the median provider price is a better choice than volume-weighted average, because of significant skew in price distributions (however weighted) and because the objective of the analysis is simply to compare regulatory models in highly different circumstances:
- a fast-growing retail provider space which customers can choose between legacy providers and new entrants, all of whom operate in a common wholesale market, and
- a slower-growing traditional utility space which is highly diverse in terms of location, climate, wholesale market, entity type (Municipal, Co-op, Investor-Owned-Utility), and size.
So, the circumstances argue for a median, but which median? Here again, the median provider price is more representative – For the graphs above, the median customers are with TXU at ¢13.19 for retail providers and Austin Energy at ¢8.98 for traditional utilities[3]. So, for the retail provider space, until new entrants gain sufficient market share, legacy providers serve the median customer. So once again we’re back to the provider median as the best (but still imperfect) measure for our purposes. The important part here is the discussion of tradeoffs: any choice of central tendency (or other statistical measure) should be accompanied by robust discussion and analysis.
We conclude this section by isolating the effect of cherry-picking the residential segment to compare pricing: using the reviewed article’s weighted-average method – how would the headline graph above had appeared if the whole market were accounted for?[3]

So, in summary, the only way to obtain the reviewed article’s dramatic headline graph and $28 Billion headline number is to (1) use the weighted average price measure and (2) select only the residential market, in order to obtain a price series where retail providers appear to offer a consistently worse alternative.
What results obtain if we analyze the whole market, and use median provider prices?
Comparing median price differences as depicted above multiplied by retail provider sales volumes yields the following dollar-denominated bar graph (bear in mind the convention here is traditional utility price minus retail provider price)3:

So this was the same process the reviewed article used to calculate the "$28 Billion" value in the headline (price difference multiplied by sales volume), but here, our results are much different. Summing across time in the graph above yields:
So, by using median provider prices against the entire retail market, we calculate a result almost exactly opposite to that asserted in the article, and we discern a trend which shows increasing benefit for deregulated residential and commercial customers during the timespan of this analysis. More importantly, here we have improved on the article standards by stating our assumptions and the implications behind them clearly, and making our analysis and the dataset public.
So are we done with the analysis?
No. Our analysis here (and in the reviewed article) lacks context. A billion dollars might make for a good headline, but that figure doesn’t carry meaning in terms of the impact to a typical consumer. So let’s add that context! First, what does a typical monthly bill look like for this study period? For customers in the retail provider space, we sum across years to get the following in terms of average monthly consumption volume, total bill, and unit price3:
So that chart gives context to the unit price graphs above. Recasting the bar graph (in $billions, above) to one that calculates monthly billing impact yields the following[3]:

Now we have a meaningful results; we know the price impact of the median price difference on the monthly customer bill, and the average bill for those customers – in a word, context.
What are some other insights that can be gained from this data?
Perhaps the most interesting insight here with respect to consumer education and potential other policy implications is the consumer cost of remaining with the legacy supplier. We have referenced this multiple times in this article and previously; in this section we will quantify that cost.
Who are the legacy providers?
Our first task is to identify the legacy companies in the data, and right away we run into the practical problem that our data lack common company names and parent company identification. Naively searching for "TXU" and "Reliant" in the Entity
field we find[3]:
Examining sales and duration of these entities, we find that some appear to be special-purpose providers of limited duration. Eliminating those, we define legacy provider as any of the following to get one Reliant Entity and one TXU entity per year[3]:
So now we wish to characterize the prices of these providers versus the prices of numerous non-legacy providers over time. One interesting way to do that is a box-and-whisker plot (number of non-legacy providers in gold at the bottom of each plot)[3]:

We examine those unit prices in the context of these sales volumes for legacy providers:

We see:
- Legacy provider prices almost always exceed the median provider, and consistently exceed the 75th percentile-provider.
- Disturbingly, residential legacy provider price spreads vs median grow in the last few years (even as legacy market share drops).
This is arguably the most important story in this data set: unlike the counterfactual of the ‘traditional utilities’, as discussed above, here we have evidence of a persistent price disadvantage for a group of customers who are nonetheless fully capable of making a different choice. We can quantify the impact of that choice as before; start with legacy volumes multiplied by median price difference (Legacy – Non-Legacy)[3]:

Summing across years, we have the following gross impact in $Billions:
And, as before, we estimate the billing impact in terms of $/Customer-Month:

Summary: In this two-article series, we have examined retail electricity prices in Texas as part of a critique of an article in the popular business press which examined the same data. We found issues with the use of the weighted-average as a measure of central tendency in this data, and also with the use of a subset of the retail market (residential) as opposed to analyzing the whole market. We have presented an alternative analysis which corrects for these deficiencies, and extended that analysis to examine an actionable issue uncovered by the data, namely the consistently high prices paid by customers who have declined to switch from legacy providers.
Next Steps: My objectives in writing these articles was to (1) demonstrate better analysis practices than I saw in the reviewed article, (2) provide an alternative analysis consistent with the data and with my own personal experience as a 12-year residential retail customer of the Texas electricity market, and (3) highlight use of the Energy Information Administration retail prices dataset for consumer and policy use. I’m planning to follow this analysis up by writing to the Texas Public Utilities Commission urging them to make some use of this dataset on their consumer-facing website powertochoose.org. PowertoChoose.org already posts consumer reviews of retail providers, and the posting of historical price performance of market participants there would be complementary and straightforward. I would encourage readers who are Texas residents to join me in this advocacy – this dataset bears on consumer decisionmaking, it is a public dataset maintained at taxpayer expense, and the data seem to be underutilized – the solution would be to publish it more widely to allow consumers to be better educated. I’m hopeful other interested readers may utilize this data for other regions or analysis purposes as appropriate.
References
- Main Code Repository: github.com/dkfurrow/eia-retail-analysis
- Article graph reproduction: wsj_graph_reproduction.py
- Main analysis, visualizations: eia_retail_analysis1.py