How air pollution is measured in Ulaanbaatar
4 different sources, 2 different methods
Air pollution is a big issue during the winter months in Ulaanbaatar, Mongolia. The effects of air pollution are quite severe, with those living in the most polluted areas suffering from lowered lung function, increased rates of respiratory infection, and shortened lifespans. In an attempt to solve this issue, the government has been focusing on the source of air pollution, the peri-urban ger districts surrounding the city center. In these areas, houses burn coal for both cooking and heat.
Beginning in May 2019, burning raw coal in Ulaanbaatar was made illegal. To replace the fuel the government coordinated the manufacture and supply of “enhanced charcoal briquettes”, which is supposed to reduce air pollution in Ulaanbaatar by 40–50%.
Along with this change in fuel, the Ministry of Environment and Tourism made a rule change in October 2018 (and implemented in February 2019) to how the air quality index (AQI) is calculated. This change has two impacts. First, physical values of pollution are no longer reported, only AQI. This means researchers and open data platforms (like openaq.org) can no longer reliably use the data reported. Second, instead of the previous used 1 hour average of air pollution being reported, a 3-hour moving average is used to calculated AQI. This change smoothes out peaks and troughs, and, as I will show later, can easily make it appear that air pollution is reducing.
Before we get into that, let’s take a look at the quite messy situation of air pollution data in Ulaanbaatar.
Data Sources
Most people following air pollution are aware of Agaar.mn, the air quality monitoring site that updates hourly showing air quality at 12 stations around the city. There are 4 different sources of air quality data in Ulaanbaatar:
- Agaar.mn — part of the Ministry of Environment and Tourism — 12 stations
- Tsag-agaar.gov.mn — managed by the National Agency of Meteorology and Environmental Monitoring — 4 stations
- StateAir.mn — operated by the US Embassy in Ulaanbaatar — 1 station
- People in Need — new air quality stations were launched in 2019 by the international NGO — 16 stations
To make things more confusing, the air quality stations run by the Mongolian government use the new reporting standard mentioned above. This means 16 stations are using the new Mongolian reporting standard and 17 stations using the US EPA reporting standard. Here is a table breaking down each station, the source, and the type of air quality monitoring each station performs.
To understand this visually, I put together an interactive map that will allow you to see how these stations are distributed. One of the biggest criticisms of the government air quality sensors is that relatively few are located in the ger district, the area that has the worst air pollution. People in Need has done an excellent job filling in the gaps in the government network.
October 2018 Rule Change
In the past few days, several Mongolian news outlets have been reporting that air pollution has reduced in Ulaanbaatar compared to previous years. Ikon.mn reported that for October 10, 2019, air pollution appears to be lower than the same day in 2017. In addition, Eagle News reported (see below) that October 2019 has seen a 50% reduction in air pollution.
Both reports seem seriously flawed. The Ikon.mn report’s sample size is much too small to make this conclusion, comparing one day of air pollution in 2019 and one day in 2017. That is like saying there is global warming because two years ago that day was colder. The Eagle News report gave no data source for their story. In addition, October wasn’t even half over when this report was made.
One key question is whether it is currently possible to directly compare air quality from before and after the rule change. Let’s break down the differences in calculating the air quality before and after the change. There are several pieces that go into AQI calculations:
- AQI category breakpoints
- Physical value breakpoints
- Aggregation method
- AQI formula
The current Mongolian standard made changes to all three out of the four pieces. Let’s take a look at the AQI category breakpoints compared to the US EPA ones.
These are quite comparable to the US EPA breakpoints, and the colors and general descriptions roughly coincide. The physical value breakpoints had more changes.
Here we can see significant changes. The biggest change is the breakpoints for PM2.5, arguably the most dangerous type of pollution. The US EPA has a limit of 12 micrograms per cubic meter to have “good” quality air. The new Mongolian “good” quality can go up to 35 micrograms per cubic meter. At these levels the US EPA consider the air to be “moderate”. However, at higher values the match up much closer. Here is a plot showing the US vs Mongolian AQI values for PM2.5.
As you can see at lower PM2.5 concentrations the Mongolian AQI is reported to be lower, but meet around 150 micrograms per cubic meter. At higher PM2.5 concentrations the Mongolian AQI is actually higher than the US AQI.
What does this mean? If you are comparing only counts of air quality categories and asking questions like, “How many hours was air quality moderate or above for this month?”, you will see more greens and fewer oranges.
The next big one is aggregation method. The breakpoint tables above say that PM2.5 values are measured on a 24-hour basis. That means a person would need to be exposed to PM2.5 AQI values of above 200 for 24 hours to be considered unhealthy or hazardous on the US EPA scale. However, most PM2.5 stations actually are reported at 1-hour intervals. This makes seeing air pollution spikes and trend easier, but gives a much less clear picture as to the danger air pollution causes in a particular hour.
The new Mongolian air quality standard changes the way AQI values are aggregated. Instead of a 1-hour average of PM2.5, now a 3-hour average is calculated and then converted to the new AQI scale. Let’s take a look at how this changes the physical measurements. (Hint: The chart is interactive, feel free to play with the data.)
If we look at the conversion to US and MN AQI standards, we can see a similar situation as above.
The formula used to calculate AQI was not changed in the recent rule change and is identical to the one used by the US EPA. You can see both formulas here, with the left being the Mongolian formula and the right being the US EPA formula.
What does this mean? Put simply the new Mongolian standard smoothes out the peaks and makes it appear that air quality is lower hour by hour. Does this mean that the data is lying or that the new standard is bad? Not necessarily. What it does mean is that data recorded before and after the rule change cannot be directly compared without transforming the data to a single standard.
What does this mean? Put simply the new Mongolian standard smoothes out the peaks and makes it appear that air quality is lower hour by hour.
What happens if we aggregate the data at the day level (24-hour average)? Let’s see how the US and MN standards compare now.
As you can see, the aggregated at a daily level the standards are quite comparable. This is noticeably different than the hourly comparison above. Towards the end of January when AQI levels were lower, the Mongolian standard has an AQI that is noticeably different. Keep in mind these numbers are based on exactly the same physical values, and the only thing changing is the method used to calculate AQI and aggregate the data.
Conclusion
The recent ban on raw coal, the introduction of a controversial new fuel type, and a change in the way air quality are calculated means that this situation is more complicated than a headline or a soundbite. The move by the Ministry of Environment and Tourism away from reporting physical values is definitely a backward step in bringing transparency to air pollution.
Additionally, I have shown that air pollution data (especially PM2.5 or PM10, which has the same aggregation change) reported before February 2019 should not be compared to data after the rule change. This makes the job of answering the question on everyone’s mind that much harder. Will air pollution get better this year?
You can find the data, code, and the new Mongolian AQI index document on Github here.
Thanks for reading. You can find more about me here. Consider subscribing to get notifications when I publish. If you would like to directly support my articles you can use my referral link to sign up to become a Medium member.
