The head of China's statistics bureau ordered statisticians to stop using methods that introduce innaccuracies in grain statistics. In doing so, he reveals some of the problems that plague China's statistical system.
On June 9, 2013, Ma Jiantang--commuist party chief and director of China's National Bureau of Statistics--was inspecting the summer grain crop in Anhui Province when he called for local statistical survey personnel to ensure the correctness and truthfulness of statistics on grain output. His comments implicitly reveal some of the problems with the statistics.
Actually, China's grain statistics are among the few agricultural data items that are estimated with statistically sound survey methods. Most agricultural data is still reported up from level to level using administrative methods. But for grain crops, the statistics bureau selects a national sample of villages and plots of land and sends out survey teams to collect cuttings from fields in order to estimate yields. Area planted is based on surveys of farmers' reported plantings in the sample villages which are benchmarked to total crop area measured in decennial agricultural censuses.
The main problem seems to be that the survey teams fail to strictly follow procedures and the National Bureau has no means of knowing how the survey teams actually collect the data. The survey teams are composed of provincial and local statisticians, town and village officials. Based on Mr. Ma's orders, it sounds like local survey teams adopt varying methods, either out of laziness or intent to bias the statistics.
Mr. Ma ordered sample survey teams to strictly implement scientific sampling and estimation methods in a unified and standardized manner. In other words, "Do it the way we told you to do it!"
Ma ordered the statisticians to take actual cuttings from actually measured sample fields without expanding or shrinking the number. In other words, "Don't just make up numbers because it's too much trouble to go out to the village and actually take samples. Record what the number actually was, not what you think it should be! Measure the field with a tape measure, not a bamboo rod!"
Ma said not to miss any grain on the stalks and carefully measure every grain; thresh the grain carefully; and don't take grain from another field to measure output from a sample field. In other words, "Don't be lazy about measuring the grain, and if all the stalks in a field were killed by bugs put down a zero--don't go looking for a 'better' field as a substitute!"
Ma told them to carefully dry the grain, accurately measure the moisture and weight, and make reasonable estimates of losses based on local practices in harvesting and storing grain. In other words, "Don't record an overestimate of the weight of wet grain and don't forget to deduct the grain that falls on the ground and gets eaten by rats when it's shoveled into bags!"
Mr. Ma said to follow strict regulations in auditing and reporting data up to the next level. In other words, "Don't pad the numbers to make the local officials look good or to qualify for subsidies!"
Ma Jiantang's orders are a reminder that procedures are rarely carried out as they are written. While the leaders may have devised "scientific" methods learned from western "experts," they rarely have any control over how procedures are implemented on the ground by people who have a grade school education or who have an interest in manipulating the data. China's "scientific" approach to development is rarely scientific in actuality since one of the foundations of science is strict standardized procedures. Standardization is antithetical to Chinese society which is based on the "hills are high and emperor is far away" maxim that has always conferred flexibility on local officials in implementing the "emperor's" orders.
The implicit acknowledgement of the statistics' lack of accuracy and truthfulness will come as a surprise to Mr. Chen Xiwen, the Chinese communist party's top rural policy advisor. In 2008, Mr. Chen felt compelled to assure the world that China's grain statistics are true and reliable because they are based on scientific survey methods.
On June 9, 2013, Ma Jiantang--commuist party chief and director of China's National Bureau of Statistics--was inspecting the summer grain crop in Anhui Province when he called for local statistical survey personnel to ensure the correctness and truthfulness of statistics on grain output. His comments implicitly reveal some of the problems with the statistics.
Actually, China's grain statistics are among the few agricultural data items that are estimated with statistically sound survey methods. Most agricultural data is still reported up from level to level using administrative methods. But for grain crops, the statistics bureau selects a national sample of villages and plots of land and sends out survey teams to collect cuttings from fields in order to estimate yields. Area planted is based on surveys of farmers' reported plantings in the sample villages which are benchmarked to total crop area measured in decennial agricultural censuses.
The main problem seems to be that the survey teams fail to strictly follow procedures and the National Bureau has no means of knowing how the survey teams actually collect the data. The survey teams are composed of provincial and local statisticians, town and village officials. Based on Mr. Ma's orders, it sounds like local survey teams adopt varying methods, either out of laziness or intent to bias the statistics.
Mr. Ma ordered sample survey teams to strictly implement scientific sampling and estimation methods in a unified and standardized manner. In other words, "Do it the way we told you to do it!"
Ma ordered the statisticians to take actual cuttings from actually measured sample fields without expanding or shrinking the number. In other words, "Don't just make up numbers because it's too much trouble to go out to the village and actually take samples. Record what the number actually was, not what you think it should be! Measure the field with a tape measure, not a bamboo rod!"
Ma said not to miss any grain on the stalks and carefully measure every grain; thresh the grain carefully; and don't take grain from another field to measure output from a sample field. In other words, "Don't be lazy about measuring the grain, and if all the stalks in a field were killed by bugs put down a zero--don't go looking for a 'better' field as a substitute!"
Ma told them to carefully dry the grain, accurately measure the moisture and weight, and make reasonable estimates of losses based on local practices in harvesting and storing grain. In other words, "Don't record an overestimate of the weight of wet grain and don't forget to deduct the grain that falls on the ground and gets eaten by rats when it's shoveled into bags!"
Mr. Ma said to follow strict regulations in auditing and reporting data up to the next level. In other words, "Don't pad the numbers to make the local officials look good or to qualify for subsidies!"
Ma Jiantang's orders are a reminder that procedures are rarely carried out as they are written. While the leaders may have devised "scientific" methods learned from western "experts," they rarely have any control over how procedures are implemented on the ground by people who have a grade school education or who have an interest in manipulating the data. China's "scientific" approach to development is rarely scientific in actuality since one of the foundations of science is strict standardized procedures. Standardization is antithetical to Chinese society which is based on the "hills are high and emperor is far away" maxim that has always conferred flexibility on local officials in implementing the "emperor's" orders.
The implicit acknowledgement of the statistics' lack of accuracy and truthfulness will come as a surprise to Mr. Chen Xiwen, the Chinese communist party's top rural policy advisor. In 2008, Mr. Chen felt compelled to assure the world that China's grain statistics are true and reliable because they are based on scientific survey methods.
Comments