See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/335800618

Mapping paddy rice by the object-based random forest method using time
series Sentinel-1/ Sentinel-2 data

Article  in  Advances in Space Research · September 2019

DOI: 10.1016/j.asr.2019.08.042

CITATIONS READS

62 1,705

3 authors, including:

Yaotong Cai Meng Zhang

Sun Yat-Sen University Central South University of Forestry and Technology
6 PUBLICATIONS   161 CITATIONS    15 PUBLICATIONS   285 CITATIONS   

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Yaotong Cai on 16 February 2020.

The user has requested enhancement of the downloaded file.

 Available online at www.sciencedirect.com

ScienceDirect
Advances in Space Research 64 (2019) 2233–2244
www.elsevier.com/locate/asr

Mapping paddy rice by the object-based random forest method

using time series Sentinel-1/Sentinel-2 data
Yaotong Cai, Hui Lin, Meng Zhang ⇑
Research Center of Forestry Remote Sensing & Information Engineering, Central South University of Forestry & Technology, Changsha 410004, China
Key Laboratory of Forestry Remote Sensing Based Big Data & Ecological Security for Hunan Province, Changsha 410004, China
Key Laboratory of State Forestry Administration on Forest Resources Management and Monitoring in Southern Area, Changsha 410004, China

Received 1 March 2019; received in revised form 10 July 2019; accepted 28 August 2019
Available online 13 September 2019

Abstract

Rice is one of the world’s major staple foods, especially in China. In this study, we proposed an object-based random forest (RF)
method for paddy rice mapping using time series Sentinel-1 and Sentinel-2 data. Firstly, the Robust Adaptive Spatial Temporal Fusion
Model (RASTFM) was used to blend MODIS and Sentinel-2 data for achieving multi-temporal Sentinel-2 data. Subsequently, the
Savitzky-Golay filter (S-G) was applied to smooth the time series Sentinel-2 NDVI data. And the phenological parameters were derived
from the filtered time series NDVI using the threshold method. Then, the optimum feature combination for paddy mapping was formed
on the basis of Sentinel-2 MSI images, time series Sentinel-2 NDVI, phenology data and time series Sentinel-1 SAR backscattering
images by using the JBh distance. Finally, an object-based Random Forest classifier was used to extract paddy rice with the optimum
feature combination. The result showed that fused Sentinel-2 NDVI time series using RASTFM has a high correlation with the original
Sentinel-2 image. The overall accuracy and Kappa coefficient of the classification results are higher than 95% and 0.93, respectively, when
use the optimum feature combination and object-based RF method. The proposed method can provide technology support for rice map-
ping in areas with a lot of cloudy and rainy weathers.
Ó 2019 COSPAR. Published by Elsevier Ltd. All rights reserved.

Keywords: Paddy rice; Time series; Sentinel-2; Sentinel-1; RASTFM; Object-based

1. Introduction Vegetation phenology information derived from time

series satellite images plays an important role in vegetation
Paddy rice is a widely planted staple food crop in China. monitoring and land cover classification, because it can
Currently, the production of paddy rice faces many chal- capture vegetation features at different growth stages
lenges due to the population growth, urbanization expan- (Kennedy et al. 2010; Bullock et al. 2019). In the last dec-
sion and environment degradation and natural hazards ades, time-series remote sensing data, especially optical
(Dong and Xiao, 2016; Zhang and Lin, 2018). Therefore, remote sensing images, like MODIS and Landsat, has been
timely and accurately monitoring of paddy rice planting widely used in monitoring and mapping paddy rice.
area is of great significance for national food security and Moderate resolution imaging spectroradiometer (MODIS)
management (see Fig. 1). data was frequently employed to get paddy rice informa-
tion at global or national scales because of its high tempo-
ral and moderate spatial resolution (Xiao et al. 2005;
Pittman et al. 2010; Peng et al. 2011; Gumma et al. 2014;
⇑ Corresponding author. Yeom and Kim. 2015; Shi and Huang. 2015; Teluguntla
E-mail address: [email protected] (M. Zhang).

https://doi.org/10.1016/j.asr.2019.08.042
0273-1177/Ó 2019 COSPAR. Published by Elsevier Ltd. All rights reserved.
2234 Y. Cai et al. / Advances in Space Research 64 (2019) 2233–2244

Fig. 1. Location of the study area (Sentinel-2 image on April 18 of true color composite).

et al. 2015; Zhang et al. 2015). However, its low spatial res- scheduling and other issues. The robust adaptive spatial
olution (250–500 m) may compromise detailed paddy temporal fusion model (RASTFM) (Zhao et al., 2018),
information in the region with heavily heterogeneity. With which fully considers the aforementioned problems, can
a higher spatial resolution data may increase the accuracy achieve a higher prediction accuracy than that of tradi-
of paddy rice mapping (Qin et al. 2015; Dong et al. tional fusion models. In this study, we used RASTFM to
2015), such as Landsat and Sentinel-2 data, especially the generate a time series Sentinel-2-like NDVI data.
Sentinel-2 imagery with spatial resolutions of 10 m, 20 m, This study developed a method for mapping paddy rice
60 m, temporal resolution of 5 days and 13 bands. But with high accuracy in subtropical areas. Although a good
Sentinel-2 is vulnerable to rainy and cloudy weathers, classification result of wetland vegetation can be obtained
which make it difficult to get enough clear imagery for rice from the time series Sentinel-2 image, a time series syn-
paddy monitoring in tropical and subtropical area (Whyte thetic aperture radar (SAR) data may provide higher veg-
et al., 2018). etation classification accuracy, as the radar signal can
Fusing high-spatial/low-temporal resolution data with penetrate clouds, rain, vegetation and soil. Many studies
high temporal/low spatial resolution data (e.g., Landsat have demonstrated that combining optical and radar data
and MODIS) can generate high temporal/high spatial res- can increase the classification accuracy in tropical areas
olution data. A lots of spatial-temporal fusion models have (Jin et al., 2017; Tricht et al., 2018; Hakdaoui et al.,
been proposed and applied in land use/cover researches, 2019). Thus, time series Sentinel-2 and Sentinel-1 imagery
such as the spatial and temporal adaptive reflectance fusion were employed to get the paddy map of the Dongting Lake
model (STARFM), the enhanced STARFM, and the area of Hunan Province, China, a humid region. The field
spatio-temporal integrated temperature fusion model data was used to validate the accuracy of the paddy map.
(STITFM) (Hilker et al., 2009; Walker et al., 2012;
Eemlyanova et al., 2013; Jia et al., 2014). But these fusion 2. Study area and data
algorithms have some limitations (Dao and Liou, 2015; Wu
et al., 2012; Wang and Atkinson, 2017). First, the complex- 2.1. Study area
ity of land cover change in homogeneous or heterogeneous
landscape is not considered comprehensively. The land The Dongting Lake wetland (latitudes 28°300 to 29°310 N
cover change includes invisible change and shape change. and longitudes 111°400 to 113°100 E) is located in the middle
Second, most of spatial-temporal fusion algorithms need reaches of Yangtze River, and on the south bank of the
two or more low-high spatial resolution images as the pri- Jingjiang River section. This area mainly distributes low
ori information, which are usually insufficient due to the plains, with an elevation smaller than 50 m. It has a sub-
cloud pollution, sensor aging or failure, satellite mission tropical monsoon climate, with the annual average temper-
 Y. Cai et al. / Advances in Space Research 64 (2019) 2233–2244 2235

ature between 15.8 °C and 17.4 °C, and an annual precipi- 10 m  10 m, representing a typical vegetation community
tation between 1200 mm and 1500 mm. This area is an with homogeneous vegetation and land cover types. The
important grain production center of China. The growing land use maps of Hunan Province (Hunan Provincial
season of paddy rice was April-October (Zhang et al., Department of Natural Resources, 2018) and Google earth
2017). It also grows natural wetland vegetation, sedge images were also used for training samples selection.
and reed, whose growing seasons are March–October and
March–November, respectively. 3. Method

2.2. Data and processing The proposed wetland vegetation classification process
has three main steps (Fig. 3). First, the RASTFM model
The Sentinel-2 data has 13 bands, including visible, near was employed to generate a synthetic NDVI with high spa-
infrared and short-wave bands. The five near infrared tial and temporal resolution from Sentinel-2 and MODIS
bands (4 red edge bands and 1 NIR band) can be used images. Second, use the separability analysis method to
for vegetation monitoring and analysis. Clear Sentinel-2 generate the optimum feature combination using the
images (<2% cloud cover) over the study area (path/row: Sentinel-2 MSI images, time series Sentinel-2 NDVI, phe-
N0205_R075_T49RFM) were acquired in 2018 (Fig. 2). nology data and time series Sentinel-1 SAR data. Finally,
Terrain correction and atmospheric correction were con- extract the wetland vegetation types by the object-based
ducted to the Level-1C data using SRTM DEM (http:// RF method.
www.gscloud.cn) and Sen2Cor Version-2.5.5 (Luis et al.,
2016; Available online: http://step.esa.int/main/third- 3.1. The RASTFM model for time series Sentinel-2
party-plugins-2/sen2cor/). The spectral bands with 10 m reflectance data
spatial resolutions (including Blue, Green, Red and NIR
bands) were used in this study. The robust adaptive spatial temporal fusion model
We selected 23 16-day composited vegetation index (RASTFM) employs a pair of high and coarse resolution
products (MOD13Q1) between January and December images obtained on the same day, and a coarse resolution
2018 acquired from the United States Geological Survey image obtained on the prediction day to predict the fine
(USGS) for the study. These images have a spatial resolu- resolution on the prediction day. RASTFM considers both
tion of 250 m in the sinusoidal projection. After removing the non-shape change prediction of land cover and the
the invalid values using the pixel reliability images, all of shape change prediction, which is better than other
the MOD13Q1 time series were transformed into the spatial-temporal fusion models that only consider one
UTM (WGS84) projection, zone 49 (North), the same as change. The RASTFM model consists of five parts, which
Sentinel-2 MSI (Zhang and Lin, 2018). are the relative radiometric normalization and radiometric
In addition, 29 Sentinel-1 GRD images (C-band, double de-normalization, non-shape change prediction, shape
polarizations with VV and VH) in 2018 with a spatial res- change detection, shape change prediction and high-pass
olution of 10 m were downloaded from European Space modulation. The specific process and implement steps can
Agency (ESA) (scihub.copernicus.eu/). These images were be found in Zhao et al. (2018).
processed by orbital correction, thermal noise removal, We selected the Sentinel-2 data from day of year (DOY)
radiometric calibration, speckle filtering, range-doppler 108 and the MODIS data from DOY113 (the time nearest
terrain correction in Sentinel Application Platform (SNAP) to the acquisition date of Sentinel-2 data) as the input base
software provide by ESA (Schubert et al., 2015; Diñeiro pair images, and MODIS data from DOY273 as the coarse
et al., 2019). In this study, we utilized the backscattering resolution image on the prediction day to predict the
data (r0VV and r0VH ) to extract the wetland vegetation Sentinel-2 data in DOY278 (Table 1).
information.
The field data used in this study were collected in May 3.2. Sentinel-2 NDVI time series reconstruction and
2018. Considering the vegetation growth and distribution, phenology data extraction
we selected 186 sample sites and obtained their geographi-
cal coordinates by GPS. Using the stratified sampling, we In order to eliminate the noise in the fused time series
classified the vegetation in the sample sites into sedge, reed Sentinel-2 NDVI data caused by cloud contamination
and paddy rice. Each sample site has an area of and errors during prediction, the Savizky-Golay (S-G) fil-

Fig. 2. Date of the remote sensing imagery (S-1 and S-2 represent Sentinel-1 and Sentinel-2, respectively).
2236 Y. Cai et al. / Advances in Space Research 64 (2019) 2233–2244

Fig. 3. Flowchart of the wetland vegetation classification process.

Table 1 threshold. The Dongting Lake wetland has a subtropical

Input images of the RASTFM for prediction and validation. monsoon climate. The crops ripe one or two times per year.
Input Input Input Validation Considering the growth characteristics of rice and other
Sentinel t0 MODIS t0 MODIS tk Sentinel tk vegetation, five vegetation phenological parameters
Test DOY108 DOY113 DOY273 DOY278 were mapped on the basis of the fitted NDVI time series
(Table 2).

ter was employed (Fig. 4). This filter could clearly describe 3.3. Optimum feature combination selection
minor changes in the study area, despite the complex crop
types and broken plots (Jönsson and Eklundh, 2004; The optimum feature combination analysis was used to
Kontgis et al., 2015; Zhang et al., 2017). The S-G filtering detect the image combination, which can effectively
was performed with a locally adapted moving window, improve the highest classification accuracy with lowest
which utilizes a polynomial least-squares regression to fit data redundancy and time consumption (Dobson et al.,
the time-series data. 1992; San Miguel-Ayanz and Biging, 1997; Murakami
The phenology data were derived from the Sentinel-2 et al., 2001; Van Niel et al., 2005). And a separability anal-
NDVI time series by the threshold method (Jönsson and ysis method (JBh) was used to select the optimum feature
Eklundh, 2004) that assumes that a phenological phe- combination using Sentinel-2 MSI images, time series
nomenon occurs when the NDVI values exceed a given Sentinel-2 NDVI, phenology data and time series

Fig. 4. A pixel of paddy rice of the Sentinel-2 NDVI time series before and after the S-G filtering.
 Y. Cai et al. / Advances in Space Research 64 (2019) 2233–2244 2237

Table 2
Phenological parameters.
Phenological parameters Definition
Start of the season (SOS) Time for which the left edge of NDVI curve has increased to 20% of the seasonal amplitude measured from the left
minimum level.
End of the season (EOS) Time for which the right edge of NDVI curve has decreased to 20% of the seasonal amplitude measured from the right
minimum level.
Length of the season (LOS) EOS-SOS
Max of NDVI (MON) The largest NDVI value of the growing season.
Amplitude of NDVIVI The difference between the maximum NDVI and the base level.
(AON)

Sentinel-1 SAR backscattering images (a total of 105 fea- accuracy (UA). We also classify the prototype objects by
tures). Considering the separability of multi-classes, classes several other classifiers (such as nearest neighbor (NN),
are given different weights. JBh (Eq. (1)) was used to select Maximum likelihood (MLC) classification and regression
optimal temporal windows for multi-classes and it gives tree (CART), and support vector machine (SVM). For
great importance to classes with high a priori probabilities comparison, pixel-based NN, MLC, CART, SVM and
in the selection process. RF classifiers were also used for classification.
XN X N qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
J Bh ¼ pðwi Þ  pðwj Þ  JM 2 ði; jÞ ð1Þ
i¼1 j>i
4. Result and analysis

where N is the number of classes, p(wi) and p (wj) are the a 4.1. The fused Sentinel-2 NDVI data
priori probabilities of classes i and j, respectively. They
were calculated using the combination of training and val- The MODIS NDVI, fused Sentinel-2 NDVI and actual
idation samples. JM is the Jeffreys-Matusita distance which Sentinel-2 NDVI of the study area on DOY278 are shown
was usually employed to express the separability of classes. in Fig. 6. The fused NDVI image using RASTFM has a
In this section, the optimum feature combination was much higher spatial resolution, and more details than the
selected using MATLAB. MODIS image. The fused NDVI and actual Sentinel-2
NDVI image are visually similar, even at the junctions of
3.4. Wetland vegetation classification complex land covers, such as along the river bench, near

The object-based RF method was performed with the

optimum feature combination obtained from the section
mentioned above. The features combination was first seg-
mented using the multi-scale resolution and the ESP2 was
used to evaluate the optimal segmentation scale parame-
ters. The experimental results showed that the optimal seg-
mentation parameters are 10 (scale), 0.2 (shape) and 0.5
(compactness), respectively. Using segmented object
images, a total of 1017 objects were randomly selected on
the basis of Google earth imagery and filed data. The min-
imum distance between any two samples was set as 500 m.
These samples are distributed in six land types, water
(1 3 2), sedge (1 5 2), reed (1 1 4), single season rice
(2 0 1), double cropping rice (1 8 5), Forest (1 2 9), others
(1 0 4) (Fig. 5). Two-thirds of the 1017 samples were
assigned for training, and the remaining were used for
validation.
A Random Forest classifier (RF) was employed, which
has better performance than other machine learning algo-
rithm for remote sensing datasets (Ma et al., 2017; Belgiu
and Csillik, 2018; Silveira et al., 2019). An optimizer was
utilized to achieve the best parameters (500 trees, 3 maxi-
mum features) and highest out-of-bag score (>0.9). The
Fig. 5. The distribution of samples, true color composites of Sentinel-2
classification accuracy was assessed with a confusion data: red: band 4; green: band 3; blue: band 2. (For interpretation of the
matrix, which calculates the overall accuracy and Kappa references to color in this figure legend, the reader is referred to the web
coefficient as well as the producer accuracy (PA) and user version of this article.)
2238 Y. Cai et al. / Advances in Space Research 64 (2019) 2233–2244

Fig. 6. (a) The MODIS NDVI of DOY (Day of year) 273, (b) the fused NDVI generated by the RASTFM model of DOY 273, and (c) the actual Sentinel-
2 NDVI of DOY 278.

the lake shore, and in some sparse vegetation areas. The growing peaks, which are on DOY145 and DOY241. The
accuracy assessment results show that the fused image single season rice grows fast before DOY209, then the
derived from RASTFM has a high correlation with the NDVI plummets in the rest growing period. The NDVI
actual Sentine-2 image (R2 = 0.934, RMSE = 0.0729) values of reed grow relatively quick during DOY1 and
(Fig. 7). Therefore, the accuracies of the fused NDVI DOY113, but the growth becomes slow and stable from
images are reliable for the identification of paddy rice. DOY129 to DOY257. As for forest, the NDVI value shows
Then, we used the RASTFM algorithm to predict time ser- no obvious change. The NDVI time series of different veg-
ies Sentinel-2-like images (16 d interval and 10 m etation have distinctive features, especially during critical
resolution). growth stages, so they can be used to identify vegetation
types.
The phenological parameters of different vegetation, as
4.2. The smoothed Sentinel-2 NDVI time series and
well as the NDVI time series, are different during critical
phenology data
growth season. Especially in critical phenological stages,
which have been proved to be able to use for vegetation
As the smoothed Sentinel-2 NDVI time series in Fig. 8
classification (Dymond et al., 2002; Kiptala et al., 2013;
shows, forest has the most stable and longest growing sea-
Bargiel, 2017). Five parameters were extracted to show
son, followed by reed. The NDVI value of sedge grows fast
the phenological differences among different vegetation of
in the period of DOY1 to DOY97, but slides quickly dur-
the whole study area, which may increase the mapping
ing DOY113–DOY209, and on DOY209 reaches the low-
accuracy (Fig. 9). Forest show a long growing season
est point of the year. The double cropping rice has two
(~191 days) with an early start (82th ‘day-of-year’; DOY)
and a late end (DOY273). Reed has a similar start of sea-
son (SOS) and end of season (EOS) with forest. The SOS of
both single and double paddy rice is late (DOY113 and 98,
respectively), but the EOS is early (DOY256 and 263,
respectively). Sedge begins to grow around March (DOY
75) and finishes growing around October (DOY 274).
The SOS and EOS of reed are similar to that of sedge. For-
est has the largest NDVI (0.96), while double cropping rice
has the minimum (0.75). The NDVI values of other vegeta-
tion types are very close. On the contrary, forest has the
minimum seasonal NDVI amplitude, but rice and sedge
have large seasonal NDVI amplitudes.

4.3. Separatability among vegetation types using optimal

feature combination

Fig. 7. Scatterplots between the actual Sentinel-2 NDVI and the fused The JBh distance of different numbers of feature combi-
NDVI images of DOY 278. nations are shown in Fig. 10. We selected 22 features as the
 Y. Cai et al. / Advances in Space Research 64 (2019) 2233–2244 2239

Fig. 8. Smoothed Sentinel-2 NDVI time series.

Fig. 9. Five phenological parameters of the study area (a) SOS, (b) EOS, (c) LOS, (d) MON and (e) AON.

optimal combination for paddy rice classification, because SOS and LOS, and the Sentine-1 (r0VV and r0VH ) on April
additional images have little contribution to separability 12, May 30, July 17, August 10, September 27 were utilized
(i.e., JBh increased by less than 0.2). Considering data to map the paddy rice. The JBh was 9.82 when use the opti-
redundancy and computing time, the Sentinel-2 MSI on mum feature combination, the JM distances among the
February 12, April 18, October 07, November 13, fused vegetation types were above 1.9 (Table 3). Because the for-
Sentinel-2 NDVI on April 7, May 25, June 10, July 12, est phenological characteristics are different from other
August 13, September 30, October 16, phenology data of vegetation, the JM distance between forest and other
2240 Y. Cai et al. / Advances in Space Research 64 (2019) 2233–2244

Fig. 10. JBh distance of different feature combinations.

vegetation types does not need to calculate. The results

show that using the optimum feature combination could
achieve a good separability among the vegetation types.

4.4. Classification results

The classification results are shown in Fig. 11. The sedge

and most part of reed are the natural wetland vegetation,
which strongly depends on water resources. Therefore, Fig. 11. Dongting Lake wetland vegetation map.
the sedge and reed dominate the area around the Dongting
Lake. The paddy fields are man-made wetlands, which also
need a lot of water and can be irrigated artificially. Thus, based method. When use the same classifiers, McNemar’s
they are always distributed sporadically around the lake results showed that the difference of NN, BA, CART and
or along rivers. Rice and reed are the dominant wetland SVM was not significant (p > 0.05) (Table 5), but it was
vegetation types, covering 1306 km2 (26.4%) and statistically significant (p < 0.05) when utilize the RF
1152 km2 (23.3%) of the study area, respectively. Sedge (p = 0.0015), and the variation of the overall accuracy
and forest cover 375.6 km2 (7.6%) and 238.6 km2 (4.8%), between the pixel-based and object-based methods using
respectively. Furthermore, the double cropping rice the RF classifier was nearly 5%. For the object-based
(870.6 km2) has a higher coverage than single season rice method, only RF and NN, RF and MLC, RF and SVM
(435.4 km2). have statistically significant difference, with p = 0.0082,
To assess the classification accuracy of wetland vegeta- p = 0.0316 and p = 0.0075, respectively (Table 6). And
tion mapping of the object-based RF method, we used the differences of the overall accuracy between RF and
other object-based and pixel-based classifiers. The classifi- NN, RF and MLC, RF and SVM are nearly 10%, 7%
cation accuracies shown in Table 4 demonstrate that and 6%, respectively.
object-based classifiers have higher accuracy (including In order to assess the feasibility of combinations of phe-
overall accuracy, Kappa coefficient, user accuracy) than nology data and Sentinel-1 data in paddy rice mapping,
pixel-based classifiers, except the NN and SVM classifier. classifications with different feature combinations and
For the object-based method, the overall accuracy and object-based RF method were performed. The classifica-
Kappa coefficient are above 85% and 0.8, respectively, tion accuries are shown in Table 7. When NDVI and phe-
and the user accuracy wetland vegetation types are above nology data are added, the overall accuracy, Kappa
80%. Using the object-based classifier, the random forest coefficient, user and producer accuracy all increase, demon-
(RF) achieved the highest overall accuracy (95.26) and strating the potential of phenology data in paddy mapping.
Kappa coefficient (0.93), as well as the user and producer In addition, the Sentinel-1 SAR data is useful for improv-
accuracy of paddy rice. ing the paddy classification, as the classification accuacy
The McNemar’s test was applied to compare the classi- reaches the maximum (95.26%, 0.93) when the Sentinel-2
fication results of the object-based method and the pixel- MSI, NDVI, phenology data and Sentinel-1 are used.

Table 3
J-M distance among the four vegetation types on different combination of Landsat NDVI data.
JBh J-M distance
All classes Sedge and Double Sedge and Single Reed and Double Reed and single Double cropping rice and Reed and
cropping rice season rice cropping rice season rice single season rice Sedge
9.826 2.000 2.000 2.000 1.989 1.994 1.982
 Y. Cai et al. / Advances in Space Research 64 (2019) 2233–2244 2241

Table 4
Classification accuracies of the two methods with several classifiers.
Overall accuracy Kappa coefficient UA (DCR) PA (DCR) UA (SSR) PA (SSR)
Pixel-based method
NN 85.48% 0.84 88.42% 86.93% 79.25% 86.48%
MLC 87.26% 0.86 88.26% 84.69% 81.33% 87.56%
CART 90.15% 0.88 90.65% 90.15% 85.74% 91.28%
SVM 89.23% 0.88 90.36%. 87.24% 85.26% 90.85%
RF 90.64% 0.89 93.42% 91.63% 88.65% 94.63%
Object-based method
NN 85.24% 0.84 88.68% 86.24% 78.32% 86.52%
MLC 88.35% 0.85 88.67% 88.38% 83.52% 89.46%
CART 91.63% 0.90 92.54% 91.33% 87.45% 93.62%
SVM 89.19% 0.88 90.15% 88.26% 85.64% 90.49%
RF 95.26% 0.93 96.76% 95.58% 92.18% 96.85%
*The SSR and DCR represent single season rice and double cropping rice, respectively.

Table 5 5. Discussion
Statistically significant comparison between pixel-based and object-based
methods. The Dongting Lake area is a plain formed by the impact
Comparison methods McNemar’s p-Value of sediments from the Yangtze River, and it is also an
chi-Squared important natural wetland in China. In the past50 years,
NN Pixel-based vs. NN Object-based 1.8726 0.2583 a major part of the natural wetland has been converted
MLC Pixel-based vs. MLC Object-based 2.3847 0.0864 to farmland (paddy fields) or other artificial land due to
CART Pixel-based vs. CART Object-based 3.7912 0.0632
SVM Pixel-based vs. SVM Object-based 2.9358 0.5247
human activities (Wang et al., 2014; Zhang et al., 2018).
RF Pixel-based vs. RF Object-based 9.0000* 0.0015* Recently, with the implementation of ecological protection
policies, some paddy fields gradually become natural wet-
lands. Thus, the fragmentation of vegetation patches
Table 6 becomes increasingly serious. Monitoring the paddy fields
Statistically significant comparison between object-based classifiers. change in the Dongting Lake area is of great significance
Comparison methods McNemar’s p-Value to regional ecological environment and management.
chi-Squared Many studies have employed MODIS, Landsat and GF
NN Object-based vs. MLC Object-based 0.2748 0.5643 (16 m) to get the paddy information (Zhang and Lin,
NN Object-based vs. CART Object-based 1.8852 0.0895 2018). Paddy maps generated from Sentinel-2 and
NN Object-based vs. SVM Object-based 1.7561 0.1679
Sentinel-1 data have a higher resolution and more detailed
NN Object-based vs. RF Object-based 6.4738* 0.0082*
MLC Object-based vs. CART Object-based 1.8257 0.1746 spatial information. Especially, they can provide many
MLC Object-based vs. SVM Object-based 1.8645 0.1680 details in a complex heterogeneous region with different
MLC Object-based vs. RF Object-based 5.7825* 0.0316* vegetation types. On these maps, the fragmented paddy
CART Object-based vs. SVM Object-based 3.3533 0.0645 in the Dongting Lake area can be more accurately and
CART Object-based vs. RF Object-based 2.6253 0.0898
easily separated from other vegetation types.
SVM Object-based vs. RF Object-based 7.2537* 0.0068*
Using time series optical images to identify paddy rice
The symbol ‘‘*” in Tables 5 and 6 indicates that the differences is statis-
accurately is an effective way in some regions. However,
tically significant.
it is difficult to obtain enough clear optical images with

Table 7
Classification accuracies with different feature combinations.
Feature sequences Class Producer accuracy User accuracy Overall accuracy Kappa coefficient
Only Sentinel-2 MIS Paddy rice 87.15% 83.52% 84.28% 0.75
Non rice 86.34% 87.36%
Sentinel-2 MSI + NDVI Paddy rice 88.69% 85.74% 86.53% 0.81
Non rice 90.25% 89.64%
Sentinel-2 MSI + NDVI + phenology Paddy rice 92.36% 88.72% 90.12% 0.88
Non rice 91.83% 91.66%
Sentinel-2 MSI + NDVI + phenology + Sentinel-1 Paddy rice 95.62% 94.89% 95.26% 0.93
Non rice 97.58% 96.37%
2242 Y. Cai et al. / Advances in Space Research 64 (2019) 2233–2244

high spatial and temporal resolutions in rainy and cloudy fication, the contribution of the band reflectance in
days. Spatial and temporal fusion model is a feasible way Sentinel-2 and Sentinel-1 data to the FVC estimation need
to generate images with high spatial and temporal resolu- further research. Additionally, the RF was chosen for
tion, and we employed RASTFM to get Sentinel-2 time paddy identification. Convolutional Neural Network
series with high accuracy in this study. Although optical (CNN), as one of the most popular algorithms in the
multispectral data have higher classification accuracy, but extended deep learning, has been widely used in remote
is insufficient to distinguish wetland vegetations that are sensing image classification and has achieved good results
typically composed of multiple vegetations which have sim- (Zhang et al., 2018). Therefore, the CNN algorithm may
ilar spectral characteristics. Many studies have shown that increase the paddy rice mapping accuracy using the time
SAR data can replace optical images with the same resolu- series remote sensing data with high spatial resolutions.
tion in high cloud, mist and smoke areas because of its
immunity to weather conditions (Erinjery et al., 2018). Acknowledgement
Recent studies have shown that combining optical and
SAR data can improve the extraction accuracy of land The authors would like to thank the editors and anony-
cover information in heavily heterogeneous areas (Veloso mous reviewers for the valuable comments, which are sig-
et al., 2018; Whyte et al., 2018). nificant for improving this manuscript. This Work was
Both object-based and pixel-based methods were used to supported in part by the National Natural Science Founda-
identify paddy rice in this study, but the classification accu- tion of China (41901385), and in part by the China Post-
racy has been improved by object-based classifiers. Paddy doctoral Science Foundation (2019M652815).
fields are severely fragmented due to the persistent impacts
of climate change and anthropogenic factor. Recently, the
object-based image analysis (OBIA) has been widely and References
successfully applied in many fields, which can reduce the
Bargiel, D., 2017. A new method for crop classification combining time
effect of neighboring pixels and the ‘‘salt and pepper” phe- series of radar images and crop phenology information. Rem. Sens.
nomena by contexture procedures and integrating neigh- Environ. 198, 369–383. https://doi.org/10.1016/j.rse.2017.06.022.
borhood information. Some researches also have Belgiu, M., Csillik, O., 2018. Sentinel-2 cropland mapping using pixel-
demonstrated the advantages of the object-based classifica- based and object-based time-weighted dynamic time warping analysis.
tion method over the pixel-based method when using high Rem. Sens. Environ. 204, 509–523. https://doi.org/10.1016/j.
rse.2017.10.005.
spatial resolution remote sensing images. In this study, the Bullock, E.L., Woodcock, C.E., Holden, C.E., 2019. Improved change
accuracy of double cropping rice was higher than that of monitoring using an ensemble of time series algorithms. Remote Sens.
single season rice, because a lot of single season rice fields Environ. 111165. https://doi.org/10.1016/j.rse.2019.04.018.
were smaller than the Sentinel-2 image pixel and caused Dobson, M.C., Pierce, L., Sarabandi, K., Urbay, F.T., Sharick, T., 1992.
mixed-pixel (mixed-object) problem. Preliminary analysis of ERS-1 SAR for forest ecosystem studies. IEEE
Trans. Geos. Rem Sens. 30, 203–211. https://doi.org/10.1109/
The proposed paddy rice mapping method is suitable for 36.134071.
regions with a lot of rainy and cloudy weathers. However, Dymond, Caren C., Mladenoff, David J., Radeloff, Volker C., 2002.
there are some factors affecting the classification accuracy. Phenological differences in Tasseled Cap indices improve deciduous
First, the residual could contaminate the MODIS data, and forest classification. Rem. Sens. Environ. 80, 460–472. https://doi.org/
reduce the accuracy of the fused Sentinel-2 time series. Sec- 10.1016/S0034-4257(01)00324-8.
Dao, P.D., Liou, Y.A., 2015. Object-based flood mapping and affected rice
ond, the mixed pixel problem also reduces the accuracy. field estimation with landsat 8 OLI and MODIS data. Rem. Sens. 7,
Also, the favorable hydrothermal condition of the Dongt- 5077–5097. https://doi.org/10.3390/rs70505077.
ing Lake area makes various kinds of vegetation grow, Dong, J.W., Xiao, X.M., Kou, W.L., Qin, Y.W., Zhang, G.L., Li, L., Jin,
which leads to high spatial heterogeneity and mixed pixels. C., Zhou, Y.T., Wang, J., Biradar, C., Liu, J.Y., Moore III, B., 2015.
Tracking the dynamics of paddy rice planting area in 1986–2010
through time series Landsat images and phenology-based algorithms.
6. Conclusions and future work Rem. Sens. Environ. 160, 99–113. https://doi.org/10.1016/j.
rse.2015.01.004.
In this study, the object-based RF method and time ser- Dong, J.W., Xiao, X.M., 2016. Evolution of regional to global paddy rice
ies Sentinel-2 and Sentinel-1 data were utilized to get paddy mapping methods: a review. Isprs. J. Photogramm. 119, 214–227.
rice information in the Dongting Lake region. This study https://doi.org/10.1016/j.isprsjprs.2016.05.010.
Diñeiro, R.V., Pérez, B.A., Sanchez, J.M.L., Sánchez, N., 2019. Multi-
demonstrated the potential for using optical and SAR data temporal dual- and quad-polarimetric synthetic aperture radar data
combined with object-based RF for paddy rice identifica- for crop-type mapping. Rem. Sens. 11, 1518–1537. https://doi.org/
tion at large scales. Despite some factors affecting the clas- 10.3390/rs11131518.
sification accuracy, the overall accuracy and Kappa Eemlyanova, I.V., McVicar, T.R., Van Niel, T.G., Li, L.T., van Dijk, A.I.
coefficient were higher than 95% and 0.93, respectively, J.M., 2013. Assessing the accuracy of blending Landsat-MODIS
surface reflectances in two landscapes with contrasting spatial and
when using the proposed method. The method proposed temporal dynamics: a framework for algorithm selection. Rem. Sens.
in this paper can provide technology support for rice map- Environ. 133, 93–209. https://doi.org/10.1016/j.rse.2013.02.007.
ping in areas with a lot of cloudy and rainy weathers. We Erinjery, J.J., Singh, M., Kent, R., 2018. Mapping and assessment of
firstly selected the optimum feature combination for classi- vegetation types in the tropical rainforests of the Western Ghats using
 Y. Cai et al. / Advances in Space Research 64 (2019) 2233–2244 2243

multispectral Sentinel-2 and SAR Sentinel-1 satellite imagery. Rem. analysis of time series Landsat 8 (OLI), Landsat 7 (ETM+) and
Sens. Environ. 216, 345–354. https://doi.org/10.1016/j.rse.2018.07.006. MODIS imagery. Isprs. J. Photogramme 105, 220–233. https://doi.
European Space Agency. Sen2cor–version 2.5.5. 2018. Available online: org/10.1016/j.isprsjprs.2015.04.008.
http://step.esa.int/main/third-party-plugins-2/sen2cor/. San, Miguel-Ayanz J., Biging, G.S., 1997. Comparison of single-stage and
Gumma, M.K., Thenkabail, P.S., Maunahan, A., Lslam, S., Nelson, A., mutli-stage classification approaches for cover type mapping with TM
2014. Mapping seasonal rice cropland extent and area in the high and SPOT data. Rem. Sens. Environ. 59, 92–104. https://doi.org/
cropping intensity environment of Bangladesh using MODIS 500m 10.1016/S0034-4257(96)00109-5.
data for the year 2010. Isprs. J. Photogramm. 91, 98–113. https://doi. Shi, J., Huang, J., 2015. Monitoring spatio-temporal distribution of rice
org/10.1016/j.isprsjprs.2014.02.007. planting area in the Yangtze River Delta region using MODIS images.
Hiker, H., Wulder, M.A., Coops, N.C., Seitz, N., White, J.C., Gao, F., Rem. Sens. 7, 8883–8905. https://doi.org/10.3390/rs70708883.
Masek, J.G., Stenhouse, G., 2009. Generation of dense time series Schubert, A., Small, D., Miranda, N., Geudtner, D., Meier, E., 2015.
synthetic Landsat data through dada blending with MODIS using a Sentinel-1A product geolocation accuracy: commissioning phase
spatial and temporal adaptive reflectance fusion model. Remote Sens. results. Remote Sens. 7, 9431–9449. https://doi.org/10.3390/
Environ. 113, 1988–1999. https://doi.org/10.1016/j.rse.2009.05.011. rs70709431.
Hakdaoui, S., Emran, A., Pradhan, B., Lee, C.W., Nguemhe Fils, S.C., Silveira, E.M.O., G. Silva, S.H., W. Acerbijunior, F., Carvalho, M.C., T.
2019. A collaborative change detection approach on multi-sensor Carvalho, L.M., S. Scolforo, J.R., A. Wulder, M., 2019. Object-based
spatial imagery for desert wetland monitoring after a flash flood in random forest modelling of aboveground forest biomass outperforms
southern Morocco. Rem. Sens. 11, 1042–1067. https://doi.org/ a pixel-based approach in a heterogeneous and mountain tropical
10.3390/rs11091042. environment. Rem. Sens. Environ. 78, 175–188.
Jönsson, P., Eklundh, L., 2004. A program for analyzingtime-series of Teluguntla, P., Ryu, D., George, B., Walker, J.P., 2015. Mapping flooded
satellite sensor data. Comput. Geosci. 30, 833–845. https://doi.org/ rice paddies using time series of MODIS imagery in the Krishna River
10.1016/j.cageo.2004.05.006. Basin. India. Rem. Sens. 7, 8858–8882. https://doi.org/10.3390/
Jia, K., Liang, S., Zhang, N., Wei, X., Gu, X., Zhao, X., Yao, Y., Xie, X., rs70708858.
2014. Land cover classification of finer resolution remote sensing data Tricht, V.K., Gobin, A., Gilliams, S., Piccard, L., 2018. Synergistic use of
integrating temporal features from time series coarser resolution data. radar Sentinel-1 and optical Sentinel-2 imagery for crop mapping: a
Isprs. J. Photogramm. 93, 49–55 http://doi.rog/10.1016/j.isprsjprs. case study for Belgium. Rem. Sens. 10, 1642–1663. https://doi.org/
2014.04.004. 10.3390/rs10101642.
Jin, X.L., Li, Z.H., Yang, G.J., Yang, H., Feng, H.K., Xu, X.G., Wang, J. Van Niel, T.G., McVicar, T.R., Datt, B., 2005. On the relationship
H., Li, X.C., Luo, J.H., 2017. Winter wheat yield estimation based on between training sample size and data dimensionality: Monte Carlo
multi-source medium resolution optical and radar imaging data and analysis of broadband multi-temporal classification. Rem. Sens.
the AquaCrop model using the particle swarm optimization algorithm. Environ. 98, 468–480. https://doi.org/10.1016/j.rse.2005.08.011.
Isprs. J. Photogramm. 126, 24–37. https://doi.org/10.1016/j. Veloso, A., Mermoz, S., Bouvet, A., Toan, T.L., Planells, M., Dejoux, J.
isprsjprs.2017.02.001. F., Ceschia, E., 2017. Understanding the temporal behavior of crops
Kennedy, R.E., Yang, Z., Cohen, W.B., 2010. Detecting trends in forest using Sentinel-1 and Sentinel-2-like data for agricultural applications.
disturbance and recovery using yearly Landsat time series: 1. Land- Rem. Sens. Environ. 199, 415–426. https://doi.org/10.1016/j.
Trendr – temporal segmentation algorithms. Remote Sens. Environ. rse.2017.07.015.
114, 2897–2910. https://doi.org/10.1016/j.rse.2010.07.008. Wu, M.Q., Niu, Z., Wang, C., Wu, C., Wang, L., 2012. Use of MODIS
Kiptala, J.K., Mohamed, Y., Mul, M.L., Cheema, M.J.M., Zaag, P., Van and Landsat time series data to generate high-resolution temporal
der, 2013. Land use and land cover classification using phenological synthetic Landsat data using a spatial and temporal reflectance fusion
variability from MODIS vegetation in the Upper Pangani River Basin, model. J. Appl. Rem. Sens. 6, 063507. https://doi.org/10.1117/1.
Eastern Africa. Phys. Chem. Earth 66, 112–122. https://doi.org/ JRS.6.063507.
10.1016/j.pce.2013.08.002. Walker, J.J., Beurs, K.M., Wynne, R.H., Gao, F., 2012. Evaluation of
Kontgis, C., Schneider, A., Mutlu, O., 2015. Mapping rice paddy extent Landsat and MODIS data fusion products for analysis of dryland
and intensification in the Vietnamese Mekong River Delta with dense forest phenology. Rem. Sens. Environ. 117, 381–393. https://doi.org/
time stacks of Landsat data. Rem. Sens. Environ. 169, 255–269. 10.1016/j.rse.2011.10.014.
https://doi.org/10.1016/j.rse.2015.08.004. Wang, Z., Ye, T., Li, J.B., Li, M., Meng, Y.C., 2014. Drought disaster risk
Luis, J., Debaecker, V., Pflug, B., Main-Knorn, M., Bieniarz, J., Mueller- assessment of paddy rice in the northern area of Dongting Lake region.
Wilm, U., Cadau, E., Gascon, F. Sentinel-2 Sen2cor: L2A processor J. Natural Disasters 23, 168–175 https://doi.org/10.13577/j.jnd.2014.
for users. In: Proceedings of the Living Planet Symposium 2016, 0621.
Prague, Czech Republic, 9–13 May 2016. Wang, Q., Atkinson, P.M., 2017. Spatio-temporal fusion for daily
Ma, L., Li, M.C., Ma, X.X., Cheng, L., Du, P.J., Liu, Y.X., 2017. A Sentinel-2 images. Rem. Sens. Environ. 204, 65–80. https://doi.org/
review of supervised object-based land-cover image classification. 10.1016/j.rse.2017.10.046.
Rem. Sens. Environ. 130, 277–293. https://doi.org/10.1016/j. Whyte, A., Ferentinos, K.P., Petropoulos, G.P., 2018. A new synergistic
isprsjprs.2017.06.001. approach for monitoring wetlands using Sentinels -1 and 2 data with
Murakami, T., Ogawa, S., Ishitsuka, N., Kumagai, K., Saito, G., 2001. object-based machine learning algorithms. Remote Sens. Environ.
Crop discrimination with multitemporal SPOT/HRV data in the Saga https://doi.org/10.1016/j.envsoft.2018.01.023.
Plains. Japan. Int. J. Rem. Sens. 22, 1335–1348. https://doi.org/ Xiao, X.M., Boles, S., Liu, J.Y., Zhuang, D.F., Frolking, S., Li, C.S.,
10.1080/01431160151144378. Salas, W., Moore III, B., 2005. Mapping paddy rice agriculture in
Pittman, K., Hansen, M.C., Becker-Reshef, I., Potapov, P.V., Justice, C. southern China using multi-temporal Modis images. Remote Sens.
O., 2010. Estimating global cropland extent with multi-year MODIS Environ. 95, 480–492. https://doi.org/10.1016/j.rse.2004.12.009.
data. Rem. Sens. 2, 1844–1863. https://doi.org/10.3390/rs2071844. Yeom, J.M., Kim, H.O., 2015. Comparison of NDVIs from GOCI and
Peng, D., Huete, A.R., Huang, J., Wang, F.M., Sun, H.S., 2011. Detection MODIS data towards improved assessment of crop temporal dynam-
and estimation of mixed paddy rice cropping patterns with MODIS ics in the case of paddy rice. Rem. Sens. 7, 11326–11343. https://doi.
data. Int. J. Appl. Earth Obs. Geoinf. 13, 13–23. https://doi.org/ org/10.3390/rs70911326.
10.1016/j.jag.2010.06.001. Zhang, M., Zeng, Y.N., Zhu, Y.S., 2017. Wetland mapping of Donting
Qin, Y.W., Xiao, X.M., Dong, J.W., Zhou, Y.T., Zhu, Z., Zhang, G.L., Lake Basin based on time-series MODIS data and object-oriented
Du, G.M., Jin, C., Kou, W.L., Wang, J., Li, X.P., 2015. Mapping method. J. Rem. Sens. 21, 479–492 https://doi.org/10.11834/jrs.
paddy rice planting area in cold temperate climate region through 20176129.
2244 Y. Cai et al. / Advances in Space Research 64 (2019) 2233–2244

Zhang, G., Xiao, X., Dong, J., Kou, W., Jin, C., Qin, Y., Zhou, Y., Wang, Remote Sens. Environ. 208, 42–62. https://doi.org/10.1016/j.
J., Michael, A.M., Chandrashekhar, B., 2015. Mapping paddy rice rse.2018.02.009.
planting areas through time series analysis of MODIS land surface Zhang, C., Sargent, L., Pan, X., Li, H., Gardiner, A., Hare, J., Atkinson,
temperature and vegetation index data. Isprs. J. Photogramm. 106, P.M., 2018a. An object-based convolutional neural network (OCNN)
157–171. https://doi.org/10.1016/j.isprsjprs.2015.05.011. for urban land use classification. Rem. Sens. Environ. 216, 57–70.
Zhang, M., Lin, H., 2018. Object-based rice mapping using time-series and https://doi.org/10.1016/j.rse.2018.06.034.
phenological data. Adv. Space Res. 63, 190–202. https://doi.org/ Zhang, M., Lin, H., Wang, G., Sun, H., Fu, J., 2018b. Mapping paddy
10.1016/j.asr.2018.09.018. rice using a Convolutional Neural Network (CNN) with Landsat 8
Zhao, Y.Q., Huang, B., Song, H.H., 2018. A robust adaptive spatial and datasets in the Dongting Lake Area. China. Rem. Sens. 10, 1840–1861.
temporal image fusion model for complex land surface changes. https://doi.org/10.3390/rs10111840.

View publication stats