Asia Pacific Journal of Tourism Research

ISSN: 1094-1665 (Print) 1741-6507 (Online) Journal homepage: https://www.tandfonline.com/loi/rapt20

A recommender system based on personal

constraints for smart tourism city

Il Young Choi, Young U. Ryu & Jae Kyeong Kim

To cite this article: Il Young Choi, Young U. Ryu & Jae Kyeong Kim (2019): A recommender
system based on personal constraints for smart tourism city, Asia Pacific Journal of Tourism
Research, DOI: 10.1080/10941665.2019.1592765

To link to this article: https://doi.org/10.1080/10941665.2019.1592765

Published online: 16 Mar 2019.

Submit your article to this journal

Article views: 24

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=rapt20
ASIA PACIFIC JOURNAL OF TOURISM RESEARCH
https://doi.org/10.1080/10941665.2019.1592765

A recommender system based on personal constraints for smart

tourism city*
Il Young Choia, Young U. Ryub and Jae Kyeong Kima
a
School of Management, Kyung Hee University, Dongdaemun-gu, Seoul, Korea; bSchool of Management, University of Texas at
Dallas, Richardson, TX, USA

ABSTRACT KEYWORDS
In a smart tourism ecosystem, travel communication websites play a critical role in Smart tourism; smart tourism
choosing destinations and hotels. This research suggests a travel recommender city; recommender system;
system for automating word-of-mouth (WOM) effects and providing personalized travel recommender system;
travel-planning services to tourists. Collaborative filtering (CF)-based recommender travel planning; travel-
planning service; travel
systems have been extensively employed for personalization services in diverse package; approximate
areas; the basic principle of CF is WOM communication. This research proposes a constraint satisfaction;
travel recommender system that helps a tourist build his/her personalized travel constraint satisfaction; word-
plan based on CF and constraint satisfaction filtering. Constraint satisfaction filtering of-mouth communication
is adopted to profile a tourist’s needs and circumstances. For this purpose, this
research modifies the existing constraint satisfaction method to an approximate
constraint satisfaction filtering method that incorporates indifference intervals into
constraints. We build a prototype system and a benchmark system to evaluate the
effectiveness, usability, and novelty of the proposed travel recommender system.
The experimental results demonstrate a methodology for performing personalized
tourist’s travel planning and automating WOM communication outperforms the
benchmark system.

1. Introduction 2016; Schuckert, Liu, & Law, 2015). Tourists have

exchanged travel information, advice, and tips on
Due to the rapid growth of smart tourism technol- travel community. With the information and knowl-
ogies, such as cloud computing, big data, Internet of edge gotten from travel community, tourists can
things applications, and social networking services plan their personalized travels by not only saving
recently, the concept of a smart tourism city efforts, time and money but also gaining more attrac-
emerges as a means to provide value-added services tive travel options (Litvin, Goldsmith, & Pan, 2008;
to tourists (Gretzel, Sigala, Xiang, & Koo, 2015; Rabanser & Ricci, 2005).
Gretzel, Zhong, & Koo, 2016). A smart tourism city However, tourists often make complex decisions and
can enhance tourists’ experiences by analyzing travel substantially spend their time planning their travels
patterns and offering personalized attractions. Conse- because their travel plans need to consider their desti-
quently, the satisfaction of tourists increases (Lee, Lee, nation, flights, hotels, travel packages, and their time
Chung, & Koo, 2018). and money constraints. Abundant information about
In a smart tourism ecosystem, travel community travel community websites complicates the decision-
such as TripAdvisor have a critical role not only in making of tourists. To solve these problems and
choosing tourism destinations (Kim & Canina, 2015; provide tourists with better services, travel recommen-
Schuckert, Liu, & Law, 2016) but also as hubs for der systems have been introduced by major travel
travel planning (Cox, Burgess, Sellitto, & Buultjens, agencies (e.g. http://www.travelocity.com). Generally, a
2009; Liu, Schuckert, & Law, 2015; Schuckert et al., travel recommender system attempts to offer more

CONTACT Jae Kyeong Kim [email protected] School of Management, Kyung Hee University, 1 Hoegi-dong, Dongdaemun-gu, Seoul, Korea
*Portions of this paper were previously presented in the 48th Hawaii International Conference on System Sciences, 2015.
© 2019 Asia Pacific Tourism Association
2 I. Y. CHOI ET AL.

customized travel-planning services by analyzing tour- product instead of a single product. A composite
ists’ past travel histories. However, the system has a ten- product indicates that travel planning includes the
dency to imitate a travel agent’s role and does not simultaneous selection of a destination, hotel, and
consider the word-of-mouth (WOM) communication attraction. Thus, an existing CF-based recommender
effect (Rabanser & Ricci, 2005). WOM communication system is not more useful for a travel-planning
has an important role in travel decision-making service. Second, travel planning is a very complicated
(Baloglu, Pekcan, Chen, & Santos, 2004; Cai, Feng, & process that needs many constraints, such as traveling
Breiter, 2004; Gretzel & Yoo, 2008; Hernández-Méndez, period, duration, and travel cost. A constraint satisfac-
Muñoz-Leiva, & Sánchez-Fernández, 2015; Li, Tung, & tion filtering process is needed in existing CF-based
Law, 2017; Murphy, Mascardo, & Benckendorff, 2007; recommender systems. This research models the
Tung & Ritchie, 2011). WOM communication has a posi- requirements of each tourist as an approximate con-
tive impact on the destination image and destination straint satisfaction problem. Because the constraint
choice (Jalilvand, 2017; Jalilvand, Samiei, Dini, & satisfaction method is often too strict, a minor differ-
Manzari, 2012; Tham, Croy, & Mair, 2013). Online consu- ence may have a role in changing a critical decision
mer reviews based on WOM communication have (Choi, Kim, & Ryu, 2015; Ryu, 1998). Therefore, this
helped consumers to make difficult and complex pur- research proposes an approximate constraint satisfac-
chase decisions (Bae & Lee, 2011; Park, Lee, & Han, 2007). tion-based methodology by adopting indifference
Collaborative filtering (CF) recommends items to intervals to derive a flexible and robust result. A tour-
target user based on the purchase histories of neigh- ist’s personal requirements are modeled by con-
bors who have purchased almost the same items as straints, and an approximate constraint satisfaction
target users. Thus, CF operates in a manner similar method automates tourists’ decision processes for
to WOM communication. CF has been utilized as a choosing their personalized travel packages to
representative methodology in recommender enhance tourists’ convenience and satisfaction. In
systems over the last 40 years (Adomavicius & Tuzhilin, this research, constraint satisfaction is approximated
2005; Herlocker, Konstan, & Reidl, 2000; Sarwar et al., by the following two ways: First, if tourist’s constraint
1998). CF and its extended hybrid methodology for travel planning is not satisfied, the degree or
have been extensively employed for recommending amount of dissatisfaction is used instead of deleting
products (books, clothes, and shoes) and contents the candidate tourist’s package. Second, in the case
(movies and music) of almost all areas (Park, Kim, of multiple constraints, the constraints are ordered
Choi, & Kim, 2012). However, applying existing CF- according to their important factor, and the satisfying
based recommender systems to travel-planning ser- value of a more important factor is used to select a
vices, such as selecting traveling destination, hotels, better tourist package.
and flights has many drawbacks. The underlying In this research, we propose a CF-based travel
basic principle of CF recommender systems is to deter- recommender system coupled with an approximate
mine a target user’s preference based on the prefer- constraint satisfaction process for automating WOM
ence of similar neighbor users (Kim, Kim, & Ryu, communication and offering personalized travel-plan-
2009). Thus, recommending items based on the past ning services. For a web-based travel recommender
purchase histories of the target user and other users system, a prototype is developed, and a benchmark
has produced excellent results in the online market, system is also developed for comparison with our pro-
such as movies, booth, music, books, images, online posed system. We conduct a survey to evaluate tour-
advertisements, and grocery (Choi, Oh, Kim, & Ryu, ists’ perceptions of the recommendation results of
2016; Kim, Moon, An, & Choi, 2018; Moon, Kim, & our prototype system. Our experimental findings are
Ryu, 2013; Park et al., 2012). CF-based recommender summarized in the results.
systems for group users have been well developed
(Garcia, Sebastia, & Onaindia, 2011; Kim, Kim, Oh, &
Ryu, 2010; Kim, Oh, Gu, & Kim, 2011). 2. Literature review
However, two challenges exist for adapting a
2.1. Travel recommender systems
travel-planning service in CF-based systems due to
the difference between the characteristics of a As the development of economic growth, there has
travel-planning service and books or music. First, a been a cultural and social revolution in the tourism
travel-planning service consists of a composite industry. To survive in the escalated competition,
 ASIA PACIFIC JOURNAL OF TOURISM RESEARCH 3

travel agencies have provided extensive travel products purchased by a set of users with a high degree of simi-
or services. For example, Travelocity offers more than larity among the users, known as neighbors. CF has
one million flights, hotels, and packages. However, been introduced as an important method for main-
tourists experience difficulty in looking for the necess- taining a sustainable competitive advantage by Inter-
ary information even though the amount of travel infor- net leaders such as Amazon (Linden, Smith, & York,
mation is substantially increased. Tourists who need 2003), Google (Das, Datar, Garg, & Rajaram, 2007)
information want to receive more smart services and Netflix (Bennett & Lanning, 2007). Thus, CF is
when considering their preferences. One of the tech- known to be a commercially successful recommen-
niques to solve this problem is to use the travel recom- dation technique.
mender systems, which help tourists find travel In general, a CF system employs three phases, as
products and services that fulfill their preferences. shown in Figure 1. The first phase creates a user
Some researchers focus on travel recommender profile using purchase history data. The second phase
systems to find personalized tourist attractions. forms a set of users, which are known as neighbors, by
Huang and Bian (2009) have proposed an intelligent comparing the degree of similarity among the users.
travel system to estimate a tourist’s travel preferences The final phase generates a top-N recommendation list
based on a Bayesian network, and prioritize his/her of items that the target user is likely to purchase.
attractions by the analytic hierarchy process (AHP). However, a CF system poses some issues. The first
Yeh and Cheng (2015) have developed an attraction issue is related to a sparsity problem. Because users pur-
recommender system using a Delphi panel and reper- chase a very small portion of a large item set, the rec-
tory grid techniques. Other studies have concentrated ommendation quality of a CF system based on the k-
on offering personalized hotels (Nilashi, bin Ibrahim, nearest neighbors (kNN) algorithm decreases (Balaba-
Ithnin, & Sarmin, 2015; Schiaffino & Amandi, 2009), novic & Shoham, 1997; Sarwar, Karypis, Konstan, &
flights (Coyle & Cunningham, 2003), destinations Riedl, 2000). The second issue is related to a scalability
(Gavalas, Konstantopoulos, Mastakas, & Pantziou, problem. The kNN algorithm is often very time-consum-
2014), and travel packages (Liu et al., 2014; Lorenzi, ing because a recommender system addresses the user
Bazzan, & Abel, 2007; Lorenzi, Loh, & Abel, 2011) to profile, which consists of millions of users and items. The
tourists. However, previous studies overlook the final issue is related to a new item ramp-up problem. If
effect of WOM communications in the smart city and newly released items do not have any transaction
tourism industry, which has a critical role in selecting records, a CF system cannot recommend them (Choi
travel destinations (Murphy et al., 2007). According et al., 2016). To address these problems, numerous
to TotalMedia (2010), 25% of tourists have participated hybrid recommender systems to incorporate content-
in a social networking site, such as TripAdvisor.com, to based filtering, demographic filtering, and other data
seek tourist information. Approximately 50% of US mining techniques into CF are developed to rec-
tourists have obtained tourist information by com- ommend better items (Balabanovic & Shoham, 1997;
munication with their family, friends, and colleagues. Cho, Kim, & Kim, 2002; Kim, Lee, Cho, & Kim, 2004; Mel-
Travel opinion leaders enjoy sharing their experiences ville, Mooney, & Nagarajan, 2002). These hybrid
with others (Litvin et al., 2008). Therefore, a travel approaches are successful in some applications, but
recommender system that incorporates tourists’ selecting travel products or services that satisfy a tour-
implicit preferences is needed. ist’s requirements among the recommended products
and services remains as the role of the tourist. Thus, to
provide acceptable recommendations, travel recom-
2.2. Collaborative filtering
mender systems prepare a countermeasure for these
Collaborative filtering (CF) introduced by Goldberg drawbacks in the tourism domain.
et al. (1992) has been known to be one of the repre-
sentative models of recommender systems. CF
attempts to automate WOM recommendations 3. CF-based travel recommendation with
received from famil, relatives, friends, colleagues, approximate constraint satisfaction
and other socially related people (Adomavicius & Tuz-
3.1. Overall View
hilin, 2005; Herlocker et al., 2000; Sarwar et al., 1998).
The idea of CF is that a high probability exists that a Many tourists try to get much travel information as
user will purchase items that were frequently possible by interpersonal communication instead of
4 I. Y. CHOI ET AL.

Figure 1. CF procedure.

travel brochures, travel agencies, and social media catalog of the candidate destinations with the target
reviews (Litvin et al., 2008; TotalMedia, 2010). tourist’s requirements (e.g. means of transport, types
Travel community websites such as TripAdvisor are of hotel, and price) are compared. Here, the target
surrounded by a considerable amount of travel tourist’s requirements are named as constraints. And
information. As a result, tourists encounter then the destination and the best set of items that
difficulty in choosing travel packages that satisfy approximately satisfy the tourist’s constraints are rec-
their preferences. Information gathering and proces- ommended to the target tourist.
sing in this way may be time-consuming and the
vast amount of information suggested by many
commercial travel sites causes confusion and
3.2. Phase 1: CF-Based filtering
hinders the ability of tourists to make a decision.
Therefore, we suggest a CF-based recommendation The proposed CF-based filtering is divided into three
methodology that involves the automation of WOM steps. First, a tourist profile indicates a collection of
communication to create a travel plan that is not m tourists’ preference ratings on n destinations.
easily constructed using other tourists’ preferences Thus, the tourist profile is represented in the space
and satisfies a tourist’s implicit preferences using vector of m x n matrices, R = (rij), where i = 1 to m, j
approximate constraint filtering. = 1 to n, rij ∈[1, 5]. Here, m and n are the total
The proposed methodology is composed of the fol- number of tourists and the total number of desti-
lowing two phases in Figure 2. In the first phase, a nations, respectively, and rij is the tourist ith rating
tourist profile on destinations is created and other on the jth destination (1 = “very dissatisfied”,
tourists (called neighbors) who have had a similar pre- 5 = “very satisfied”).
ference for the target tourist’ destination, are ident- Second, the similarity is computed to form the
ified. And then, top-N recommended destinations neighborhood of a target tourist. The Pearson corre-
are generated as candidate destinations. In the lation coefficient is used to compute the similarity
second phase, travel items (e.g. flight and hotel) in a among tourists (Kim et al., 2004; Sarwar et al., 2000).

Figure 2 .#Overall procedure.

 ASIA PACIFIC JOURNAL OF TOURISM RESEARCH 5

Last, the selection likelihood score (SLS) of the target If a constraint contains real numbers (d1 and d2),
tourist a on the jth destination is used to generate the then the error function for the arithmetic constraint is
top-N candidate destinations. Assume that Ka is the
target tourist a’s neighbor set; SLS(a, j) is calculated e(xi = d1 , u) = |u − d1 |
as follows: e(xi ≤ d1 , u) = max(u − d1 , 0)


i[K rij · sim(a, i)
e(xi , d1 , u) = max(u − d1 + 1, 0)
SLS(a, j) =
a , (1)
i[Ka sim(a, i) e(xi ≥ d2 , u) = max(d2 − u, 0)
(2)
e(xi . d2 , u) = max(d2 − u + 1, 0)
where sim(a, i) indicates the similarity between the ⎧
target tourist a and his/her neighbor i. The higher is ⎨ u − d1 if u . d1
⎪
the SLS value, the more likely the target tourist will e(d12 ≤ xi ≤ d1 , u) = d2 − u if u , d2
choose a destination. Therefore, top-N destinations ⎪
⎩
0 otherwise,
with high SLS values are selected as candidate
destinations.
where ε > 0 is an infinitesimally small number.
If the constraint is categorical, the error can be
measured in one of the following two ways: First, if
3.3. Phase 2: approximate constraint-based categorical values can be converted into scalar
filtering values or vectors, such as colors, then the error is cal-
A typical CF-based travel recommender system culated as the scalar difference or the vector distance.
suggests destinations by a comparison between For instance, destination scenery can be mapped into
the target tourist’s profile of destinations and the a vector of affectivity, physical atmosphere, and acces-
neighbors’ profiles of destinations. Even if tourists sibility values. Second, if categorical values cannot be
travel the same destinations, they may differ with converted into scalars or vectors, then a categorical
regard to their likes and dislikes of these desti- constraint is evaluated as a binary constraint. Thus,
nations. Thus, we suggest an additional filtering the evaluation result will be either true (valued 0) or
layer of approximate constraint satisfaction to false (valued 1).
enable Tourists to have more customized travel- Assume that tourist i gives p constraints {ci,1, ci,2, … ,
planning services. Tourists may directly provide ci,p} such that constraint ci,s is more important than
their needs or constraints, such as budget limits, constraint ci,t for s < t, that is, the constraints follow a
types of accommodations, and means of transport, strict linear order. For the constraint ci,k, we define
that are suitable for their travel products. These ≻ci,k by items in the electronic catalog:
tourist constraints are not absolute criteria but
rather are expressions of preference. For instance, u ≻ci,k u′ if and only if e(ci,k , u) , e(ci,k , u′ ). (3)
a small difference (e.g. $30) in the price may be dis-
regarded but a large difference (e.g. $130) may be We define ≈ci,k :
considered to be significant. Tourists may discover
that certain constraints (e.g. flight duration) are u ≈ci,k u′ if and only if e(ci,k , u) = e(ci,k , u′ ). (4)
more important than other constraints (e.g. hotel
ratings). We also assume that trade-offs between With (4) and (5), we obtain the binary relation ≻ of
constraints are not allowed. the preference on items in the electronic catalog:
Tourists’ constraints are then applied to evaluate
the candidate destinations that are available in the u ≻ u′ if and only if for some ci,k , u ≻ci,k u′
(5)
electronic catalog. The evaluation yields the degree and for all ci,j with j , k, u ≈ci,j u′ .
of dissatisfaction between a constraint and an item
(i.e. product) in the electronic catalog. For constraint Tourist i prefers the catalog item θ to the catalog item
ci, for tourist i expressed as a relation such as θ′ if and only if (a) the satisfaction error of ci,k for θ is
“price ≤ 500” or “flight = American Airlines,” the evalu- less than that for θ′ for some constraint ci,k and (b)
ation result e(ci, θ) for the electronic catalog item θ is for all constraints ci,j that are more important than ci,
the error function that returns the satisfaction error, k, the satisfaction error of ci,j for θ is same as that for
i.e. degree of dissatisfaction. θ′ for some constraint ci,j. We also define the ≈ of
6 I. Y. CHOI ET AL.

the indifference in the items in the electronic catalog: 4. An Illustrative example

To understand the proposed method, a simple
u ≈ u′ if and only if both u ≻ u′
(6) example is presented in this section. Assume that
and u′ ≻ u do not hold. the tourist T1 is collecting travel information from an
online travel agency for planning a vacation in
Tourist i is indifferent between θ and θ′ if and only if i Europe or North America. However, he/she has
prefers neither θ nor θ′ . trouble organizing his/her itinerary due to the large
The constraint satisfaction at the right-hand side of number of destinations, hotels, and attractions and
(3) and (4) is often too strict. For instance, assume that numerous concerns, as shown in Table 3.
a tourist has a price constraint of $1,500 or less and We consider the recommendation process for
that travel products θ1, θ2, and θ3 are priced at tourist T1. Assume that eight tourists, ten destinations,
$1,510, $1,511, and $1,600, respectively. Although θ1 a combined total of 22 flights, a combined total of 21
is less expensive than θ2 such that e(price ≤ $1,500, hotels, and a combined total of 21 activities exist for all
θ1) > e(price ≤ $1,500, θ2), the tourist may be indiffer- destinations (refer to Tables A1, A2, A3, and A4 in the
ent between θ1 and θ2 when only prices are con- Appendix). We assume that tourists’ rated the 10
sidered (due to the difference of only $1). However, visited destinations as shown in Table A5 in the
the tourist may consider the prices of θ1 and θ3 to Appendix.
be significantly different. To relax this strictness in
the constraint evaluation, we adopt the concept of
the indifference interval (Luce, 1956). Therefore, we 4.1. Phase 1: CF-Based filtering
have a relaxed version of (3):
Given the tourist profile, CF-based filtering requires
′ ′
u ≻ci,k u if and only if e(ci,k , u ) − e(ci,k , u) the following two steps. In the first step, we apply
the Pearson-r correlation coefficient to identify the
. di,k , (3′ ) neighbors of the target tourist. For the target
tourist T1, the similarity between T1 and the other
where di,k ≥ 0 is i’s indifference interval for the con- neighbors are shown in Table 1. If two neighbors
straint ci,k, that is, θ is recommended if and only if exist, then the neighbors of T1 are T2 and T6, who
the satisfaction error of ci,k for θ′ is significantly are highly similar to T1.
greater than that for θ. Similarly, i is indifferent The SLS value is computed with the destinations
between θ and θ′ in the evaluation with constraint ci, rated by neighbors of T1 to generate candidate desti-
k if and only if the satisfaction error of ci,k for θ and nations (refer to Table 2). Assume that two candidate
θ′ is sufficiently similar. Thus, we have a relaxed destinations exist. The following two destinations are
version of (4): selected as the candidate destinations: Mad (Madrid)
and New (New York).
u ≈ci,k u′ if and only if |e(ci,k , u) − e(ci,k , u′ )|
≤ di,k . (4′ )
4.2. Phase 2: approximate constraint-based
The destinations selected in Phase 1 are evaluated by filtering
(2) and re-ranked by (5) and (6), that is, the approxi- Assume that the target tourist T1 regards the con-
mate constraint satisfaction method of (2), (5), and straints listed in Table 3 as important. In addition,
(6) filters the CF-method’s results and changes their assume that the most important constraint of T1 is
rankings. the airfare and a difference of $200 is not sufficient
A key issue in approximate constraint satisfaction is for discriminating among flights.
the selection of indifference intervals in (3′ ) and (4′ ) for For generating the recommendation set, we
constraints. They can be experimentally determined if compute the satisfaction error e for the constraint
sufficient data are available, that is, optimal indiffer-
ence intervals can be obtained by cross-validation of
Table 1. Similarity value.
recommendation experiments. If these data are not
T2 T3 T4 T5 T6 T7 T8
available, professional travel agents may help deter-
T1 0.32 −0.63 0.06 −0.09 0.41 −0.16 0.02
mine them.
 ASIA PACIFIC JOURNAL OF TOURISM RESEARCH 7

Table 2. SLS. A_Mad002 are less than $10. The target tourist’s
Los Mad New seventh constraint is compared with the candidate
1.75 3.00 2.25 recommendation set. Consequently, we only exclude
A_Mad002 from the candidate recommendation set
because the location of A_Mad002 is far from the rec-
Table 3. Constraints for target tourist T1. ommended hotel H_Mad001 (Hilton Madrid Airport).
Indifference Thus, we recommend the {destination = Mad
No Constraint Interval (Madrid), flight = F_Mad002 (Turkish Airlines), hotel =
1 The airfare is $900 or less $200 H_Mad001 (Hilton Madrid Airport), and activity
2 An one stop flight None
3 The hotel price is $150 or less $10 (events and shows) = A_Mad001 (Flamenco Show at
4 A 4.0-star hotel or above None Café de Chinitas)} to the target tourist T1.
5. Desired activities include events and shows None
6 The activity price is $100 or less $10
7 The activity location is within 10 Km of the 1 Km
recommended hotel 5. Experiments
To perform experiments, a prototype web-based
recommender system is developed for the planning
and the set of candidate recommendations. First, we
travel service shown in Figure 3. To evaluate the
have e(flight price ≤ $900, F_Mad001) = 580, e(flight
proposed prototype system, we developed a bench-
price ≤ $900, F_Mad002) = 154, e(flight price ≤ $900,
mark recommender system that resembles a
F_Mad003) = 50, e(flight price ≤ $900, F_ New001) =
popular online travel booking system. We also sur-
930, e(flight price ≤ $900, F_ New002) = 470, and e
veyed the tourists’ perceptions of the recommen-
(flight price ≤ $900, F_New003) = 190 for the flight.
dation results.
We exclude only F_Mad001, F_New001, and
F_New002 from the set of candidate recommen-
dations because the price difference of $200 is not
5.1. Data collection
sufficient for discriminating among flights. Second,
we perform an analysis according to the target tour- To evaluate the proposed prototype system, 100
ist’s second constraint among the remaining flights. undergraduate students at the K university in
Third, the satisfaction errors for the second constraint Korea participated in the experiments. We assigned
are expressed as follows: e(number of stops per flight 50 students to use the prototype system and the
= 1, F_Mad002) = 0, e(number of stops per flight = 1, remaining 50 students to use the benchmark
F_Mad003) = 1 and e(number of stops per flight = 0, system. We collected the data over a period of
F_New003) = 1. Accordingly, we recommend one and a half months and received survey
F_Mad002 to the target tourist T1. results from 49 students for the prototype recom-
For hotels, we have e(hotel price ≤ $150, mender system and 50 students for the benchmark
H_Mad001) = 0, e(hotel price ≤ $150, H_Mad002) = 40, recommender system.
and e(hotel price ≤ $150, H_Mad003) = 16. We only The data collection was performed in three steps.
exclude H_Mad002 from the candidate recommen- We selected 15 popular destinations in the U.S and
dation set because the price difference of $20 is not Canada and 12 popular destinations in Europe that
sufficiently significant to discriminate the hotel. We students visited. First, students rated the preferences
have e(hotel star rating ≥ 4.0, H_Mad001) = 0 and e of the destinations (among 27 destinations), which
(hotel star rating ≥ 4.0, H_Mad003) = 1. Thus, we only they visited on a five-point scale (5 = “very satisfied”,
select H_Mad001 (Hilton Madrid Airport). 4 = “satisfied”, 3 = “neither”, 2 = “dissatisfied”, and
For activities, we only include A_ Mad001 and 1 = “very dissatisfied”). Second, students input their
A_Mad002 as the candidate recommendation set constraints related to flight, hotel, car, and attraction
because they satisfy the target tourist’s fifth con- into the prototype recommender system. Students
straints. We have e(activity price ≤ $100, A_Mad001) determined the indifference interval values for the
= 0 and e(activity price ≤ $100, A_Mad002) = 7. We prototype recommender system with the help of
also include A_Mad001 and A_Mad002 as the candi- two professional travel agents. Last, each student
date recommendation set because the satisfaction offered travel package recommendations. Here, infor-
error of A_Mad001 and the satisfaction error of mation about flights, hotels, cars, and attractions was
8 I. Y. CHOI ET AL.

Figure 3. Travel recommender system.

Table 4. Number of information usages for recommendation.

Continent Nation Destination Flight Hotel Car Attraction
North America 2 15 57 375 90 418
Europe 9 12 54 383 71 269
Total 11 27 111 758 161 687

extracted from the existing popular online booking typed recommendations; Thus, existing metrics that
system, as shown in Table 4. measure a single item only is not more useful.
Instead, we evaluated the effectiveness, usability,
and novelty of the prototype travel recommender
5.2. Evaluation metrics
system on five-point scales (5 = “strongly agree”, 4
Metrics such as the mean absolute error (MAE), root = “agree”, 3 = “undecided, 2 = “disagree”, and 1
mean square error (RMSE), recall, precision, or F1 = “strongly disagree”), as shown in Table 5 (Ngai &
value have been employed to evaluate the accuracy Wat, 2003; Pu, Chen, & Hu, 2011).
of recommender systems in many studies (Good A travel recommender system does not provide
et al., 1999; Herlocker et al., 2000; Herlocker, students with new information when they are
Konstan, Terveen, & Reidl, 2004; Kim et al., 2009; already interested in the recommended information.
Sarwar et al., 2000). However, these metrics cannot However, if additional novel and detailed travel infor-
be utilized because travel recommender systems mation is provided to students, they will more likely
suggest composite items, such as a tour package- find information that they may not have discovered.

Table 5. Evaluation metrics.

Metrics Items
Effectiveness EFF1 The recommended destination reflects what I want.
EFF2 Although the recommended destination is exactly what I want, I gained a new insight in selecting the destinations.
EFF3 The recommended tour package suits my needs.
EFF4 The recommended tour package reflects what I want.
Usability USA1 System is easy to use.
USA2 System is user friendly.
USA3 Screen display is well designed.
USA4 Response time in the system is acceptable.
USA5 System interface is easy to understand.
USA6 The questions included in the system are relevant to determining the most suitable tour package.
Novelty NOV1 The items (destination, flight, car, or attraction) recommended to me are novel and interesting.
NOV2 System helps me discover new items (destination, flight, car, or attraction).
NOV3 I could find new items (destination, flight, car, or attraction) through the system.
 ASIA PACIFIC JOURNAL OF TOURISM RESEARCH 9

Table 6. Mean responses to the system evaluation by users. Table 7. Exploratory factor analysis.
Benchmark Metrics Items Loading Eigenvalue Cronbach’s α
Prototype System System Effectiveness EFF1 0.863 3.551 0.913
Mean Mean EFF2 0.770
Items Rating S.D. Rating S.D. EFF3 0.789
(On effectiveness of the EFF4 0.825
system) Usability USA1 0.801 4.570 0.926
EFF1 4.0 1.3 4.3 1.0 USA2 0.845
EFF2 3.9 1.3 3.9 1.0 USA3 0.855
EFF3 4.0 1.3 4.1 0.9 USA4 0.653
EFF4 3.9 1.2 3.9 1.1 USA5 0.713
(On usability of the system) USA6 0.707
USA1 4.0 1.1 3.8 1.0 USA7 0.719
USA2 3.9 1.2 3.6 1.0 Novelty NOV1 0.860 2.449 0.869
USA3 4.0 1.2 3.5 1.1 NOV2 0.893
USA4 4.2 1.1 4.2 0.8 NOV3 0.911
USA5 4.0 1.2 4.1 1.0
USA6 3.9 1.2 3.6 1.1
USA7 4.0 1.2 3.7 0.9
(On usability of the system) that of the benchmark recommender system was
USA1 4.0 1.0 3.7 0.9 observed at a 0.05 level of significance. Recommen-
USA2 4.0 1.1 3.4 1.0
USA3 4.0 1.1 3.6 0.8 dations of the current prototype recommender
system are as effective as those of existing benchmark
recommender systems, i.e. travel agencies. However,
as use of the proposed prototype system increases,
5.3. Experimental results and discussions
the prototype recommender system is expected to
Our experiments were performed to compare the rela- be more effective than travel agencies over time.
tive effectiveness, usability, and novelty of the proto- The performance of the prototype recommender
type recommender system with those of the system will increase as the system is updated by learn-
benchmark recommender system. The results of the ing student preferences, with the exception of travel
survey are presented in Table 6. Students rated at agencies.
least 3.9 points for the effectiveness, usability, and Second, no difference was observed between the
novelty of the prototype recommender system. The usability of the prototype recommender system and
effectiveness, usability, and novelty score of the bench- that of the benchmark recommender system at a
mark recommender system were at least 3.4 points. 0.05 level of significance. This result was not our
By the simple descriptive statistics, we cannot expected one. Generally, one of the goals of the
determine a significant difference between the proto- travel recommender system is to solve the complexity
type recommender systems and the benchmark of establishing travel plans (Lorenzi et al., 2007; Raban-
recommender system. For a more detailed analysis, ser & Ricci, 2005; Schiaffino & Amandi, 2009). There-
we performed an exploratory factor analysis (EFA) fore, students may consider that the prototype
and a paired t-test. First, the EFA was performed to recommender system is easier to use than the bench-
investigate the unidimensionality of items by the prin- mark recommender system if information about
cipal component with a varimax rotation, factor load- flights, hotels, rental cars, and attractions in the bench-
ings greater than 0.4 and an eigenvalue greater than mark recommender system is represented with details
1.0. We use Cronbach’s alpha to assess the reliability similar to current travel agencies.
of items. The Cronbach alpha estimated for effective- Last, a significant difference in the mean ratings of
ness, usability, and novelty was 0.913, 0.926 and novelty between the prototype system and the
0.869, respectively (refer to Table 7.). benchmark system. This result indicates that the pro-
Second, we conducted a paired t-test to determine posed system provides novel destinations, flights,
whether the mean rating of effectiveness, usability, hotels, or attractions and accidental discovery regard-
and novelty for the prototype system differ from less of the students’ expectations. This finding also
those for the benchmark system. Table 8 shows the indicates that tourists can take serendipitous travel
results of the paired t-tests. opportunities to enjoy inexperienced destinations,
First, no significant difference between the effec- hotels, or attractions if they use the proposed
tiveness of the prototype recommender system and system. Serendipity tourists can save more time on
10 I. Y. CHOI ET AL.

Table 8. Paired t-test.

Mean Rating S.D
Metrics Prototype System Benchmark System Prototype System Benchmark System t-value p-value
Effectiveness 4.0 4.1 1.2 0.8 −.464 0.664
Usability 4.0 3.8 1.0 0.8 1.121 0.265
Novelty 4.0 3.5 0.9 0.8 2.743 0.007

travel planning and information collection about des- Second, tourists can save time on travel planning,
tinations, hotels, or attractions instead of following a and travel agencies can benefit from increasing the
structured itinerary (Huang et al., 2014). Therefore, sales diversity by providing serendipitous travel
the novelty can generated perceived usefulness of opportunities, which will motivate voluntary partici-
the proposed system. The novelty can increase the pation and improve the health of the smart tourism
sales diversity by changing tourists’ preferences on ecosystem. Third, existing CF-based recommender
destinations, hotels, or attractions (Aggarwal, 2016; systems generally recommend items in a single
Pu et al., 2011). domain only; thus, the simultaneous recommendation
of multi items (for example, hotels, flight, and attrac-
tions). However, the suggested travel recommender
6. Conclusions system combined with approximate constraint satis-
faction simultaneously provides destination, hotels,
For the enhancement of tourists’ experiences, smart flight, and attraction. This study suggested a clue or
tourism cities can offer personalized travel-planning a heuristic methodology to expand the application
service by analyzing traveling patterns of tourists of CF-based recommender systems.
and learning their results to increase the satisfaction However, we are able to derive only limited con-
of tourists. For this purpose of smart tourism cities, clusions on our research by the evaluation with a
learning travel patterns, such as tourists’ preferences, small subject set. Conducting experiments using a
time, and budget, and offering personalized attrac- large number of samples will be a promising future
tions, such as shopping, events, and shows, are critical. research area. Another limitation is the process of eli-
Travel recommender systems are designed for the citing tourists’ constraints. In this prototype system,
implementation of smart tourism cities. However, tourists supply their constraints by themselves, but
existing travel planning systems cannot support the this process is not easy for tourists. We are currently
automation of WOM communication. They only designing algorithms to facilitate or automate the eli-
imitate the role of human travel agents. This research citation process of tourist’s constraints. Other method-
suggests a CF-based travel recommender system for a ologies that simultaneously recommend multiple
personalized travel-planning service to enhance the items in a different domain is a promising research
tourists’ experience. The suggested recommender area related to the travel recommender system.
system analyzes tourists’ preferences and offers per-
sonalized destinations, flights, hotels, and attractions.
The basic methodology of the proposed travel recom-
Disclosure statement
mender system is to combine CF with approximate
constraint satisfaction for the personalized rec- No potential conflict of interest was reported by the authors.
ommendations. CF is applied to recommend a desti-
nation and approximate constraint satisfaction is
applied to recommend flight, hotel, and attraction References
packages. Therefore, the proposed system supports Adomavicius, G., & Tuzhilin, A. (2005). Toward the next gener-
the automation of WOM communication and a per- ation of recommender systems: A survey of the state-of-the-
sonalized travel-planning service. art and possible extensions. IEEE Transactions on Knowledge
This finding implies that the proposed method- & Data Engineering, 17(6), 734–749.
ology can assist tourists in finding destinations, Aggarwal, C. C. (2016). Recommender systems. (pp. 1–28). Cham:
Springer International Publishing.
hotels, or attractions that are suitable to their prefer- Bae, S., & Lee, T. (2011). Product type and consumers’ perception
ences. Consequently, tourists’ experiences will be of online consumer reviews. Electronic Markets, 21(4), 255–
reinforced and their satisfaction will be increased. 266.
 ASIA PACIFIC JOURNAL OF TOURISM RESEARCH 11

Balabanovic, M., & Shoham, Y. (1997). Fab: Content-based colla- Hernández-Méndez, J., Muñoz-Leiva, F., & Sánchez-Fernández, J.
borative recommendation. Communications of the ACM, 40 (2015). The influence of e-word-of-mouth on travel decision-
(3), 66–72. making: Consumer profiles. Current Issues in Tourism, 18(11),
Baloglu, S., Pekcan, A., Chen, S. L., & Santos, J. (2004). The relation- 1001–1021.
ship between destination performance, overall satisfaction, Huang, Y., & Bian, L. (2009). A Bayesian network and analytic hier-
and behavioral intention for distinct segments. Journal of archy process based personalized recommendations for
Quality Assurance in Hospitality & Tourism, 4(3-4), 149–165. tourist attractions over the internet. Expert Systems with
Bennett, J., & Lanning, S. (2007, August). The netflix prize. In Applications, 36(1), 933–943.
Proceedings of KDD cup and workshop (Vol. 2007, p. 35). Huang, W. J., Norman, W. C., Hallo, J. C., Mcgehee, N. G., Mcgee, J.,
Cai, L. A., Feng, R., & Breiter, D. (2004). Tourist purchase decision & Goetcheus, C. L. (2014). Serendipity and independent travel.
involvement and information preferences. Journal of Vacation Tourism Recreation Research, 39(2), 169–183.
Marketing, 10(2), 138–148. Jalilvand, M. R. (2017). Word-of-mouth vs. Mass media: Their con-
Cho, Y. H., Kim, J. K., & Kim, S. H. (2002). A personalized recommen- tributions to destination image formation. Anatolia, 28(2),
der system based on web usage mining and decision tree 151–162.
induction. Expert Systems With Applications, 23(3), 329–342. Jalilvand, M. R., Samiei, N., Dini, B., & Manzari, P. Y. (2012).
Choi, I. Y., Kim, J. K., & Ryu, Y. U. (2015, January). A two-tiered Examining the structural relationships of electronic word of
recommender system for tourism product recommendations. mouth, destination image, tourist attitude toward destina-
In 2015 48th Hawaii International Conference on System tion and travel intention: An integrated approach. Journal of
Sciences. 3354–3363. Destination Marketing & Management, 1(1-2), 134–143.
Choi, I. Y., Oh, M. G., Kim, J. K., & Ryu, Y. U. (2016). Collaborative Kim, J. Y., & Canina, L. (2015). An analysis of smart tourism system
filtering with facial expressions for online video recommen- satisfaction scores: The role of priced versus average quality.
dation. International Journal of Information Management, 36 Computers in Human Behavior, 50, 610–617.
(3), 397–402. Kim, J. K., Kim, H. K., Oh, H. Y., & Ryu, Y. U. (2010). A group rec-
Cox, C., Burgess, S., Sellitto, C., & Buultjens, J. (2009). The role of ommendation system for online communities. International
user-generated content in tourists’ travel planning behavior. Journal of Information Management, 30(3), 212–219.
Journal of Hospitality Marketing & Management, 18(8), 743–764. Kim, H. K., Kim, J. K., & Ryu, Y. U. (2009). Personalized recommen-
Coyle, L., & Cunningham, P. (2003). Exploiting re-ranking infor- dation over a user network for ubiquitous shopping. IEEE
mation in a case-based personal travel assistant. In Proceedings Transactions on Services Computing, 2(2), 140–151.
of the 5th International Conference on Case-based Reasoning. Kim, C., Lee, J., Cho, Y., & Kim, D. (2004). VISCORS: A visual con-
Das, A. S., Datar, M., Garg, A., & Rajaram, S. (2007, May). Google tents recommender system on the mobile web. IEEE
news personalization: scalable online collaborative filtering. Intelligent Systems, 19(6), 32–39.
In Proceedings of the 16th international conference on World Kim, J. K., Moon, H. S., An, B. J., & Choi, I. Y. (2018). A grocery rec-
Wide Web (pp. 271–280). ACM. ommendation for off-line shoppers. Online Information
Garcia, I., Sebastia, L., & Onaindia, E. (2011). On the design of indi- Review, 42(4), 468–481.
vidual and group recommender systems for tourism. Expert Kim, H. K., Oh, H. Y., Gu, J. C., & Kim, J. K. (2011). Commenders: A
Systems with Applications, 38(6), 7683–7692. recommendation procedure for online book communities.
Gavalas, D., Konstantopoulos, C., Mastakas, K., & Pantziou, G. Electronic Commerce Research and Applications, 10(5), 501–509.
(2014). Mobile recommender systems in tourism. Journal of Lee, H., Lee, J., Chung, N., & Koo, C. (2018). Tourists’ happiness:
Network and Computer Applications, 39, 319–333. Are there smart tourism technology effects? Asia Pacific
Goldberg, D., Nichols, D., Oki, B. M., & Terry, D. (1992). Using col- Journal of Tourism Research, 23(5), 486–501.
laborative filtering to weave an information tapestry. Li, N., Tung, V., & Law, R. (2017). A fuzzy comprehensive evalu-
Communications of the ACM, 35(12), 61–71. ation algorithm for analyzing electronic word-of-mouth. Asia
Good, N., Schafer, B., Konstan, J., Borchers, A., Sarwar, B., Pacific Journal of Tourism Research, 22(6), 592–603.
Herlocker, J., & Riedl, J. (1999). Combining collaborative Linden, G., Smith, B., & York, J. (2003). Amazon.com recommen-
filtering with personal agents for better recommendations. dations: Item-to-item collaborative filtering. IEEE Internet
In Proceedings of the AAAI’99 conference. (pp. 439–446). Computing, 7(1), 76–80.
Gretzel, U., Sigala, M., Xiang, Z., & Koo, C. (2015). Smart tourism: Litvin, S. W., Goldsmith, R., & Pan, B. (2008). Electronic word-of-
Foundations and developments. Electronic Markets, 25(3), mouth in hospitality and tourism management. Tourism
179–188. Management, 29(3), 458–468.
Gretzel, U., & Yoo, K. (2008). Use and impact of online travel Liu, Q., Chen, E., Xiong, H., Ge, Y., Li, Z., & Wu, X. (2014). A cocktail
reviews. Information and Communication Technologies in approach for travel package recommendation. IEEE
Tourism, 2008, 35–46. Transactions on Knowledge and Data Engineering, 26(2), 278–
Gretzel, U., Zhong, L., & Koo, C. (2016). Application of smart 293.
tourism to cities. International Journal of Tourism Cities, 2(2). Liu, X., Schuckert, M., & Law, R. (2015). Can response manage-
Herlocker, J. L., Konstan, J. A., & Reidl, J. (2000). Explaining colla- ment benefit hotels? Evidence from Hong Kong hotels.
borative filtering recommendations. In Proceedings of the 2000 Journal of Travel & Tourism Marketing, 32(8), 1069–1080.
Conference on Computer Supported Cooperative Work. Lorenzi, F., Bazzan, A. L. C., & Abel, M. (2007). Truth maintenance
(pp. 241–250). task negotiation in multi agent recommender system for
Herlocker, J. L., Konstan, J. A., Terveen, L. G., & Reidl, J. (2004). tourism. In Proceedings of the AAAI Workshop on Intelligent
Evaluating collaborative filtering recommender systems. Techniques for Web Personalization and Recommender
ACM Transactions on Information Systems (TOIS), 22(1), 5–53. Systems in E-Commerce (AAAI 2007). (pp. 122–125).
12 I. Y. CHOI ET AL.

Lorenzi, F., Loh, S., & Abel, M. (2011, August). PersonalTour: A Rabanser, U., & Ricci, F. (2005). Recommendation systems: Do
recommender system for travel packages. In Web they have a viable business model in e-tourism. In
Intelligence and Intelligent Agent Technology (WI-IAT), 2011 Conference of the International Federation for IT & Travel and
IEEE/WIC/ACM International Conference on (Vol. 2, pp. 333– Tourism, 160–171.
336). IEEE. Ryu, Y. U. (1998). Approximate constraint satisfaction over a con-
Luce, R. D. (1956). Semiorders and a theory of utility discrimi- straint hierarchy. In Proceedings of the 31st Hawaii
nation. Econometrica, 24(2), 178–191. International Conference on System Science. 5, 134–141.
Melville, P., Mooney, R. J., & Nagarajan, R. (2002). Content- Sarwar, B., Karypis, G., Konstan, J. A., & Riedl, J. (2000). Analysis of
boosted collaborative filtering for improved recommen- recommendation algorithms for e-commerce. In Proceedings
dations, In Proceedings of the Eighteenth National Conference of ACM E-commerce 2000 conference. 158–167.
on Artificial Intelligence (AAAI 02). (pp. 187–192). Sarwar, B. M., Konstan, J. A., Borchers, A., Herlocker, J., Miller, B., &
Moon, H. S., Kim, J. K., & Ryu, Y. U. (2013). A sequence-based Riedl, J. (1998, November). Using filtering agents to
filtering method for exhibition booth visit recommendations. improve prediction quality in the grouplens research colla-
International Journal of Information Management, 33(4), 620– borative filtering system. In Proceedings of the 1998 ACM con-
626. ference on Computer supported cooperative work (pp. 345–
Murphy, L., Mascardo, G., & Benckendorff, P. (2007). Exploring 354). ACM.
word-of-mouth influences on travel decisions: Friends and Schiaffino, S., & Amandi, A. (2009). Building an expert travel agent
relatives vs. Other travellers. International Journal of as a software agent. Expert Systems with Applications, 36(2),
Consumer Studies, 31(5), 517–527. 1291–1299.
Ngai, E. W. T., & Wat, F. K. T. (2003). Design and development of a Schuckert, M., Liu, X., & Law, R. (2015). Hospitality and tourism
fuzzy expert system for hotel selection. Omega, 31(4), 275– online reviews: Recent trends and future directions. Journal
286. of Travel & Tourism Marketing, 32(5), 608–621.
Nilashi, M., bin Ibrahim, O., Ithnin, N., & Sarmin, N. H. (2015). A Schuckert, M., Liu, X., & Law, R. (2016). Insights into suspicious
multi-criteria collaborative filtering recommender system for online ratings: Direct evidence from TripAdvisor. Asia Pacific
the tourism domain using Expectation Maximization (EM) Journal of Tourism Research, 21(3), 259–272.
and PCA–ANFIS. Electronic Commerce Research and Tham, A., Croy, G., & Mair, J. (2013). Social media in destina-
Applications, 14(6), 542–562. tion choice: Distinctive electronic word-of-mouth dimen-
Park, D. H., Kim, H. K., Choi, I. Y., & Kim, J. K. (2012). A literature sions. Journal of Travel & Tourism Marketing, 30(1-2),
review and classification of recommender systems research. 144–155.
Expert Systems with Applications, 39(11), 10059–10072. TotalMedia. (2010). Social travel. Accessed online (www.
Park, D. H., Lee, J. M., & Han, I. G. (2007). The effects of on-line con- totalmedia.co.uk) on April 25, 2012
sumer reviews on consumer: The moderating role of involve- Tung, V. W. S., & Ritchie, J. B. (2011). Exploring the essence of
ment. International Journal of Electronic Commerce, 11(4), 125– memorable tourism experiences. Annals of Tourism Research,
148. 38(4), 1367–1386.
Pu, P., Chen, L., & Hu, R. (2011, October). A user-centric evaluation Yeh, D. Y., & Cheng, C. H. (2015). Recommendation system for
framework for recommender systems. In Proceedings of the popular tourist attractions in Taiwan using Delphi panel
fifth ACM conference on Recommender systems (pp. 157–164). and repertory grid techniques. Tourism Management, 46,
ACM. 164–176.
 ASIA PACIFIC JOURNAL OF TOURISM RESEARCH 13

Appendix

Table A1. Travel Database: Destinations.

Destination_ID Destination Destination_ID Destination
Ber Berlin Mon Montreal
Chi Chicago New New York
Lon London Par Paris
Los Los Angeles Rom Rome
Mad Madrid Tor Toronto

Table A2. Travel database: flights.

Flight_ID Destination Flight Stopover Price
F_ Ber001 Berlin Lufthansa Airline Nonstop $1,325
F_ Ber002 Berlin Finnair 1 stop (via Helsinki) $1,036
F_ Chi001 Chicago Delta Air Lines Nonstop $1,132
F_Chi002 Chicago Asiana Airlines 1+ stop (via Seattle) $932
F_Lon001 London British Airways Nonstop $2,503
F_Lon002 London Air China 1 stop (via Beijing) $1,818
F_Los001 Los Angeles United Airlines Nonstop $1,110
F_Los002 Los Angeles Japan Airlines 1 stop (via Osaka) $947
F_Mad001 Madrid Iberia Airlines Nonstop $1,480
F_Mad002 Madrid Turkish Airlines 1 stop (via Istanbul) $1,054
F_Mad003 Madrid Korean Air 2+ stop (via Dubai and Rome) $950
F_Mon001 Montreal Air Canada Nonstop $1,715
F_Mon002 Montreal Air Canada 1 stop (via Toronto) $1,498
F_New001 New York American Airlines Nonstop $1,830
F_New002 New York United Airlines 1 stop (via San Francisco) $1,370
F_New003 New York Delta Air Lines 2+ stop (via Tokyo and Los Angeles) $1,090
F_Par001 Paris Air France Nonstop $1,457
F_Par002 Paris Korean Air 1 stop (via Rome) $1,234
F_Rom001 Rome Alitalia Nonstop $1,683
F_Rom002 Rome Asiana Airlines 1 stop (via Amsterdam) $1,450
F_Tor001 Toronto Air Canada Nonstop $2,350
F_Tor002 Toronto Korean Air 1 stop (via Vancouver) $1,910

Table A3. Travel database: hotels.

Hotel_ID Destination Hotel Rating Price
H_ Ber001 Berlin SO/ Berlin Das Stue 5.0 $254
H_ Ber002 Berlin Hotel Berlin 4.5 $199
H_ Chi001 Chicago Holiday Inn Chicago Mart Plaza River North 3.0 $150
F_Chi002 Chicago Hilton Chicago Ohare Airport 4.0 $174
H_Lon001 London Park Plaza Westminster Bridge London 4.0 $330
H_Lon002 London Days Hotel London Waterloo 3.0 $125
H_Los001 Los Angeles Millennium Biltmore Hotel Los Angeles 3.5 $330
H_Los002 Los Angeles Hilton Los Angeles Airport 4.0 $157
H_Mad001 Madrid Hilton Madrid Airport 4.0 $130
H_Mad002 Madrid Melia Madrid Princesa 5.0 $190
H_Mad003 Madrid Hotel Regente 3.0 $166
H_Mon001 Montreal Hyatt Regency Montreal 4.0 $140
F_Mon002 Montreal Quality Hotel Midtown 3.0 $85
H_New001 New York Intercontinental New York Barclay 4.0 $203
H_New002 New York The Hotel Edison 3.0 $180
H_Par001 Paris Villa Pantheon 4.0 $121
H_Par002 Paris Eiffel Rive Gauche 3.0 $102
H_Rom001 Rome Una Hotel Roma 4.0 $185
H_Rom002 Rome IQ Hotel Roma 4.0 $169
H_Tor001 Toronto Hilton Toronto 4.0 $157
H_Tor002 Toronto Comfort Hotel Downtown 3.0 $125
14 I. Y. CHOI ET AL.

Table A4. Travel database: activities.

Activity_ID Destination Activity Class Activity Name Price Hotel within 10 Km from here
A_ Ber001 Berlin Events and Shows Friedrichstadt-Palast VIVID Grand Show $46.00 SO/ Berlin Das Stue
A_ Ber002 Berlin Tours and Berlin Hop-On Hop-Off Bus Tou $25.00 Hotel Berlin
Sightseeing
A_ Chi001 Chicago Shopping and Loop Interior Architecture & Pedway $28.00 Holiday Inn Chicago Mart Plaza
Fashion Walking Tour River North
A_Chi002 Chicago Events and Shows Tommy Gun’s Garage Dinner Show $67.00 Hilton Chicago Ohare Airport
A_Lon001 London Arts and Culture The Household Cavalry Museum $9.07 Park Plaza Westminster Bridge
London
A_Lon002 London Events and Shows Sister Act $93.87 Days Hotel London Waterloo
A_Los001 Los Angeles Events and Shows Medieval Times Dinner & Tournament $63.00 Millennium Biltmore Hotel Los
Angeles
A_Los002 Los Angeles Tours and Hollywood Sign & Griffith Park Hiking $39.00 Hilton Los Angeles Airport
Sightseeing Tour
A_Mad001 Madrid Events and Shows Flamenco Show at Café de Chinitas $48.00 Hilton Madrid Airport
A_Mad002 Madrid Events and Shows Anastasia The Musical $107.00 Melia Madrid Princesa
A_Mad003 Madrid Tours and Day Tour of Toledo & Segovia with $79.00 Hotel Regente
Sightseeing Alcazar Entrance
A_Mon001 Montreal Arts and Culture Montreal Art Gallery $17.18 Hyatt Regency Montreal
A_Mon002 Montreal Air and Land Montreal Bicycle Tour $66.2 Quality Hotel Midtown
Adventures
A_New001 New York Events and Shows Mamma Mia! $96.5 Intercontinental New York Barclay
A_New002 New York Events and Shows The Lion King $108.5 The Hotel Edison
A_Par001 Paris Events and Shows Crazy Horse Cabaret Show $127.98 Villa Pantheon
A_Par002 Paris Theme Parks Disneyland® Paris Tickets $62 Eiffel Rive Gauche
A_Rom001 Rome Arts and Culture Papal Audience $38.66 Una Hotel Roma
A_Rom002 Rome Tours and Rome in One Day Tour $141.16 IQ Hotel Roma
Sightseeing
A_Tor001 Toronto Events and Shows Jersey Boys $75.34 Hilton Toronto
A_Tor002 Toronto Events and Shows The Second City $23.4 Comfort Hotel Downtown

Table A5. User profile.

Ber Chi Lon Los Mad Mon New Par Rom Tor
T1 1 3 1 2 2
T2 4 4 3 2 1
T3 2 3 2 2 3 4
T4 2 3 4 1 4 5 3
T5 3 3 4 4
T6 5 3 4 4
T7 2 2 4 4 2 5
T8 3 4 6 3 5 4
Note: The figure in each cell indicates the tourist’s rating based on how much they liked the destination (5 = highest, 1 = lowest).