Social media and deep learning capture the aesthetic quality of the landscape

Social media and deep learning capture the aesthetic quality of the landscape
  • 1.

    Daniel, T. C. et al. Contributions of cultural services to the ecosystem services agenda. Proc. Natl. Acad. Sci. USA 109, 8812–8819. https://doi.org/10.1073/pnas.1114773109 (2012).

    ADS 
    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • 2.

    Gobster, P. H., Nassauer, J. I., Daniel, T. C. & Fry, G. The shared landscape: What does aesthetics have to do with ecology?. Landsc. Ecol. 22, 959–972. https://doi.org/10.1007/s10980-007-9110-x (2007).

    Article 

    Google Scholar
     

  • 3.

    Abraham, A., Sommerhalder, K. & Abel, T. Landscape and well-being: A scoping study on the health-promoting impact of outdoor environments. Int. J. Public Health 55, 59–69. https://doi.org/10.1007/s00038-009-0069-z (2010).

    Article 
    PubMed 

    Google Scholar
     

  • 4.

    Rice, W. L. et al. Changes in recreational behaviors of outdoor enthusiasts during the COVID-19 pandemic: Analysis across urban and rural communities. J. Urban Ecol. https://doi.org/10.1093/jue/juaa020 (2020).

  • 5.

    Venter, Z. S., Barton, D. N., Gundersen, V., Figari, H. & Nowell, M. Urban nature in a time of crisis: Recreational use of green space increases during the COVID-19 outbreak in Oslo, Norway. Environ. Res. Lett. 15, 104075. https://doi.org/10.1088/1748-9326/abb396 (2020).

    ADS 
    Article 
    CAS 

    Google Scholar
     

  • 6.

    Maes, J. et al. Mainstreaming ecosystem services into EU policy. Curr. Opin. Environ. Sustain. 5, 128–134. https://doi.org/10.1016/j.cosust.2013.01.002 (2013).

    Article 

    Google Scholar
     

  • 7.

    Paracchini, M. L. et al. Mapping cultural ecosystem services: A framework to assess the potential for outdoor recreation across the EU. Ecol. Indic. 45, 371–385. https://doi.org/10.1016/j.ecolind.2014.04.018 (2014).

    Article 

    Google Scholar
     

  • 8.

    Hein, L. et al. Progress in natural capital accounting for ecosystems. Science 367, 514–515. https://doi.org/10.1126/science.aaz8901 (2020).

    ADS 
    Article 
    PubMed 
    CAS 

    Google Scholar
     

  • 9.

    Díaz, S. et al. Assessing nature’s contributions to people. Science 359, 270–272. https://doi.org/10.1126/science.aap8826 (2018).

  • 10.

    Martínez-Harms, M. J. & Balvanera, P. Methods for mapping ecosystem service supply: A review. Int. J. Biodiv. Sci. Ecosyst. Serv. Manag. 8, 17–25. https://doi.org/10.1080/21513732.2012.663792 (2012).

    Article 

    Google Scholar
     

  • 11.

    Raymond, C. M., Kenter, J. O., Plieninger, T., Turner, N. J. & Alexander, K. A. Comparing instrumental and deliberative paradigms underpinning the assessment of social values for cultural ecosystem services. Ecol. Econ. 107, 145–156. https://doi.org/10.1016/j.ecolecon.2014.07.033 (2014).

    Article 

    Google Scholar
     

  • 12.

    Bateman, I. J. et al. Bringing ecosystem services into economic decision-making: Land use in the United Kingdom. Science 341, 45–50. https://doi.org/10.1126/science.1234379 (2013).

    ADS 
    Article 
    PubMed 
    CAS 

    Google Scholar
     

  • 13.

    Hernández-Morcillo, M., Plieninger, T. & Bieling, C. An empirical review of cultural ecosystem service indicators. Ecol. Indic. 29, 434–444. https://doi.org/10.1016/j.ecolind.2013.01.013 (2013).

    Article 

    Google Scholar
     

  • 14.

    Hermes, J., Albert, C. & von Haaren, C. Assessing the aesthetic quality of landscapes in Germany. Ecosyst. Serv. 31, 296–307. https://doi.org/10.1016/j.ecoser.2018.02.015 (2018).

    Article 

    Google Scholar
     

  • 15.

    Uuemaa, E., Antrop, M., Roosaare, J., Marja, R. & Mander, U. Landscape metrics and indices: An overview of their use in landscape research. Living Rev. Landsc. Res. 3, 1–28. https://doi.org/10.12942/lrlr-2009-1 (2009).

    Article 

    Google Scholar
     

  • 16.

    Schirpke, U., Tasser, E. & Tappeiner, U. Predicting scenic beauty of mountain regions. Landsc. Urban Plan. 111, 1–12. https://doi.org/10.1016/j.landurbplan.2012.11.010 (2013).

    Article 

    Google Scholar
     

  • 17.

    Tveit, M., Ode, Å. & Fry, G. Key concepts in a framework for analysing visual landscape character. Landsc. Res. 31, 229–255. https://doi.org/10.1080/01426390600783269 (2006).

    Article 

    Google Scholar
     

  • 18.

    Ode, Å., Tveit, M. & Fry, G. Capturing landscape visual character using indicators: Touching base with landscape aesthetic theory. Landsc. Res. 33, 89–117. https://doi.org/10.1080/01426390701773854 (2008).

    Article 

    Google Scholar
     

  • 19.

    de Groot, R. S., Alkemade, R., Braat, L., Hein, L. & Willemen, L. Challenges in integrating the concept of ecosystem services and values in landscape planning, management and decision making. Ecol. Complex. 7, 260–272. https://doi.org/10.1016/j.ecocom.2009.10.006 (2010).

    Article 

    Google Scholar
     

  • 20.

    Schröter, M., Remme, R. P., Sumarga, E., Barton, D. N. & Hein, L. Lessons learned for spatial modelling of ecosystem services in support of ecosystem accounting. Ecosyst. Serv. 13, 64–69. https://doi.org/10.1016/j.ecoser.2014.07.003 (2015).

    Article 

    Google Scholar
     

  • 21.

    Tenerelli, P., Püffel, C. & Luque, S. Spatial assessment of aesthetic services in a complex mountain region: Combining visual landscape properties with crowdsourced geographic information. Landsc. Ecol. 32, 1097–1115. https://doi.org/10.1007/s10980-017-0498-7 (2017).

    Article 

    Google Scholar
     

  • 22.

    Wood, S. A., Guerry, A. D., Silver, J. M. & Lacayo, M. Using social media to quantify nature-based tourism and recreation. Sci. Rep. 3, 1–7. https://doi.org/10.1038/srep02976 (2013).

    Article 

    Google Scholar
     

  • 23.

    van Zanten, B. T. et al. Continental-scale quantification of landscape values using social media data. Proc. Natl. Acad. Sci. USA 113, 12974–12979. https://doi.org/10.1073/pnas.1614158113 (2016).

    ADS 
    Article 
    PubMed 
    PubMed Central 
    CAS 

    Google Scholar
     

  • 24.

    Tenerelli, P., Demšar, U. & Luque, S. Crowdsourcing indicators for cultural ecosystem services: A geographically weighted approach for mountain landscapes. Ecol. Indic. 64, 237–248. https://doi.org/10.1016/j.ecolind.2015.12.042 (2016).

    Article 

    Google Scholar
     

  • 25.

    Richards, D. R. & Tunçer, B. Using image recognition to automate assessment of cultural ecosystem services from social media photographs. Ecosyst. Serv. 31, 318–325. https://doi.org/10.1016/j.ecoser.2017.09.004 (2018).

    Article 

    Google Scholar
     

  • 26.

    Sinclair, M., Mayer, M., Woltering, M. & Ghermandi, A. Valuing nature-based recreation using a crowdsourced travel cost method: A comparison to onsite survey data and value transfer. Ecosyst. Serv. 45, 101165. https://doi.org/10.1016/j.ecoser.2020.101165 (2020).

    Article 

    Google Scholar
     

  • 27.

    Antoniou, V. et al. Investigating the feasibility of geo-tagged photographs as sources of land cover input data. ISPRS Int. J. Geo-Inf.https://doi.org/10.3390/ijgi5050064 (2016).

    Article 

    Google Scholar
     

  • 28.

    Mancini, F., Coghill, G. M. & Lusseau, D. Quantifying wildlife watchers’ preferences to investigate the overlap between recreational and conservation value of natural areas. J. Appl. Ecol. 56, 387–397. https://doi.org/10.1111/1365-2664.13274 (2019).

  • 29.

    Hollenstein, L. & Purves, R. Exploring place through user-generated content: Using Flickr tags to describe city cores. J. Spatial Inf. Sci. 1, 21–48. https://doi.org/10.5311/JOSIS.2010.1.3 (2010).

    Article 

    Google Scholar
     

  • 30.

    Donahue, M. L. et al. Using social media to understand drivers of urban park visitation in the Twin Cities, MN. Landsc. Urban Plan. 175, 1–10. https://doi.org/10.1016/j.landurbplan.2018.02.006 (2018).

    Article 

    Google Scholar
     

  • 31.

    LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444. https://doi.org/10.1038/nature14539 (2015).

    ADS 
    Article 
    PubMed 
    PubMed Central 
    CAS 

    Google Scholar
     

  • 32.

    Naik, N., Kominers, S. D., Raskar, R., Glaeser, E. L. & Hidalgo, C. A. Computer vision uncovers predictors of physical urban change. Proc. Natl. Acad. Sci. USA 114, 7571–7576. https://doi.org/10.1073/pnas.1619003114 (2017).

  • 33.

    Toivonen, T. et al. Social media data for conservation science: A methodological overview. Biol. Conserv. 233, 298–315. https://doi.org/10.1016/j.biocon.2019.01.023 (2019).

    Article 

    Google Scholar
     

  • 34.

    Zhang, F., Zhou, B., Ratti, C. & Liu, Y. Discovering place-informative scenes and objects using social media photos. R. Soc. Open Sci. 6, 181375. https://doi.org/10.1098/rsos.181375 (2019).

    ADS 
    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • 35.

    Srivastava, S., Vargas Muñoz, J. E., Lobry, S. & Tuia, D. Fine-grained landuse characterization using ground-based pictures: A deep learning solution based on globally available data. Int. J. Geogr. Inf. Sci. 34, 1117–1136. https://doi.org/10.1080/13658816.2018.1542698 (2020).

    Article 

    Google Scholar
     

  • 36.

    Egarter Vigl, L. et al. Harnessing artificial intelligence technology and social media data to support cultural ecosystem service assessments. People Nat. 3, 673–685. https://doi.org/10.1002/pan3.10199 (2021).

    Article 

    Google Scholar
     

  • 37.

    ScenicOrNot. ScenicOrNot Dataset (2015). http://scenicornot.datasciencelab.co.uk.

  • 38.

    Seresinhe, C. I., Moat, H. S. & Preis, T. Quantifying scenic areas using crowdsourced data. Environ. Plan. B Urban Anal. City Sci. 45, 567–582. https://doi.org/10.1177/0265813516687302 (2017).

    Article 

    Google Scholar
     

  • 39.

    Chesnokova, O., Nowak, M. & Purves, R. S. A crowdsourced model of landscape preference. In 13th International Conference on Spatial Information Theory (COSIT 2017), vol. 86. https://doi.org/10.4230/LIPIcs.COSIT.2017.19 (2017).

  • 40.

    Seresinhe, C. I., Tobias, P. & Moat, H. S. Using deep learning to quantify the beauty of outdoor places. R. Soc. Open Sci.. https://doi.org/10.1098/rsos.170170 (2017).

    MathSciNet 
    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • 41.

    Workman, S., Souvenir, R. & Jacobs, N. Understanding and mapping natural beauty. In 2017 IEEE International Conference on Computer Vision (ICCV), vol. 4, 5590–5599. https://doi.org/10.1109/ICCV.2017.596 (2017).

  • 42.

    Marcos, D. et al. Contextual semantic interpretability. In Proceedings of the Asian Conference on Computer Vision (2020).

  • 43.

    Arendsen, P., Marcos, D. & Tuia, D. Concept discovery for the interpretation of landscape scenicness. Mach. Learn. Knowl. Extract.https://doi.org/10.3390/make2040022 (2020).

    Article 

    Google Scholar
     

  • 44.

    Levering, A., Marcos, D. & Tuia, D. On the relation between landscape beauty and land cover: A case study in the U.K. at Sentinel-2 resolution with interpretable AI. ISPRS J. Photogram. Remote Sens. 177, 194–203. https://doi.org/10.1016/j.isprsjprs.2021.04.020 (2021).

    ADS 
    Article 

    Google Scholar
     

  • 45.

    Havinga, I., Bogaart, P. W., Hein, L. & Tuia, D. Defining and spatially modelling cultural ecosystem services using crowdsourced data. Ecosyst. Serv. 43, 101091. https://doi.org/10.1016/j.ecoser.2020.101091 (2020).

    Article 

    Google Scholar
     

  • 46.

    Oteros-Rozas, E., Martín-López, B., Fagerholm, N., Bieling, C. & Plieninger, T. Using social media photos to explore the relation between cultural ecosystem services and landscape features across five European sites. Ecol. Indic. 94, 74–86. https://doi.org/10.1016/j.ecolind.2017.02.009 (2018).

    Article 

    Google Scholar
     

  • 47.

    Englund, O., Berndes, G. & Cederberg, C. How to analyse ecosystem services in landscapes—A systematic review. Ecol. Indic. 73, 492–504. https://doi.org/10.1016/j.ecolind.2016.10.009 (2017).

    Article 

    Google Scholar
     

  • 48.

    Zhou, B., Lapedriza, A., Khosla, A., Oliva, A. & Torralba, A. Places: A 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40, 1452–1464. https://doi.org/10.1109/TPAMI.2017.2723009 (2017).

    Article 
    PubMed 

    Google Scholar
     

  • 49.

    Patterson, G., Xu, C., Su, H. & Hays, J. The SUN attribute database: Beyond categories for deeper scene understanding. Int. J. Comput. Vis. 108, 59–81. https://doi.org/10.1007/s11263-013-0695-z (2014).

    Article 

    Google Scholar
     

  • 50.

    Lee, H., Seo, B., Koellner, T. & Lautenbach, S. Mapping cultural ecosystem services 2.0—Potential and shortcomings from unlabeled crowd sourced images. Ecol. Indic. 96, 505–515. https://doi.org/10.1016/j.ecolind.2018.08.035 (2019).

    Article 

    Google Scholar
     

  • 51.

    Ulrich, R. S. Visual landscapes and psychological well-being. Landsc. Res. 4, 17–23. https://doi.org/10.1080/01426397908705892 (1979).

    Article 

    Google Scholar
     

  • 52.

    Cordingley, J. E., Newton, A. C., Rose, R. J., Clarke, R. T. & Bullock, J. M. Habitat fragmentation intensifies trade-offs between biodiversity and ecosystem services in a heathland ecosystem in Southern England. PLOS ONE 10, e0130004. https://doi.org/10.1371/journal.pone.0130004 (2015).

    Article 
    PubMed 
    PubMed Central 
    CAS 

    Google Scholar
     

  • 53.

    Newton, A. C. et al. Impacts of grazing on lowland heathland in north-west Europe. Biol. Conserv. 142, 935–947. https://doi.org/10.1016/j.biocon.2008.10.018 (2009).

    Article 

    Google Scholar
     

  • 54.

    Tveit, M. S. Indicators of visual scale as predictors of landscape preference; A comparison between groups. J. Environ. Manag. 90, 2882–2888. https://doi.org/10.1016/j.jenvman.2007.12.021 (2009).

    Article 

    Google Scholar
     

  • 55.

    Frank, S., Fürst, C., Koschke, L., Witt, A. & Makeschin, F. Assessment of landscape aesthetics—validation of a landscape metrics-based assessment by visual estimation of the scenic beauty. Ecol. Indic. 32, 222–231. https://doi.org/10.1016/j.ecolind.2013.03.026 (2013).

    Article 

    Google Scholar
     

  • 56.

    Graham, L. J. & Eigenbrod, F. Scale dependency in drivers of outdoor recreation in England. People Nat. 1, 406–416. https://doi.org/10.1002/pan3.10042 (2019).

    Article 

    Google Scholar
     

  • 57.

    Ryo, M. & Rillig, M. C. Statistically reinforced machine learning for nonlinear patterns and variable interactions. Ecosphere 8, e01976. https://doi.org/10.1002/ecs2.1976 (2017).

    Article 

    Google Scholar
     

  • 58.

    Karasov, O., Vieira, A. A. B., Külvik, M. & Chervanyov, I. Landscape coherence revisited: GIS-based mapping in relation to scenic values and preferences estimated with geolocated social media data. Ecol. Indic. 111, 105973. https://doi.org/10.1016/j.ecolind.2019.105973 (2020).

    Article 

    Google Scholar
     

  • 59.

    Foltête, J.-C., Ingensand, J. & Blanc, N. Coupling crowd-sourced imagery and visibility modelling to identify landscape preferences at the panorama level. Landsc. Urban Plan. 197, 103756. https://doi.org/10.1016/j.landurbplan.2020.103756 (2020).

    Article 

    Google Scholar
     

  • 60.

    Labib, S. M., Huck, J. J. & Lindley, S. Modelling and mapping eye-level greenness visibility exposure using multi-source data at high spatial resolutions. Sci. Total Environ. 755, 143050. https://doi.org/10.1016/j.scitotenv.2020.143050 (2021).

    ADS 
    Article 
    PubMed 
    CAS 

    Google Scholar
     

  • 61.

    Li, H. & Wu, J. Use and misuse of landscape indices. Landsc. Ecol. 19, 389–399. https://doi.org/10.1023/B:LAND.0000030441.15628.d6 (2004).

    Article 

    Google Scholar
     

  • 62.

    Weather, U. K. UK seasonal weather summary—Winter 2009/2010. Weather 65, 99. https://doi.org/10.1002/wea.601 (2010).

    Article 

    Google Scholar
     

  • 63.

    Lenormand, M. et al. Multiscale socio-ecological networks in the age of information. PLOS ONE 13, e0206672. https://doi.org/10.1371/journal.pone.0206672 (2018).

    Article 
    PubMed 
    PubMed Central 
    CAS 

    Google Scholar
     

  • 64.

    Li, L., Goodchild, M. F. & Xu, B. Spatial, temporal, and socioeconomic patterns in the use of Twitter and Flickr. Cartogr. Geogr. Inf. Sci. 40, 61–77. https://doi.org/10.1080/15230406.2013.777139 (2013).

    Article 
    CAS 

    Google Scholar
     

  • 65.

    Uuemaa, E., Mander, Ü. & Marja, R. Trends in the use of landscape spatial metrics as landscape indicators: A review. Ecol. Indic. 28, 100–106. https://doi.org/10.1016/j.ecolind.2012.07.018 (2013).

    Article 

    Google Scholar
     

  • 66.

    Daniel, T. C. Whither scenic beauty? Visual landscape quality assessment in the 21st century. Landsc. Urban Plan. 54, 267–281. https://doi.org/10.1016/S0169-2046(01)00141-4 (2001).

    Article 

    Google Scholar