Skip to main content
Springer Nature Link
Log in

PVII: A pedestrian-vehicle interactive and iterative prediction framework for pedestrian’s trajectory

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Advanced Driving Assistance System (ADAS) can predict pedestrian’s trajectory, in order to avoid traffic accidents and guarantee driving safety. A few current pedestrian trajectory prediction methods use a pedestrian’s historical motion to predict the future trajectory, but the pedestrian’s trajectory is also affected by the vehicle using the ADAS for prediction (target vehicle). Other studies predict the pedestrian’s and vehicle’s trajectories separately, and use the latter to adjust the former, but their interaction is a continuous process and should be considered during prediction rather than after. Therefore, we propose PVII, a pedestrian-vehicle interactive and iterative prediction framework for pedestrian’s trajectory. It makes prediction for one iteration based on the results from previous iteration, which essentially models the vehicle-pedestrian interaction. In this iterative framework, to avoid accumulation of prediction errors along with the increased iterations, we design a bi-layer Bayesian en/decoder. For each iteration, it not only uses inaccurate results from previous iteration but also accurate historical data for prediction, and calculates Bayesian uncertainty values to evaluate the results. In addition, the pedestrian’s trajectory is affected by both target vehicle and other vehicles around it (surrounding vehicle), so we include into the framework a pre-trained speed estimation module for surrounding vehicles (SE module). It estimates the speed based on pedestrian’s motion and we collect data from pedestrian’s view for training. In experiments, PVII can achieve the highest prediction accuracy compared to the current methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Code and data

https://github.com/Quintusy/Vehicle-Pedestrian-Interaction-FrameworkPIE data: https://github.com/aras62/PIEJAAD data: https://github.com/ykotseruba/JAAD

References

  1. Louwerse WJR, Hoogendoorn SP (2004) Adas safety impacts on rural and urban highways. In: IEEE Intell Veh Symp 2004, IEEE, pp 887–890

  2. Masello L, Castignani G, Sheehan B, Murphy F, McDonnell K (2022) On the road safety benefits of advanced driver assistance systems in different driving contexts. Trans Res Interdiscip Perspect 15:100670

    Google Scholar 

  3. Zhang L, Yuan K, Chu H, Huang Y, Ding H, Yuan J, Chen H (2021) Pedestrian collision risk assessment based on state estimation and motion prediction. IEEE Trans Veh Technol 71(1):98–111

    Article  Google Scholar 

  4. Pan R, Jie L, Zhang X, Pang S, Wang H, Wei Z (2022) A v2p collision risk warning method based on lstm in iov. Secur and Commun Netw 2022

  5. Rasouli A, Kotseruba I, Tsotsos JK (2017) Agreeing to cross: How drivers and pedestrians communicate. In: 2017 IEEE Intell Veh Symp (IV), IEEE, pp 264–269

  6. Rasouli A, Tsotsos JK (2019) Autonomous vehicles that interact with pedestrians: A survey of theory and practice. IEEE Trans Intell Trans Syst 21(3):900–918

    Article  Google Scholar 

  7. Camara F, Bellotto N, Cosar S, Weber F, Nathanael D, Althoff M, Wu J, Ruenz J, Dietrich A, Markkula G et al (2020) Pedestrian models for autonomous driving part ii: high-level models of human behavior. IEEE Trans Intell Trans Syst 22(9):5453–5472

    Article  Google Scholar 

  8. Bouhsain SA, Saadatnejad S, Alahi A (2020) Pedestrian intention prediction: A multi-task perspective. Tech Rep

  9. Sui Z, Zhou Y, Zhao X, Chen A, Ni Y (2021) Joint intention and trajectory prediction based on transformer. In: 2021 IEEE/RSJ International conference on intelligent robots and systems (IROS), IEEE, pp 7082–7088

  10. Yao Y, Atkins E, Johnson-Roberson M, Vasudevan R, Du X (2021) Bitrap: Bi-directional pedestrian trajectory prediction with multi-modal goal estimation. IEEE Robot Autom Lett 6(2):1463–1470

  11. Czech P, Braun M, Kreßel U, Yang B (2022) On-board pedestrian trajectory prediction using behavioral features. In: 2022 21st IEEE International conference on machine learning and applications (ICMLA), IEEE, pp 437–443

  12. Wang C, Wang Y, Xu M, Crandall DJ (2022) Stepwise goal-driven networks for trajectory prediction. IEEE Robot Autom Lett 7(2):2716–2723

    Article  Google Scholar 

  13. Bhattacharyya A, Fritz M, Schiele B (2018) Long-term on-board prediction of people in traffic scenes under uncertainty. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4194–4202

  14. Rasouli A, Kotseruba I, Kunic T, Tsotsos JK (2019) Pie: A large-scale dataset and models for pedestrian intention estimation and trajectory prediction. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 6262–6271

  15. Malla S, Dariush B, Choi C (2020) Titan: Future forecast using action priors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11186–11196

  16. Kim K, Lee YK, Ahn H, Hahn S, Oh S (2020) Pedestrian intention prediction for autonomous driving using a multiple stakeholder perspective model. In: 2020 IEEE/RSJ International conference on intelligent robots and systems (IROS), IEEE, pp 7957–7962

  17. Neumann L, Vedaldi A (2021) Pedestrian and ego-vehicle trajectory prediction from monocular camera. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10204–10212

  18. Ruijie Q, YuWu LZ, Yi Y (2021) Holistic lstm for pedestrian trajectory prediction. IEEE Trans Image Process 30:3229–3239

    Article  Google Scholar 

  19. Bengio S, Vinyals O, Jaitly N, Shazeer N (2015) Scheduled sampling for sequence prediction with recurrent neural networks. Adv Neural Inform Process Syst 28

  20. Lamb AM, Goyal AGAP, Zhang Y, Zhang S, Courville AC, Bengio Y (2016) Professor forcing: A new algorithm for training recurrent networks. Adv Neural Inform Process Syst 29

  21. Zhang W, Feng Y, Liu Q (2021) Bridging the gap between training and inference for neural machine translation. In: Proceedings of the twenty-ninth international conference on international joint conferences on artificial intelligence, pp 4790–4794

  22. Schmidt S, Faerber B (2009) Pedestrians at the kerb-recognising the action intentions of humans. Transport Res F: Traffic Psychol Behav 12(4):300–310

    Article  Google Scholar 

  23. Alvarez WM, Moreno FM, Sipele O, Smirnov N, Olaverri-Monreal C (2020) Autonomous driving: Framework for pedestrian intention estimation in a real world scenario. In: 2020 IEEE Intelligent vehicles symposium (IV), IEEE, pp 39–44

  24. Yao Y, Xu M, Choi C, Crandall DJ, Atkins EM, Dariush Behzad (2019) Egocentric vision-based future vehicle localization for intelligent driving assistance systems. In: 2019 International conference on robotics and automation (ICRA), IEEE, pp 9711–9717

  25. Kendall A, Gal Y (2017) What uncertainties do we need in bayesian deep learning for computer vision? Adv Neural Inform Process Syst 30

  26. Costante G, Mancini M (2020) Uncertainty estimation for data-driven visual odometry. IEEE Trans Robot 36(6):1738–1757

    Article  Google Scholar 

  27. Blei DM, Kucukelbir A, McAuliffe JD (2017) Variational inference: A review for statisticians. J Am Stat Assoc 112(518):859–877

    Article  MathSciNet  Google Scholar 

  28. Gal Y, Ghahramani Z (2016) Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In: International conference on machine learning, PMLR, pp 1050–1059

  29. Rasouli A, Kotseruba I, Tsotsos JK (2017) Are they going to cross? a benchmark dataset and baseline for pedestrian crosswalk behavior. In: Proceedings of the IEEE international conference on computer vision workshops, pp 206–213

Download references

Funding

This work was funded by the National Natural Science Foundation of China (No. 62172028).

Author information

Author notes

  1. Qianwen Shen and Shien Huang contributed equally to this work.

Authors and Affiliations

  1. School of Software Engineering, Beijing Jiaotong University, No. 3 Shangyuan Residence, Haidian District, 100044, Beijing, China

    Qianwen Shen, Shien Huang & Ergude Bao

  2. School of Informatics, Computing, and Engineering, Indiana University Bloomington, 700 N Woodlawn Ave, Bloomington, 47408, Indiana, USA

    Baixi Sun & Dingwen Tao

  3. School of Electrical Engineering and Computer Science, Washington State University, 106 Van Doren, Pullman, 99164, Washington, USA

    Xinyu Chen

  4. School of Computer and Information Technology, Beijing Jiaotong University, No. 3 Shangyuan Residence, Haidian District, 100044, Beijing, China

    Huaiyu Wan

Authors
  1. Qianwen Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shien Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Baixi Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xinyu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dingwen Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Huaiyu Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ergude Bao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Dingwen Tao, Huaiyu Wan or Ergude Bao.

Ethics declarations

Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shen, Q., Huang, S., Sun, B. et al. PVII: A pedestrian-vehicle interactive and iterative prediction framework for pedestrian’s trajectory. Appl Intell 54, 9881–9891 (2024). https://doi.org/10.1007/s10489-024-05595-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05595-8

Keywords

Search

Navigation