Language-Driven Interactive Traffic Trajectory Generation

Junkai Xia, Chenxin Xu, Qingyao Xu, Chen Xie, Yanfeng Wang, Siheng Chen·May 24, 2024

Summary

The paper presents InteractTraj, a novel language-driven traffic trajectory generator for autonomous vehicles that addresses the limitations of previous methods by focusing on interactive dynamics. It uses a language-to-code encoder with an interaction-aware strategy and a code-to-trajectory decoder for controllable and realistic trajectory generation. InteractTraj outperforms state-of-the-art models like TrafficGen and LCTGen in terms of realism, controllability, and user preference, as demonstrated through experiments on the Waymo Open Motion Dataset and nuPlan. The system converts natural language descriptions into interaction, vehicle, and map codes, enabling the generation of more complex and interactive traffic scenarios for driving simulation and autonomous vehicle data generation. The model's success lies in its ability to bridge language descriptions to realistic traffic scenarios by considering multi-agent interactions and using large language models like GPT-4.

Paper digest

What problem does the paper attempt to solve? Is this a new problem?

The paper aims to address the problem of language-driven traffic trajectory generation, which involves creating realistic trajectories for traffic participants based on natural language descriptions . This problem involves leveraging Large Language Models (LLMs) like GPT-4 to convert language input into detailed codes that contain information about interactions, vehicles, and the map, enabling the generation of corresponding traffic trajectories that align with the language description .

This problem is not entirely new, as previous works have explored related aspects such as trajectory generation based on predefined control signals and the use of human natural language to control trajectory properties . However, the paper introduces a novel approach that focuses on interaction-aware code representation and refined vehicle behavior control to address the limitations of existing methods in handling complex text descriptions and interaction awareness .


What scientific hypothesis does this paper seek to validate?

The scientific hypothesis that the paper seeks to validate is related to the use of human natural language to achieve more flexible and user-friendly control in the context of traffic scenario generation models. Specifically, the paper aims to validate the hypothesis that leveraging large language models can transform text descriptions into structured representations, enabling the generation of corresponding scenarios in a more flexible and user-friendly manner . The focus is on converting language descriptions into multi-level codes and generating trajectories through attention-based information aggregation to effectively reproduce real-life scenario distribution aligned with language descriptions .


What new ideas, methods, or models does the paper propose? What are the characteristics and advantages compared to previous methods?

The paper proposes a novel model called InteractTraj for language-driven interactive traffic trajectory generation. This model converts language descriptions into multi-level codes and generates trajectories through attention-based information aggregation. It introduces several innovative elements:

  • Interaction Codes: The paper introduces interaction codes, which are derived from language descriptions and facilitate the generation of realistic trajectories by capturing the interactions between multiple traffic participants .
  • Language-Guided Generation: InteractTraj leverages language conditions to generate traffic trajectories, enhancing controllability and reducing the reliance on extensive datasets .
  • Attention-Based Information Aggregation: The model utilizes attention mechanisms for aggregating information from the interaction codes to generate scenarios aligned with the provided language descriptions .
  • Flexible Control Using Language: Unlike previous methods that rely on pre-defined control signals, InteractTraj uses human natural language to offer more flexible and user-friendly control over desired trajectory properties .
  • Multi-Level Codes: By converting language descriptions into multi-level codes, the model captures detailed information that enables the generation of diverse and realistic traffic scenarios .
  • Innovative Design Elements: The paper evaluates the effectiveness of various design elements, such as whole interaction codes, relative distance, and relative position in interaction codes, showing their benefits in enhancing trajectory generation realism .

These novel ideas and methods contribute to advancing the field of traffic trajectory generation by incorporating language-driven approaches, interaction-aware modeling, and attention mechanisms for more accurate and controllable scenario generation. InteractTraj, the proposed model for language-driven interactive traffic trajectory generation, introduces several key characteristics and advantages compared to previous methods outlined in the paper :

  • Interaction Codes: InteractTraj incorporates interaction codes, LLM prompts, and interaction-aware feature aggregation, enabling the model to generate realistic interactive traffic trajectories aligned with language descriptions .
  • Controllability and Flexibility: The model achieves high controllability by generating traffic trajectories based on language conditions, reducing the reliance on extensive datasets, and offering more flexible control using human natural language .
  • Attention Mechanisms: InteractTraj utilizes attention-based information aggregation to convert language descriptions into multi-level codes, facilitating the generation of diverse and realistic traffic scenarios .
  • Improved Performance: In quantitative evaluations, InteractTraj outperforms previous methods by significantly reducing mean Average Displacement Error (mADE) and mean Final Displacement Error (mFDE) across all metrics, demonstrating its capability to generate more realistic scenarios with vehicle interactions .
  • User Preference: User studies show that InteractTraj has a 47.5% higher average user preference compared to baseline methods, indicating its effectiveness in generating trajectories that align well with user expectations .
  • Comprehensive Information: Unlike previous methods that lack interaction information in their inputs, InteractTraj incorporates interaction details during the generation process, resulting in more comprehensive inputs for reconstruction-based evaluation and enabling more realistic and effective trajectory generation .
  • Innovative Design Elements: The model's novel interaction interpretation mechanism with LLM in the language-to-code encoder and two-step feature aggregation enhance the coherence and realism of generated trajectories, setting it apart from traditional rule-based and learning-based methods .

These characteristics and advantages collectively position InteractTraj as a pioneering approach in language-driven traffic trajectory generation, offering improved controllability, realism, and alignment with user descriptions compared to existing methods.


Do any related researches exist? Who are the noteworthy researchers on this topic in this field?What is the key to the solution mentioned in the paper?

Several related research papers exist in the field of language-driven interactive traffic trajectory generation. Noteworthy researchers in this field include:

  • Rongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren, and others
  • Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and others
  • Chenxin Xu, Robby T. Tan, Yuhong Tan, Siheng Chen, and others
  • Junkai Xia, Chenxin Xu, Qingyao Xu, and others

The key to the solution mentioned in the paper is the development of InteractTraj, which is the first language-driven traffic trajectory generator capable of producing interactive traffic trajectories. This system interprets abstract trajectory descriptions into interaction-aware numerical codes and then learns a mapping between these codes and the final interactive trajectories. The solution involves a language-to-code encoder with an interaction-aware encoding strategy and a code-to-trajectory decoder with interaction-aware feature aggregation to incorporate vehicle interactions with the environmental map and vehicle movements, resulting in more realistic and controllable generation of interactive traffic trajectories .


How were the experiments in the paper designed?

The experiments in the paper were designed with specific settings and methodologies:

  • Datasets: The experiments utilized the WOMD and nuPlan datasets for training and testing, with a specific number of scenarios selected for each dataset .
  • Baseline Models: Two existing controllable trajectory generation baselines, TrafficGen and LCTGen, were considered for comparison. These models were adjusted to standardize the number of vehicles and the length of predicted trajectories for a fair comparison .
  • Experimental Setup: The language-to-code encoder discretized inter-vehicle distances and speeds at specific intervals, while the vehicles' trajectories were sampled at regular intervals to generate trajectories. Various components like MCG blocks, transformers, and MLPs were used in the experimental setup for trajectory generation .
  • Evaluation Metrics: The experiments evaluated the realism of generated trajectories using six metrics, including mean average displacement error, minimum average displacement error, mean final displacement error, minimum final displacement error, scenario collision rate, and Hausdorff distance. These metrics were used to measure the similarity between predicted and ground truth trajectories .

What is the dataset used for quantitative evaluation? Is the code open source?

The dataset used for quantitative evaluation in the study is the Waymo Open Motion Dataset (WOMD) and nuPlan . The code for the experiment is open-source, as the study mentions using two existing controllable trajectory generation baselines, TrafficGen and LCTGen, which are open-sourced .


Do the experiments and results in the paper provide good support for the scientific hypotheses that need to be verified? Please analyze.

The experiments and results presented in the paper provide strong support for the scientific hypotheses that needed to be verified. The paper introduces InteractTraj, a novel interaction-aware language-guided traffic scenario generation model that effectively converts language descriptions into multi-level codes and generates trajectories through attention-based information aggregation . The ablation study conducted in the paper evaluates the effectiveness of proposed designs, including the addition of interaction codes, relative distance, and relative position, showing that all designs contribute to a more realistic trajectory generation . Additionally, the study on the setting of hyper-parameters demonstrates that the current parameter choices achieve the best results, further supporting the effectiveness of the proposed approach .

Furthermore, the paper compares the end-to-end method with previous approaches by analyzing trajectories generated from language descriptions. Qualitative evaluation and user studies are employed for assessment due to the absence of specific ground-truth trajectories for certain language commands. The comparison with the LCTGen model, which also transforms language input into traffic scenarios, shows that InteractTraj better aligns with language descriptions, especially in scenarios involving different types of interactions like vehicle overtaking, merging, yielding, and following. This comparison highlights the effectiveness of the interaction-aware code representation in generating scenarios that closely match the language descriptions, supporting the scientific hypotheses of the paper .


What are the contributions of this paper?

The paper "Language-Driven Interactive Traffic Trajectory Generation" proposes a novel model called InteractTraj, which focuses on generating traffic scenarios guided by language descriptions. The key contributions of this paper include:

  • Converting language descriptions into multi-level codes for generating trajectories through attention-based information aggregation.
  • Conducting an ablation study to evaluate the effectiveness of proposed designs, such as whole interaction codes, relative distance, and relative position in interaction codes .
  • Demonstrating that the InteractTraj model effectively reproduces real-life scenario distribution and aligns generated scenarios with language descriptions.
  • Highlighting the limitations of the work, which currently focuses on generating trajectories for vehicles only and is limited by the map library. Future work aims to extend the model to include more types of traffic participants and flexible map generation, as well as applying the generated traffic scenarios to train autonomous driving systems by expanding the motion dataset .

What work can be continued in depth?

To delve deeper into the research, further exploration can be conducted in the following areas:

  • Extending to More Traffic Participants: The current work focuses on generating trajectories for vehicles only. Future research can expand to include other types of traffic participants such as pedestrians, cyclists, or public transport vehicles to create a more comprehensive traffic scenario .
  • Enhancing Map Generation: The generation of maps in the current model is limited by the map library. Future work can focus on developing more flexible and detailed map generation techniques to enrich the realism of the generated traffic scenarios .
  • Application in Autonomous Driving Systems: Expanding the utilization of the generated traffic scenarios for training autonomous driving systems can be a promising direction. This involves incorporating the generated scenarios into the training data for autonomous vehicles to enhance their decision-making and planning capabilities .

Introduction
Background
Evolution of traffic trajectory generation for AVs
Limitations of existing methods (lack of interaction, controllability)
Objective
To develop a novel system for interactive dynamics
Improve realism, controllability, and user preference
Method
Language-to-Code Encoder
Interaction-Aware Strategy
Multi-agent interaction modeling
Contextual understanding of language descriptions
Architecture and Components
GPT-4 integration for language understanding
Encoding process for natural language to interaction, vehicle, and map codes
Code-to-Trajectory Decoder
Generation of realistic traffic scenarios
Control over trajectory parameters
Decoding process for code to trajectory output
Performance Metrics
Realism evaluation (Waymo Open Motion Dataset, nuPlan)
Controllability assessment
User preference studies
Experiments and Evaluation
Dataset and Setup
Waymo Open Motion Dataset
nuPlan dataset for diverse scenarios
Baseline Comparison
TrafficGen
LCTGen
Comparison of InteractTraj's performance
Applications
Driving simulation
Autonomous vehicle data generation
Advantages for training and testing AV systems
Conclusion
Advancements in language-driven trajectory generation
Significance for autonomous vehicle development
Future directions and potential improvements
Basic info
papers
robotics
artificial intelligence
Advanced features
Insights
How does InteractTraj differ from previous methods in addressing the limitations of traffic trajectory generation?
Which datasets were used to evaluate the performance of InteractTraj compared to TrafficGen and LCTGen?
What is the primary focus of InteractTraj, the traffic trajectory generator presented in the paper?
What are the key components of InteractTraj's architecture, specifically the language-to-code encoder and code-to-trajectory decoder?

Language-Driven Interactive Traffic Trajectory Generation

Junkai Xia, Chenxin Xu, Qingyao Xu, Chen Xie, Yanfeng Wang, Siheng Chen·May 24, 2024

Summary

The paper presents InteractTraj, a novel language-driven traffic trajectory generator for autonomous vehicles that addresses the limitations of previous methods by focusing on interactive dynamics. It uses a language-to-code encoder with an interaction-aware strategy and a code-to-trajectory decoder for controllable and realistic trajectory generation. InteractTraj outperforms state-of-the-art models like TrafficGen and LCTGen in terms of realism, controllability, and user preference, as demonstrated through experiments on the Waymo Open Motion Dataset and nuPlan. The system converts natural language descriptions into interaction, vehicle, and map codes, enabling the generation of more complex and interactive traffic scenarios for driving simulation and autonomous vehicle data generation. The model's success lies in its ability to bridge language descriptions to realistic traffic scenarios by considering multi-agent interactions and using large language models like GPT-4.
Mind map
User preference studies
Controllability assessment
Realism evaluation (Waymo Open Motion Dataset, nuPlan)
Encoding process for natural language to interaction, vehicle, and map codes
GPT-4 integration for language understanding
Contextual understanding of language descriptions
Multi-agent interaction modeling
Comparison of InteractTraj's performance
LCTGen
TrafficGen
nuPlan dataset for diverse scenarios
Waymo Open Motion Dataset
Performance Metrics
Architecture and Components
Interaction-Aware Strategy
Improve realism, controllability, and user preference
To develop a novel system for interactive dynamics
Limitations of existing methods (lack of interaction, controllability)
Evolution of traffic trajectory generation for AVs
Future directions and potential improvements
Significance for autonomous vehicle development
Advancements in language-driven trajectory generation
Advantages for training and testing AV systems
Autonomous vehicle data generation
Driving simulation
Baseline Comparison
Dataset and Setup
Code-to-Trajectory Decoder
Language-to-Code Encoder
Objective
Background
Conclusion
Applications
Experiments and Evaluation
Method
Introduction
Outline
Introduction
Background
Evolution of traffic trajectory generation for AVs
Limitations of existing methods (lack of interaction, controllability)
Objective
To develop a novel system for interactive dynamics
Improve realism, controllability, and user preference
Method
Language-to-Code Encoder
Interaction-Aware Strategy
Multi-agent interaction modeling
Contextual understanding of language descriptions
Architecture and Components
GPT-4 integration for language understanding
Encoding process for natural language to interaction, vehicle, and map codes
Code-to-Trajectory Decoder
Generation of realistic traffic scenarios
Control over trajectory parameters
Decoding process for code to trajectory output
Performance Metrics
Realism evaluation (Waymo Open Motion Dataset, nuPlan)
Controllability assessment
User preference studies
Experiments and Evaluation
Dataset and Setup
Waymo Open Motion Dataset
nuPlan dataset for diverse scenarios
Baseline Comparison
TrafficGen
LCTGen
Comparison of InteractTraj's performance
Applications
Driving simulation
Autonomous vehicle data generation
Advantages for training and testing AV systems
Conclusion
Advancements in language-driven trajectory generation
Significance for autonomous vehicle development
Future directions and potential improvements

Paper digest

What problem does the paper attempt to solve? Is this a new problem?

The paper aims to address the problem of language-driven traffic trajectory generation, which involves creating realistic trajectories for traffic participants based on natural language descriptions . This problem involves leveraging Large Language Models (LLMs) like GPT-4 to convert language input into detailed codes that contain information about interactions, vehicles, and the map, enabling the generation of corresponding traffic trajectories that align with the language description .

This problem is not entirely new, as previous works have explored related aspects such as trajectory generation based on predefined control signals and the use of human natural language to control trajectory properties . However, the paper introduces a novel approach that focuses on interaction-aware code representation and refined vehicle behavior control to address the limitations of existing methods in handling complex text descriptions and interaction awareness .


What scientific hypothesis does this paper seek to validate?

The scientific hypothesis that the paper seeks to validate is related to the use of human natural language to achieve more flexible and user-friendly control in the context of traffic scenario generation models. Specifically, the paper aims to validate the hypothesis that leveraging large language models can transform text descriptions into structured representations, enabling the generation of corresponding scenarios in a more flexible and user-friendly manner . The focus is on converting language descriptions into multi-level codes and generating trajectories through attention-based information aggregation to effectively reproduce real-life scenario distribution aligned with language descriptions .


What new ideas, methods, or models does the paper propose? What are the characteristics and advantages compared to previous methods?

The paper proposes a novel model called InteractTraj for language-driven interactive traffic trajectory generation. This model converts language descriptions into multi-level codes and generates trajectories through attention-based information aggregation. It introduces several innovative elements:

  • Interaction Codes: The paper introduces interaction codes, which are derived from language descriptions and facilitate the generation of realistic trajectories by capturing the interactions between multiple traffic participants .
  • Language-Guided Generation: InteractTraj leverages language conditions to generate traffic trajectories, enhancing controllability and reducing the reliance on extensive datasets .
  • Attention-Based Information Aggregation: The model utilizes attention mechanisms for aggregating information from the interaction codes to generate scenarios aligned with the provided language descriptions .
  • Flexible Control Using Language: Unlike previous methods that rely on pre-defined control signals, InteractTraj uses human natural language to offer more flexible and user-friendly control over desired trajectory properties .
  • Multi-Level Codes: By converting language descriptions into multi-level codes, the model captures detailed information that enables the generation of diverse and realistic traffic scenarios .
  • Innovative Design Elements: The paper evaluates the effectiveness of various design elements, such as whole interaction codes, relative distance, and relative position in interaction codes, showing their benefits in enhancing trajectory generation realism .

These novel ideas and methods contribute to advancing the field of traffic trajectory generation by incorporating language-driven approaches, interaction-aware modeling, and attention mechanisms for more accurate and controllable scenario generation. InteractTraj, the proposed model for language-driven interactive traffic trajectory generation, introduces several key characteristics and advantages compared to previous methods outlined in the paper :

  • Interaction Codes: InteractTraj incorporates interaction codes, LLM prompts, and interaction-aware feature aggregation, enabling the model to generate realistic interactive traffic trajectories aligned with language descriptions .
  • Controllability and Flexibility: The model achieves high controllability by generating traffic trajectories based on language conditions, reducing the reliance on extensive datasets, and offering more flexible control using human natural language .
  • Attention Mechanisms: InteractTraj utilizes attention-based information aggregation to convert language descriptions into multi-level codes, facilitating the generation of diverse and realistic traffic scenarios .
  • Improved Performance: In quantitative evaluations, InteractTraj outperforms previous methods by significantly reducing mean Average Displacement Error (mADE) and mean Final Displacement Error (mFDE) across all metrics, demonstrating its capability to generate more realistic scenarios with vehicle interactions .
  • User Preference: User studies show that InteractTraj has a 47.5% higher average user preference compared to baseline methods, indicating its effectiveness in generating trajectories that align well with user expectations .
  • Comprehensive Information: Unlike previous methods that lack interaction information in their inputs, InteractTraj incorporates interaction details during the generation process, resulting in more comprehensive inputs for reconstruction-based evaluation and enabling more realistic and effective trajectory generation .
  • Innovative Design Elements: The model's novel interaction interpretation mechanism with LLM in the language-to-code encoder and two-step feature aggregation enhance the coherence and realism of generated trajectories, setting it apart from traditional rule-based and learning-based methods .

These characteristics and advantages collectively position InteractTraj as a pioneering approach in language-driven traffic trajectory generation, offering improved controllability, realism, and alignment with user descriptions compared to existing methods.


Do any related researches exist? Who are the noteworthy researchers on this topic in this field?What is the key to the solution mentioned in the paper?

Several related research papers exist in the field of language-driven interactive traffic trajectory generation. Noteworthy researchers in this field include:

  • Rongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren, and others
  • Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and others
  • Chenxin Xu, Robby T. Tan, Yuhong Tan, Siheng Chen, and others
  • Junkai Xia, Chenxin Xu, Qingyao Xu, and others

The key to the solution mentioned in the paper is the development of InteractTraj, which is the first language-driven traffic trajectory generator capable of producing interactive traffic trajectories. This system interprets abstract trajectory descriptions into interaction-aware numerical codes and then learns a mapping between these codes and the final interactive trajectories. The solution involves a language-to-code encoder with an interaction-aware encoding strategy and a code-to-trajectory decoder with interaction-aware feature aggregation to incorporate vehicle interactions with the environmental map and vehicle movements, resulting in more realistic and controllable generation of interactive traffic trajectories .


How were the experiments in the paper designed?

The experiments in the paper were designed with specific settings and methodologies:

  • Datasets: The experiments utilized the WOMD and nuPlan datasets for training and testing, with a specific number of scenarios selected for each dataset .
  • Baseline Models: Two existing controllable trajectory generation baselines, TrafficGen and LCTGen, were considered for comparison. These models were adjusted to standardize the number of vehicles and the length of predicted trajectories for a fair comparison .
  • Experimental Setup: The language-to-code encoder discretized inter-vehicle distances and speeds at specific intervals, while the vehicles' trajectories were sampled at regular intervals to generate trajectories. Various components like MCG blocks, transformers, and MLPs were used in the experimental setup for trajectory generation .
  • Evaluation Metrics: The experiments evaluated the realism of generated trajectories using six metrics, including mean average displacement error, minimum average displacement error, mean final displacement error, minimum final displacement error, scenario collision rate, and Hausdorff distance. These metrics were used to measure the similarity between predicted and ground truth trajectories .

What is the dataset used for quantitative evaluation? Is the code open source?

The dataset used for quantitative evaluation in the study is the Waymo Open Motion Dataset (WOMD) and nuPlan . The code for the experiment is open-source, as the study mentions using two existing controllable trajectory generation baselines, TrafficGen and LCTGen, which are open-sourced .


Do the experiments and results in the paper provide good support for the scientific hypotheses that need to be verified? Please analyze.

The experiments and results presented in the paper provide strong support for the scientific hypotheses that needed to be verified. The paper introduces InteractTraj, a novel interaction-aware language-guided traffic scenario generation model that effectively converts language descriptions into multi-level codes and generates trajectories through attention-based information aggregation . The ablation study conducted in the paper evaluates the effectiveness of proposed designs, including the addition of interaction codes, relative distance, and relative position, showing that all designs contribute to a more realistic trajectory generation . Additionally, the study on the setting of hyper-parameters demonstrates that the current parameter choices achieve the best results, further supporting the effectiveness of the proposed approach .

Furthermore, the paper compares the end-to-end method with previous approaches by analyzing trajectories generated from language descriptions. Qualitative evaluation and user studies are employed for assessment due to the absence of specific ground-truth trajectories for certain language commands. The comparison with the LCTGen model, which also transforms language input into traffic scenarios, shows that InteractTraj better aligns with language descriptions, especially in scenarios involving different types of interactions like vehicle overtaking, merging, yielding, and following. This comparison highlights the effectiveness of the interaction-aware code representation in generating scenarios that closely match the language descriptions, supporting the scientific hypotheses of the paper .


What are the contributions of this paper?

The paper "Language-Driven Interactive Traffic Trajectory Generation" proposes a novel model called InteractTraj, which focuses on generating traffic scenarios guided by language descriptions. The key contributions of this paper include:

  • Converting language descriptions into multi-level codes for generating trajectories through attention-based information aggregation.
  • Conducting an ablation study to evaluate the effectiveness of proposed designs, such as whole interaction codes, relative distance, and relative position in interaction codes .
  • Demonstrating that the InteractTraj model effectively reproduces real-life scenario distribution and aligns generated scenarios with language descriptions.
  • Highlighting the limitations of the work, which currently focuses on generating trajectories for vehicles only and is limited by the map library. Future work aims to extend the model to include more types of traffic participants and flexible map generation, as well as applying the generated traffic scenarios to train autonomous driving systems by expanding the motion dataset .

What work can be continued in depth?

To delve deeper into the research, further exploration can be conducted in the following areas:

  • Extending to More Traffic Participants: The current work focuses on generating trajectories for vehicles only. Future research can expand to include other types of traffic participants such as pedestrians, cyclists, or public transport vehicles to create a more comprehensive traffic scenario .
  • Enhancing Map Generation: The generation of maps in the current model is limited by the map library. Future work can focus on developing more flexible and detailed map generation techniques to enrich the realism of the generated traffic scenarios .
  • Application in Autonomous Driving Systems: Expanding the utilization of the generated traffic scenarios for training autonomous driving systems can be a promising direction. This involves incorporating the generated scenarios into the training data for autonomous vehicles to enhance their decision-making and planning capabilities .
Scan the QR code to ask more questions about the paper
© 2025 Powerdrill. All rights reserved.