SalsaBot

SalsaBot is an innovative embodied agent, specifically developed for the first Alexa Prize Simbot Challenge. The agent’s core function is to guide users through a game in a virtual setting, focusing on a seamless, user-centric experience. It operates in complex environments, relying on visual perception and natural language to interact with diverse Alexa users to accomplish intricate tasks.

User-Centric Design & Innovative features

The design of SalsaBot emphasizes adaptability to various user backgrounds and interaction styles. It includes:

• Catering to both novice and experienced users by balancing advanced capabilities and user guidance.
• Handling a wide range of language expressions and command abstractions from a global user base.
• Ensuring high planning accuracy to maintain user trust and prevent irreversible errors.
• Systematic analysis of user interactions to understand and respond to different user needs and patterns.
• Introduction of ‘macros’: High-level actions encompassing essential elements for goal achievement, adaptable to various scenarios and user command levels.
• Advanced object localization methods with a memory module for revisiting seen objects and a systematic search for unseen ones.
• State-aware dialogue generation, providing proactive guidance and diverse, engaging responses.

Continuous Improvement and User Feedback

Use of real-time user feedback for ongoing enhancements. A focus on generalizability to new environments, games, and user types, avoiding short-term, game-specific solutions. Consistent high user ratings, often ranking in the top 5, demonstrating the success of our user-centric, innovative approach.