| Sonstiges: |
- Nachgewiesen in: USPTO Patent Applications
- Sprachen: English
- Document Number: 20240161740
- Publication Date: May 16, 2024
- Appl. No: 18/055395
- Application Filed: November 14, 2022
- Assignees: Google LLC (Mountain View, CA, US)
- Claim: 1. A computer-implemented method when executed on data processing hardware causes the data processing hardware to perform operations comprising: for each respective digital assistant in a group of digital assistants enabled for simultaneous execution on a multi-assistant device (MAD), receiving a respective active set of warm words that each specify a respective action for the respective digital assistant to perform; based on the respective active set of warm words associated with each digital assistant in the group of digital assistants, executing, by a multi-assistant interface executing on the MAD, a warm word arbitration routine to enable a final set of warm words for detection by the MAD, each corresponding warm word in the final set of warm words enabled for detection by the MAD is selected from the respective active set of warm words for at least one digital assistant in the group of digital assistants; and while the final set of warm words are enabled for detection by the MAD: receiving audio data corresponding to an utterance captured by the MAD; detecting, in the audio data, a warm word from the final set of warm words; and instructing, from the group of digital assistants, the digital assistant associated with the detected warm word to perform the respective action specified by the detected warm word.
- Claim: 2. The computer-implemented method of claim 1, wherein receiving the respective active set of warm words associated with the digital assistant comprises receiving, for at least one warm word in the respective active set of warm words, via a warm word application programming interface (API) executing on the MAD, a respective warm word model configured to detect the corresponding warm word in streaming audio without performing speech recognition.
- Claim: 3. The computer-implemented method of claim 1, wherein the operations further comprise, for a corresponding one of the digital assistants in the group of digital assistants: receiving a user command specifying a long-standing operation for the corresponding digital assistant to perform; performing, via the corresponding digital assistant, the long-standing operation specified by the user command, wherein receiving the respective active set of warm words associated with the digital assistant comprises receiving, in response to the corresponding digital assistant performing the long-standing operation, the respective active set of warm words associated with the corresponding digital assistant.
- Claim: 4. The computer-implemented method of claim 3, wherein each warm word in the respective active set of warm words is associated with a respective action for controlling the long-standing operation performed by the corresponding digital assistant.
- Claim: 5. The computer-implemented method of claim 1, wherein the operations further comprise: discovering a new digital assistant in the group of digital assistants enabled for simultaneous execution on the MAD, wherein the multi-assistant interface executes the warm word arbitration routine in response to discovering the new digital assistant in the group of digital assistants.
- Claim: 6. The computer-implemented method of claim 1, wherein the operations further comprise: determining that a digital assistant has been removed from the group of digital assistants enabled for simultaneous execution on the MAD, wherein the multi-assistant interface executes the warm word arbitration routine in response to determining that the digital assistant has been removed from the group of digital assistants.
- Claim: 7. The computer-implemented method of claim 1, wherein the operations further comprise, for a corresponding one of the digital assistants in the group of digital assistants: determining an addition of a warm word or a removal of a warm word in the respective active set of warm words associated with the corresponding digital assistant, wherein the multi-assistant interface executes the warm word arbitration routine in response to determining the addition of the warm word or the removal of the warm word in the respective set of warm words.
- Claim: 8. The computer-implemented method of claim 1, wherein the operations further comprise: determining a change in ambient context, wherein the multi-assistant interface executes the warm word arbitration routine in response to determining the change in ambient context.
- Claim: 9. The computer-implemented method of claim 1, wherein the operations further comprise: obtaining enabled warm word constraints, the enabled warm word constraints comprising at least one of: memory and computing resource availability on the MAD for detection of warm words; computational requirements for enabling each warm word in respective active set of warm words associated with each respective digital assistant in the group of digital assistants; an acceptable false accept rate tolerance; or an acceptable false reject rate tolerance, wherein a number of the warm words in the final set of warm words enabled for detection by the MAD is based on the obtained enabled warm word constraints.
- Claim: 10. The computer-implemented method of claim 1, wherein executing the warm word arbitration routine comprises: identifying any shared warm words corresponding to warm words present in at least two of the active sets of warm words; and determining the final set of warm words is based on assigning a higher priority to warm words identified as shared warm words.
- Claim: 11. The computer-implemented of claim 1, wherein executing the warm word arbitration routine comprises: for each warm word in the respective active set of warm words for each respective digital assistant in the group of digital assistants, determining a frequency of detection of the warm word by the MAD; and determining the final set of warm words is based on the determined frequency of detection of each warm word in the respective active set of warm words for each respective digital assistant in the group of digital assistants.
- Claim: 12. The computer-implemented method of claim 1, wherein executing the warm word arbitration routine comprises: for each warm word in the respective active set of warm words for each respective digital assistant in the group of digital assistants, determining a time when the warm word was most recently detected by the MAD; and determining the final set of warm words is based on the determined time that each warm word in the respective active set of warm words for each respective digital assistant in the group of digital assistants was most recently detected.
- Claim: 13. The computer-implemented method of claim 1, wherein the operations further comprise: receiving a voice command that commands the MAD to enable a first digital assistant and a second digital assistant to execute simultaneously on the MAD, the voice command spoken by a user of the MAD and captured by the MAD in streaming audio; and after receiving the voice command, enabling the first digital assistant and the second digital assistant to execute simultaneously with one another on the MAD, wherein the group of digital assistants comprises the first digital assistant and the second digital assistant.
- Claim: 14. The computer-implemented method of claim 1, wherein the operations further comprise: receiving, from a software application executing on the MAD or another device in communication with the MAD, a multi-assistant configuration request to enable a first digital assistant and a second digital assistant to execute simultaneously on the MAD; and after receiving the multi-assistant configuration request, enabling the first digital assistant and the second digital assistant to execute simultaneously with one another on the MAD, wherein the group of digital assistants comprises the first digital assistant and the second digital assistant.
- Claim: 15. A system comprising: data processing hardware; and memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising: for each respective digital assistant in a group of digital assistants enabled for simultaneous execution on a multi-assistant device (MAD), receiving a respective active set of warm words that each specify a respective action for the respective digital assistant to perform; based on the respective active set of warm words associated with each digital assistant in the group of digital assistants, executing, by a multi-assistant interface executing on the MAD, a warm word arbitration routine to enable a final set of warm words for detection by the MAD, each corresponding warm word in the final set of warm words enabled for detection by the MAD is selected from the respective active set of warm words for at least one digital assistant in the group of digital assistants; and while the final set of warm words are enabled for detection by the MAD: receiving audio data corresponding to an utterance captured by the MAD; detecting, in the audio data, a warm word from the final set of warm words; and instructing, from the group of digital assistants, the digital assistant associated with the detected warm word to perform the respective action specified by the detected warm word.
- Claim: 16. The system of claim 15, wherein receiving the respective active set of warm words associated with the digital assistant comprises receiving, for at least one warm word in the respective active set of warm words, via a warm word application programming interface (API) executing on the MAD, a respective warm word model configured to detect the corresponding warm word in streaming audio without performing speech recognition.
- Claim: 17. The system of claim 15, wherein the operations further comprise, for a corresponding one of the digital assistants in the group of digital assistants: receiving a user command specifying a long-standing operation for the corresponding digital assistant to perform; performing, via the corresponding digital assistant, the long-standing operation specified by the user command, wherein receiving the respective active set of warm words associated with the digital assistant comprises receiving, in response to the corresponding digital assistant performing the long-standing operation, the respective active set of warm words associated with the corresponding digital assistant.
- Claim: 18. The system of claim 17, wherein each warm word in the respective active set of warm words is associated with a respective action for controlling the long-standing operation performed by the corresponding digital assistant.
- Claim: 19. The system of claim 15, wherein the operations further comprise: discovering a new digital assistant in the group of digital assistants enabled for simultaneous execution on the MAD, wherein the multi-assistant interface executes the warm word arbitration routine in response to discovering the new digital assistant in the group of digital assistants.
- Claim: 20. The system of claim 15, wherein the operations further comprise: determining that a digital assistant has been removed from the group of digital assistants enabled for simultaneous execution on the MAD, wherein the multi-assistant interface executes the warm word arbitration routine in response to determining that the digital assistant has been removed from the group of digital assistants.
- Claim: 21. The system of claim 15, wherein the operations further comprise, for a corresponding one of the digital assistants in the group of digital assistants: determining an addition of a warm word or a removal of a warm word in the respective active set of warm words associated with the corresponding digital assistant, wherein the multi-assistant interface executes the warm word arbitration routine in response to determining the addition of the warm word or the removal of the warm word in the respective set of warm words.
- Claim: 22. The system of claim 15, wherein the operations further comprise: determining a change in ambient context, wherein the multi-assistant interface executes the warm word arbitration routine in response to determining the change in ambient context.
- Claim: 23. The system of claim 15, wherein the operations further comprise: obtaining enabled warm word constraints, the enabled warm word constraints comprising at least one of: memory and computing resource availability on the MAD for detection of warm words; computational requirements for enabling each warm word in respective active set of warm words associated with each respective digital assistant in the group of digital assistants; an acceptable false accept rate tolerance; or an acceptable false reject rate tolerance, wherein a number of the warm words in the final set of warm words enabled for detection by the MAD is based on the obtained enabled warm word constraints.
- Claim: 24. The system of claim 15, wherein executing the warm word arbitration routine comprises: identifying any shared warm words corresponding to warm words present in at least two of the active sets of warm words; and determining the final set of warm words is based on assigning a higher priority to warm words identified as shared warm words.
- Claim: 25. The system of claim 15, wherein executing the warm word arbitration routine comprises: for each warm word in the respective active set of warm words for each respective digital assistant in the group of digital assistants, determining a frequency of detection of the warm word by the MAD; and determining the final set of warm words is based on the determined frequency of detection of each warm word in the respective active set of warm words for each respective digital assistant in the group of digital assistants.
- Claim: 26. The system of claim 15, wherein executing the warm word arbitration routine comprises: for each warm word in the respective active set of warm words for each respective digital assistant in the group of digital assistants, determining a time when the warm word was most recently detected by the MAD; and determining the final set of warm words is based on the determined time that each warm word in the respective active set of warm words for each respective digital assistant in the group of digital assistants was most recently detected.
- Claim: 27. The system of claim 15, wherein the operations further comprise: receiving a voice command that commands the MAD to enable a first digital assistant and a second digital assistant to execute simultaneously on the MAD, the voice command spoken by a user of the MAD and captured by the MAD in streaming audio; and after receiving the voice command, enabling the first digital assistant and the second digital assistant to execute simultaneously with one another on the MAD, wherein the group of digital assistants comprises the first digital assistant and the second digital assistant.
- Claim: 28. The system of claim 15, wherein the operations further comprise: receiving, from a software application executing on the MAD or another device in communication with the MAD, a multi-assistant configuration request to enable a first digital assistant and a second digital assistant to execute simultaneously on the MAD; and after receiving the multi-assistant configuration request, enabling the first digital assistant and the second digital assistant to execute simultaneously with one another on the MAD, wherein the group of digital assistants comprises the first digital assistant and the second digital assistant.
- Current International Class: 10; 10
|
