Whole Home Voice Control

Problem

Typing is not always the best way to get information (e.g. what’s the weather?) or give commands (e.g. turn the kitchen lights off!) in the application.

Solution

The idea is to create a Voice Assistant to control devices in the SeaPod and ask for information that is available in the Ocean Builder’s applications.

The voice assistant should be smart enough to recognize users and be able to follow commands, considering users’ settings and permissions. For example, when the user asks to turn on the shower then the shrower will be turned on and adjusted automatically based on the user's preferences, but if the user doesn’t have permission to use that shower then the command will be ignored.

Prize

Get credited as a Project Contributor to the Ocean Builders Project

Turn this into your own entrepreneurial business venture and we will be your first customers and help bring you media attention and customers

Get Entrepreneurial Business Coaching to start this as a business

And here are some potential benefits:

Mass exposure with highly visible project

Build reputation

Recognized as an official collaborator/ and/or on Github

Get noticed

Product development experience

Work on projects you are passionate about

Get your project built and working in the real world

Participate in interesting work

Get grants (maybe partner with someone that can help with this or exposure to grant writers)

Change the world

Industry

Current technological level

You’ve probably heard of voice assistants like Alexa, Siri, Google Assistant, and Cortana. These voice assistants are essentially based on voice recognition, NLP, and synthesis of speech (see picture below).

There are also many open source projects like:

Mycroft

OpenAssistant

Jasper

LinTO

Rhasspy

Aimybox

Leon

Many of these open source voice assistants have come into existence quite recently and will probably take some time to develop into a more sophisticated solution.

The problem with the majority of these platforms is that they are not local and not private enough.

Some projects such as MyCroft offer solutions built around Google Home or Alexa. However, certain characteristics of these systems - no data protection and no business vocabulary adaptation - limit them to a B2C market that is not (yet) concerned by data sensitivity and criticality issues.

There are also some platforms like LinTo, that embraces these challenges from the start in order to be the engine that catapults your professional product.

One of the biggest challenges might be to implement a voice authenticator. Here are some projects to check out to see if any of them could be a fit to integrate with our system:

Here's a website about open source projects. I brought out some more interesting projects below:

https://awesomeopensource.com/project/pyannote/pyannote-audio

https://codeocean.com/capsule/7271435/tree/v1

https://github.com/mravanelli/pytorch-kaldi

https://awesomeopensource.com/project/google/uis-rnn

https://alize.univ-avignon.fr/

It has a Java version as well.

Information

Repository

<text>

License Requirement

Open Source: Can be used for private or commercial projects

Software: GNU General Public License (GNU GPL V3) here

Non-Software: Creative Commons (CC BY-SA 4.0) here

Project Areas

IoT Development (sensors, arduino and raspberry pi)

Software Development (python)?

<text>

Keywords: <text>

Project requirements

Stages and deadlines

Project Start	date
Team Formed	date
Market Research Summary (Report)	date
Project Plan Complete	date
Preliminary Product Design Complete	date
Prototype Development Complete	date
Prototype Evaluation Complete	date
Product Presentation	date
Project Completion	date

Project plan should cover the following:

stages / milestones of a project (not all stages are brought out in a table above)

activities or tasks in each phase

task start and end dates

interdependencies between tasks

Also:

skills needed

responsibilities of each team member (identify as many as you can).

Product’s general requirements

https://docs.google.com/spreadsheets/d/1u0Ca9NZvKY6ex5JPtpl8M-HoaM-K8VBF4W4NGoJWlSo/edit?usp=sharing

(Will remove URL before publishing)

Basic	Advanced	Function
		Part I
		Can it identify people via voice ID?
		Can it easily understand people's accents?
		Does it know users preferences of using home appliances?
		Can it adjust devices settings based on the users' preferences?
		Can it take commands only from people who have permission?
		Can you set permissions for commands and information? e.g. select who can open doors.
		Is the data sandboxed so personal data is not going to public cloud for AI/ML?
		Can you switch from online/offline queries.

		Part II
		Can I ask about all the information that is available in the Ocean Builders user app?
		Can I ask about all the information that is available in the Ocean Builders admin panel app?
		Can I give all the commands that are available in Ocean Builders user app?
		Can I give all the commands that are available in Ocean Builders admin panel app?
		Does it support <text> language?

Tips

Below you can find some examples of tools to use to build Voice Assistance:

gTTS (Google Text-to-Speech) is a speech synthesis library to convert text to speech.

SpeechRecognition is a library for performing speech recognition, with support for several engines and APIs, online and offline

Sphinx is the offline recognition engine called by the SpeechRecognition library.

Packt is a voice recognition library to identify the person who is speaking

Voice Authentication

https://courses.csail.mit.edu/6.857/2016/files/31.pdf

Not open source:

https://docs.google.com/document/d/1V-cyxivxKFXwYVUO21oleuXcAoXzR8QTKkpI5ug6E0A/edit

Project video link:

https://www.dropbox.com/s/j44y0z574mt1ohh/VoiceControl.mp4?dl=0