Skip to main content

Whole Home Voice Control

image02-VoiceContol.jpg

Gesture Control for Lights and Audio

Problem

Typing is not always the best way to get information (e.g. what’s the weather?) or give commands (e.g. turn the kitchen lights off!) in the application. <text>

 

Solution

 

The idea is to createdevelop a Voiceuser Assistantprogrammable gesture-controlled home system, where people can program their own gestures to control deviceslights, inaudio, theemergency commands and SeaPod and ask for information that is available in the Ocean Builder’s applications. steering.

 

The voicesystem assistanthas shouldto be smart enough to recognizerecognise users and beknow abletheir permissions to follow commands, considering users’ settings and permissions.commands. For example, when the userowner asksmight want to turngive onSeaPod thesteering showeraccess thento thea shrower will be turned on and adjusted automatically based on the user's preferences,friend but ifnot thefor user doesn’t have permission to use that shower then the command will be ignored. kids.



Also, the system has to be secure so there’s no data leakage possibilities. 

Prize

  • - Get credited as a Project Contributor to the Ocean Builders Project

    • Turn this into your own entrepreneurial business venture and we will be your first customers and help bring you media attention and customers

    • Get Entrepreneurial Business Coaching to start this as a business



    And here are some potential benefits:

    • Mass exposure with highly visible project

      • Build reputation

      • Recognized as an official collaborator/ and/or on Github

      • Get noticed

      • Product development experience

      • Work on projects you are passionate about

      • Get your project built and working in the real world

      • Participate in interesting work

      • Get grants (maybe partner with someone that can help with this or exposure to grant writers)

      • Change the world

       

      Industry

      Current technological level

       

      You’ve probably heard of voice assistants like Alexa, Siri, Google Assistant, and Cortana. These voice assistants are essentially based on voice recognition, NLP, and synthesis of speech (see picture below).

       

      There are alsoa manyfew gesture-controlled home systems in the market like Single Cue, that lets you control your home media devices with your finger movements and Fibaro Swipe. Unfortunately their reviews are not so good. 

      Below, there are listed some open source projects like:that might be helpful for your R&D: 

       Teachable Machine is a web-based tool that you can use to train a computer to recognize your own images, sounds, & poses.

      1. MycroftHere you can learn how to make a gesture-controlled Arduino robot using PictoBlox AI’s machine learning feature.

        2. In

      2. this video, the developer demonstrates how to build a gesture recognition system and use it to control the media player on the computer. OpenCV + Machine Learning + GUI Automation were used to do this.

        OpenAssistantHere is a project where Alexa is responding to sign language using webcam and TensorFlow.js

      3. JasperHere

        4. you
      4. can

        LinTO

        read
        5. an
      5. article

        Rhasspy

        about
        6. how

      6. Aimybox

      7. Leon

       

      Many of thesean open source voiceGoogle assistantssign havelanguage comeAI turns hand gestures into existence quite recently and will probably take some time to develop into a more sophisticated solution.speech.

       

      In

      this tutorial (see Thehere), problemyou withcan thelearn majorityhow ofto thesebuild platformsan isAmerican thatSign theyLanguage aretranslator notusing localcomputer vision and not private enough.



      Some projects such as MyCroft offer solutions built around Google Home or Alexa. However, certain characteristics of these systems - no data protection and no business vocabulary adaptation - limit them to a B2Cmachine marketlearning that is not (yet) concerned by data sensitivity and criticality issues.model.

       

      There are also some platforms like LinTo, that embraces these challenges from the start in order to be the engine that catapults your professional product.

       

      One of the biggest challenges might be to implement a voice authenticator. Here are some projects to check out to see if any of them could be a fit to integrate with our system:

       

      Here'sMediaPipe Hands is a websitehigh-fidelity abouthand openand sourcefinger projects.tracking Isolution. broughtIt outemploys somemachine morelearning interesting(ML) projectsto below:

      infer

       

      21

      https://awesomeopensource.com/project/pyannote/pyannote-audio

      3D

       

      landmarks

      https://codeocean.com/capsule/7271435/tree/v1

      of

       

      a

      https://github.com/mravanelli/pytorch-kaldi

      hand

       

      from

      https://awesomeopensource.com/project/google/uis-rnn

      just

       

      a

      https://alize.univ-avignon.fr/

      single

      Itframe. MediaPipe has aalso Javaother versionsolutions asto well. detect objects like faces, iris, poses and more.



      Information

       

      Repository

      <text>

       

      License Requirement

      Open Source: Can be used for private or commercial projects

      Software:Software: GNU General Public License (GNU GPL V3) here

      Non-Software: Creative Commons (CC BY-SA 4.0) here

       

      Project Areas

      • IoT DevelopmentSoftware (sensors,ML, arduinoAI, andComputer raspberryVisualization, pi)Neural Network)

      • Software Development (python)?More?

      • <text>

      Objectives



      First part of the project should be a research on what gestures to use for particular commands. 

      Keywords:The <text>second part of the project is to develop a gesture control system for lights, audio, emergency and steering.



      Project’s
      Project requirementsRequirements

       

      Stages and deadlines

       

      Project Start

      date

      Team Formed

      date

      Market Research Summary (Report)

      date

      Project Plan Complete

      date

      Preliminary Product Design Complete

      date

      Prototype Development Complete

      date

      Prototype Evaluation Complete

      date

      Product Presentation

      date

      Project Completion

      date

       

      Project plan should cover the following:

       

      • stages / milestones of a project (not all stages are brought out in a table above)

      • activities or tasks in each phase

      • task start and end dates

      • interdependencies between tasks

       

      Also:

       

      • skills needed

      • responsibilities of each team member (identify as many as you can).

       Preliminary product design should cover the following:

      • production components, raw material
      • system block diagram (This diagram specifies each electronic function and how all of the functional components interconnect).
      • preliminary Bill of Materials (BOM)
      • production cost estimation
      • performance 
      • features 
      • development feasibility 
      • manufacturability

      Product’s general requirements

      https://docs.google.com/spreadsheets/d/1u0Ca9NZvKY6ex5JPtpl8M-HoaM-K8VBF4W4NGoJWlSo/edit?usp=sharing1X7GzTJoPGgzrKe34GE-U0_3xLz8_-9yF66NEi7J3djc/edit#gid=0

      (Will remove URL before publishing)publishing)



      thanremote

      Basic

      Advanced

      FunctionFeature
         

      PartCan Iyou control the audio system?

        

      Can you control lights?

        

      Can you control TV?

        

      Can you control SeaPod steering?

        

      Can you control functions for emergencies?

        

      Is it user programmable (users can program their own gestures)?

        

      Can the system recognize users (visual identity)?

        

      Can you select who can have access to particular home control functions?

        

      Can the user create its own gestures?

        

      Is the data sandboxed to protect privacy?
         

      Can it identifyrecognize peoplegestures viaat voiceany ID?angle?

        

      Can it recognize gestures in the dark?
         

      Can it easily understand people's accents?

        

      Does it know users preferences of using home appliances?

        

      Can it adjust devices settings based on the users' preferences?

        

      Can it take commands only from people who have permission?

        

      Can you set permissions for commands and information? e.g. select who can open doors.gestures?
         

      Is theit datauser sandboxed so personal data is not goingfriendly to publicset cloud for AI/ML?up?
         

      CanIs youit switchmore fromconvenient online/offlineto queries.

      use
         
        

      Part IIcontrol?
         

      Can I askuse about all the information that is availableit in theany Ocean Builders user app?room?
         

      Can I askuse aboutit allin any location of the information that is available in the Ocean Builders admin panel app?

        

      Can I give all the commands that are available in Ocean Builders user app?

        

      Can I give all the commands that are available in Ocean Builders admin panel app?

        

      Does it support <text> language?room?

      Collaboration instructions



      It’s up to each team how to break down the project into stages & milestones but it should include at least stages that are brought out in the Project Requirements chapter.

      TipsIt’s also necessary to open an account in Trello, to manage your project. Please create at least 3 columns that are as follows: 

       1) To do

      2) Doing

      3) Done

      And give access to your team lead and Ocean Builders manager as well, so that your team lead can keep track of your progress, comment on Trello cards and also have weekly team meetings. Below you can find sometheir examples of tools to use to build Voice Assistance:

      contacts: 

      gTTS (Google Text-to-Speech) is a  speech synthesis library to convert text to speech.

       

      SpeechRecognition is a library for performing speech recognition, with support for several engines and APIs, online and offline

       

      Sphinx is the offline recognition engine called by the SpeechRecognition library.

       

      Packt is a voice recognition library to identify the person who is speaking 

       Team lead:

      VoiceName: Authentication <text>

      https://courses.csail.mit.edu/6.857/2016/files/31.pdfEmail:

       <text>

      NotOcean openBuilders source:manager:

       Name: <text>

      https://docs.google.com/document/d/1V-cyxivxKFXwYVUO21oleuXcAoXzR8QTKkpI5ug6E0A/editEmail:

       <text>

      Project video link:

      https://www.dropbox.com/s/j44y0z574mt1ohh/VoiceControl.mp4?dl=0

Back to top