Untitled Document

EVA

Innovative Claims:

1. We propose to develop an interactive head mounted display and camera stereoscopic video and binaural audio communications system which is worn by a user in the field and transmits said signals to a remote base location. These signals are time-base corrected, multiplexed and video overlayed (chroma-keyed) with hand gestures, text, photos, video or other vital data and instantly transmitted back to the user in the field in real-time; thus providing the user with an enhanced 3-D video reality and data to perform task oriented operations. In addition, the receiving parties can view a remote location and medical victims as accurately as if they were on the scene and provide assistance via the wearer in the field. Deliverables: 1. Integrated head-mounted display, stereoscopic micro-camera and binaural audio system.

2. Dual-channel video and dual-channel audio wearable microwave transmission and reception system with a 2 mile radius for communication.

3. Portable base station with abilities to time-base correct (field-lock) dual discrete video signals, field interlaced multiplexing, and chroma key video overlay. Base staion will have capabilities to receive and transmit signals via microwave to the user in the field and via satellite to anyone anywhere having web-based connectivity.

Key Personnel:

1. Dr. Dave Warner, M.D., Ph.D.
2. Bill Rojas, Ph.D. - EVA Systems Developer
3. Mathew Carbone
4. Tim Murphy
5. David Balch
6. Patrick Keller

Summary:

The EVA System allows for a shared experience between the wearer and a person or persons at a remote location or locations. By transmitting as true a representation of reality as is electronically possible, this system provides the wearer with an enhanced view of his current location, while providing a remote person(s) with an accurate 3-D representation of a location at a distance. In so doing, the wearer and the receiving parties can interact “as one” to provide assistance which individually, because of knowledge or distance, neither could accomplish. EVA System is adaptable to interface with other medical and information gathering technologies, such as GPS, and EKG, EEG, Pulse/Oxymetry or other vital signs which could be acquired and transmitted as an integrated microwave signal along with stereo-video and binaural audio. In conjunction with satellite and internet based connectivity, EVA System can fulfill the goal of true Distributed Medical Intelligence.

EVA System:

1) Interface: The interface consists of a head mounted stereoscopic display integrated with high resolution micro-video cameras and binaural audio microphones which is worn by a user in the field. The acquired video and audio signals are transmitted to a remote receiving station which is manned by a person wearing a head mounted display who sees and hears the sights and sounds of the user in the field. The person receiving the transmitted signals can interact with the wearer by chroma-key overlay of hand gestures thus visually pointing -out physical objects, specific body parts or anything else in the wearers field of view.

1.1) Possible scenarios: A soldier is dispatched into an open desolate terrain, plumes of smoke off in the distance indicate over night fighting between opposing factions in the city’s civil war has been intense. As a steady stream of the displaced, injured and confused refugees parade by, our warrior captures the scene with a head-mounted stereo-video camera and binaural audio systems and display (EVA). These 3-D images and localizing audio are transmitted back to Base Camp via wireless microwave where field surgeons evaluate the situation. Suddenly, the soldier sees the surgeons hand come into view in his HMD and is guided to a young man bend over in pain, the soldier places his sensor glove on the victim and begins to transmit medical data, temperature: 104, pulse; 127, blood-ox: 83. The surgeon directs the soldier to ask the victim a few questions in a foreign language, which he quickly repeats. Fever, malaise, cough and sore throat are diagnosed, the alert base camp team then directs the soldier to secure a throat culture and asks the victim if he can draw a blood sample. The sample is drawn and a drop of serum is quickly prepared on a glass slide. The soldier pushes a button on his waist and the cameras on his head zooms into the sample on the glass slide. The lymphocyte count appears reduced @1000 cell/mm3. An urgent message is relayed from base camp and a specialist in toxicology is located at ECU in North Carolina. Information is uplinked to SAT/COM via LAN and specialists put on their Head Mounted Displays.

2) Information Communication: Information, both video and audio, is communicated via microwave spread-spectrum transmitters and receivers. This communication system forms the core of the EVA System which can be modularly expanded to include Satellite up and down link, and internet connectivity. Both the wearer and receiver persons will have real-time audio and video interactivity with abilities to communicate in foreign languages, or technically difficult areas such as medical biology, explosive demolitions, etc.

3) Training: Training for personnel will be minimal since the human visual and auditory system can acclimate to being immersed in a 3-D video and audio environment quite readily.

Summary of the System:

1) Integrated HMD-C/M: Head mounted Display - Cameras and Mics
2) Microwave Transmitters and Receivers: Spread Spectrum
3) Base Station - TBC, Multiplexing, Chroma Key, Satellite

EVA, Experiential Video and Audio, is a 3-D video and audio interactive telepresence system. Using two micro cameras and microphones on a head mounted display, EVA collects and transmits signals to a base camp where the 3-D video sources are instantly time base corrected, multiplexed and overlayed with vital information and transmitted back to the wearer in the field in real time. This allows two persons in remote locations to share information and expedite assistance in the form of hand gestures, text or video and ultimately to "act as one".

Statement of Work:

Task 1 Research and develop a high resolution head mounted display and camera/microphone system.

R&D:
1) HMD
2) Micro-Video cameras
3) Mini-Microphones

Task 2 - Obtain and integrate spread spectrum microwave transmitters and receivers.

R&D:
1) 2 transmitters and 1 receiver on wearer
2) 2 receivers and 1 transmitter at base station

Task 3 - Obtain and integrate vital components of the base station.

R&D:
1) TBC - dual channel time base corrector
2) Multiplexer - field sequential
3) Chroma Key + video camera
4) Distribution Amplifier
5) Head Mounted Display
6) Satellite/Internet up/downlink

Task 4 - Research and develop a vest based power supply with rechargeable lithium-ion batteries.

R&D:
1) Flat lithium -ion bateries.
2) Vest designs - ergonomic power supply
3) Alternative - Solar, Fuel Cell, Hydrogen Peroxide, etc.

Task 5 - Systems testing and endurance predictability.

R&D:
1) All weather ability - a) Rain b) Snow c) Dirt and Mud
2) Time of day - a) Day b) Night - lights, night vision, starlight, etc.
3) Distance and Inteference - a) two mile radius b) Microwave alignments c) Jamming interference - field detector and ambient EMF levels
4) Satellite up/down link and Internet connectivity - a) Rapid deployment b) Portability

Task 6 - Education and community outreach.

1) Educational: Robotic TeleMedicine
2) Space Exploration: NASA EVA EVA
3) Police Department:
4) Fire/Paramedic Department:

Integration Into the Real World: Operation Strong Angel - June 10 - 16 - Wiamea, Hawaii Humanitarian assistance portion of RIMPAC 2000 Reviewed by: * NAVY 3rd Fleet * DARPA * ECU Tel-Med * SPAWAR * Japanese Defense Navy * NASA Medical Systems