Pulling the strings: How stem cells in the developing fruit fly brain organize their DNA from outside the nucleus

Cells are very particular about the proper locations of their contents. For example, DNA is packaged in the nucleus in a highly specific way, but 91自拍ers are still trying to understand how this happens. For stem cells in the brain of a larval fruit fly 鈥� model organisms used to study processes such as early development 鈥� the DNA is organized so that the center of each strand, called the centromere, is at the top of the nucleus. In a paper in the Journal of Cell Biology, UW 91自拍ers describe how proteins outside the nucleus work alongside proteins inside the nucleus to control the position of centromeres. The 91自拍ers observed this mechanism by imaging stem cells within larval fruit fly brains and then using machine learning to measure how centromere locations change when specific proteins are removed. The 91自拍 team also followed the locations of two specific genes that are far away from the centromeres. Both genes lost their usual positions in the nucleus when the centromeres strayed from their normal location, suggesting that proper centromere positioning may be important for orienting the rest of the DNA in the nucleus. Studying how and why DNA is organized in these stem cells can help 91自拍ers better understand how stem cells are regulated and how brains develop.

For more information, contact lead author Jennifer Taylor, UW postdoc in biology, at jatay@uw.edu.听听听听听听听

The other UW co-author is . A full list of funding is .

Micro language models let AI run instantly on small devices like smartwatches

Though artificial intelligence can refer to many technologies, the current AI boom has been driven by large language models, or LLMs. They power chatbots and AI agents by running predictions based on ever-increasing amounts of variables called 鈥減arameters.鈥� Current cutting-edge models are estimated to have several trillion parameters, making them far too large to run on slower phones or small devices like smartwatches. In a paper at EMNLP 2026, UW 91自拍ers introduced 鈥渕icro language models,鈥� which run with 8-30 million parameters. These models are small enough to run on devices with limited computing power and give users instant responses that fill the lag before the device can access an LLM through the cloud. For instance, a user could ask their smartwatch a question and the micro language model would start responding instantly; then after four to eight words, the LLM would complete the response. The micro language model made mistakes in 4% to 8% of responses that the other model corrected seamlessly before continuing. In a user study, nine out of 10 participants preferred the paired system to the lagging responses from only a cloud model.

For more information, contact ulm@cs.washington.edu or visit .

The UW co-authors are , , and . A full list of co-authors and funding is in .

Policy gaps impact migrant fishers in Washington state

The commercial fishing industry is key to Washington鈥檚 maritime economy, but there are concerns about exploitation and forced labor among fishers. While much of industry is regulated by the Washington Department of Fish and Wildlife, exemptions in U.S. law mean some vessels aren鈥檛 subject to the same citizenship requirements. A 91自拍 team from the UW Harry Bridges Center for Labor Studies recently focused on understanding the experiences of migrant fishers pursuing albacore tuna, a highly migratory species, who dock or unload at ports in Washington state. Researchers found the exemptions allow migrant fishers to work without authorization or visas, leaving them vulnerable to forced labor and exploitation. Meanwhile, complicated policy and regulatory frameworks make it difficult to address workers鈥� rights violations. The 91自拍ers gathered information from 15 migrant fishers, 14 U.S. citizen fishers, and more than 35 subject matter experts. Every migrant fisher interviewed by the team reported experiences aligned with the International Labour Organization鈥檚 indicators of forced labor, and both migrant and U.S. citizen fishers reported difficult living and working conditions. Approaches to combating forced labor are varied and uneven, the 91自拍ers found, and additional work is needed to better understand and support the needs of Washington鈥檚 fishers. Along with the full report, the team also published .

For more information, contact , 91自拍 director at the Bridges Center, at rerstad@uw.edu.

The other UW co-authors are , and .

Burrowing shrimp are on the move in a mudflat near you

Burrowing shrimp are a pest well known to shellfish growers. These pale-bodied creatures excavate mud as they dig into the ground, creating piles of sediment that suffocate oysters and clams growing on the surface. In estuaries on the west coast, shrimp beds reach densities of hundreds of shrimp per square meter, but the boundaries of beds change. in Estuarine, Coastal and Shelf Science shows that shrimp beds move up to 100 feet each year. The study took place at Willapa Bay over the course of three years and is the first to quantify how rapidly the boundaries of these beds shift over time. Researchers still don鈥檛 know why the shrimp move, but they observed a clear pattern in the ecosystem. Shrimp change the texture of sediment by eating organic matter and releasing finer particles, so sediment becomes muddier and more compact once they leave. After several years, these abandoned beds were colonized by eelgrass, a marker of ecosystem health. This 鈥渆ngineering鈥� effect was less evident in areas where moving water keeps mud from settling. Studying these rapid shifts and the types of sediment left behind could ultimately help shellfish growers better protect their product.

For more information, contact co-author Jennifer Ruesink, a UW professor of biology, at ruesink@uw.edu.听听听听听听听

The other UW co-authors are Wesley Hull, Haleh Mawson, and . A full list of co-authors and funding is .

Source

91自拍 91自拍ers developed the first system that incorporates tiny cameras in off-the-shelf wireless earbuds to allow users to talk with an AI model about the scene in front of them. For instance, a user might turn to a Korean food package and say, 鈥淗ey Vue, translate this for me.鈥� They鈥檇 then hear an AI voice say, 鈥淭he visible text translates to 鈥楥old Noodles鈥� in English.鈥�

The prototype system called VueBuds takes low-resolution, black-and-white images, which it transmits over Bluetooth to a phone or other nearby device. A small artificial intelligence model on the device then answers questions about the images within around a second. For privacy, all of the processing happens on the device, a small light turns on when the system is recording, and users can immediately delete images.听

The team will April 14 at the Association for Computing Machinery Conference on Human Factors in Computing Systems in Barcelona.听

鈥淲e haven鈥檛 seen most people adopt smart glasses or VR headsets, in part because a lot of people don鈥檛 like wearing glasses, and they often come with , such as recording high-resolution video and processing it in the cloud,鈥� said senior author , a UW professor in the Paul G. Allen School of Computer Science & Engineering. 鈥淏ut almost everyone wears earbuds already, so we wanted to see if we could put visual intelligence into tiny, low-power earbuds, and also address privacy concerns in the process.鈥�

Cameras use far more power than the microphones already in earbuds, so using the same sort of high-res cameras as those in smart glasses wouldn鈥檛 work. Also, large amounts of information can鈥檛 stream continuously over Bluetooth, so the system can鈥檛 run continuous video.听

The team found that using a low-power camera 鈥� roughly the size of a grain of rice 鈥� to shoot low-resolution, black-and-white still images limited battery drain and allowed for Bluetooth transmission while preserving performance.

There was also the matter of placement.听

鈥淥ne big question we had was: Will your face obscure the view too much? Can earbud cameras capture the user鈥檚 view of the world reliably?鈥� said lead author , who completed this work as a UW doctoral student in the Allen School.听

The team found that angling each camera 5-10 degrees outward provides a 98-108 degree field of view. While this creates a small blind spot when objects are held closer than 20 centimeters from the user, people rarely hold things that close to examine them 鈥� making it a non-issue for typical interactions.

Sixteen participants also wore VueBuds and tested the system鈥檚 ability to translate and answer basic questions about objects. VueBuds achieved 83-84% accuracy when translating or identifying objects and 93% when identifying the author and title of a book.

This study was designed to gauge the feasibility of integrating cameras in wireless earbuds. Since the system only takes grayscale images, it can鈥檛 answer questions that involve color in the scene.听

The team wants to add color to the system 鈥� color cameras require more power 鈥� and to train specialized AI models for specific use cases, such as translation.听听

鈥淭his study lets us glimpse what鈥檚 possible just using a general purpose language model and our wireless earbuds with cameras,鈥� Kim said. 鈥淏ut we鈥檇 like to study the system more rigorously for applications like reading a book 鈥� for people who have low vision or are blind, for instance 鈥� or translating text for travelers.鈥澨�

Co-authors include , a UW master鈥檚 student in the Allen School, and , , , and , all UW students in electrical and computer engineering.听

For more information, contact vuebuds@cs.washington.edu.

Source

UPDATE (Dec. 12, 2025): This story has been updated to correct Malek Itani’s department.

Holding a conversation in a crowded room often leads to the frustrating 鈥渃ocktail party problem,鈥� or the challenge of separating the voices of conversation partners from a hubbub. It鈥檚 a mentally taxing situation that can be exacerbated by hearing impairment.听

As a solution to this common conundrum, 91自拍ers at the 91自拍 have developed that proactively isolate all the wearer鈥檚 conversation partners in a noisy soundscape. The headphones are powered by an AI model that detects the cadence of a conversation and another model that mutes any voices which don鈥檛 follow that pattern, along with other unwanted background noises. The prototype uses off-the-shelf hardware and can identify conversation partners using just two to four seconds of audio.

The system鈥檚 developers think the technology could one day help users of hearing aids, earbuds and smart glasses to filter their soundscapes without the need to manually direct the AI鈥檚 鈥渁ttention.鈥�

The team Nov. 7 in Suzhou, China at the Conference on Empirical Methods in Natural Language Processing. The underlying code is open-source and .

鈥淓xisting approaches to identifying who the wearer is listening to predominantly involve electrodes implanted in the brain to track attention,鈥� said senior author , a UW professor in the Paul G. Allen School of Computer Science & Engineering. 鈥淥ur insight is that when we鈥檙e conversing with a specific group of people, our speech naturally follows a turn-taking rhythm. And we can train AI to predict and track those rhythms using only audio, without the need for implanting electrodes.鈥�

The prototype system, dubbed 鈥減roactive hearing assistants,鈥� activates when the person wearing the headphones begins speaking. From there, one AI model begins tracking conversation participants by performing a 鈥渨ho spoke when鈥� analysis and looking for low overlap in exchanges. The system then forwards the result to a second model which isolates the participants and plays the cleaned up audio for the wearer. The system is fast enough to avoid confusing audio lag for the user, and can currently juggle one to four conversation partners in addition to the wearer鈥檚 audio.

The team tested the headphones with 11 participants, who rated qualities like noise suppression and comprehension with and without the AI filtration. Overall, the group rated the filtered audio more than twice as favorably as the baseline.听

Gollakota鈥檚 team has been experimenting with AI-powered hearing assistants for the past few years. They developed one smart headphone prototype that can pick a person鈥檚 audio out of a crowd when the wearer looks at them, and another that creates a 鈥渟ound bubble鈥� by muting all sounds within a set distance of the wearer.听

鈥淓verything we鈥檝e done previously requires the user to manually select a specific speaker or a distance within which to listen, which is not great for user experience,鈥� said lead author Guilin Hu, a doctoral student in the Allen School. 鈥淲hat we鈥檝e demonstrated is a technology that鈥檚 proactive 鈥� something that infers human intent noninvasively and automatically.鈥�

Plenty of work remains to refine the experience. The more dynamic a conversation gets, the more the system is likely to struggle, as participants talk over one another or speak in longer monologues. Participants entering and leaving a conversation present another hurdle, though Gollakota was surprised by how well the current prototype performed in these more complicated scenarios. The authors also note that the models were tested on English, Mandarin and Japanese dialog, and that the rhythms of other languages might require further fine-tuning.

The current prototype uses commercial over-the-ear headphones, microphones and circuitry. Eventually, Gollakota expects to make the system small enough to run on a tiny chip within an earbud or a hearing aid. In that appeared at , the authors demonstrated that it is possible to run AI models on tiny hearing aid devices.

Co-authors include听, a UW doctoral student in the Allen School; and , a UW doctoral student in the electrical and computer engineering department.

This 91自拍 was funded by the Moore Inventor Fellows program.

For more information, contact proactivehearing@cs.washington.edu

Source

, a 91自拍 doctoral student, recently toured a museum in Mexico. Chen doesn鈥檛 speak Spanish, so he ran a translation app on his phone and pointed the microphone at the tour guide. But even in a museum鈥檚 relative quiet, the surrounding noise was too much. The resulting text was useless.

Various technologies have emerged lately promising fluent translation, but none of these solved Chen鈥檚 problem of public spaces. , for instance, function only with an isolated speaker; they after the speaker finishes.

Now, Chen and a team of UW 91自拍ers have designed at once, while preserving the direction and qualities of people鈥檚 voices. The team built the system, called Spatial Speech Translation, with off-the-shelf noise-cancelling headphones fitted with microphones. The team鈥檚 algorithms separate out the different speakers in a space and follow them as they move, translate their speech and play it back with a 2-4 second delay.

The Apr. 30 at the ACM CHI Conference on Human Factors in Computing Systems in Yokohama, Japan. The code for the proof-of-concept device is available for others to build on. 鈥淥ther translation tech is built on the assumption that only one person is speaking,鈥� said senior author , a UW professor in the Paul G. Allen School of Computer Science & Engineering. 鈥淏ut in the real world, you can鈥檛 have just one robotic voice talking for multiple people in a room. For the first time, we鈥檝e preserved the sound of each person鈥檚 voice and the direction it鈥檚 coming from.鈥�

The system makes three innovations. First, when turned on, it immediately detects how many speakers are in an indoor or outdoor space.

鈥淥ur algorithms work a little like radar,鈥� said lead author Chen, a UW doctoral student in the Allen School. 鈥淪o it鈥檚 scanning the space in 360 degrees and constantly determining and updating whether there鈥檚 one person or six or seven.鈥�

The system then translates the speech and maintains the expressive qualities and volume of each speaker鈥檚 voice while running on a device, such mobile devices with an Apple M2 chip like laptops and Apple Vision Pro. (The team avoided using cloud computing because of the privacy concerns with voice cloning.) Finally, when speakers move their heads, the system continues to track the direction and qualities of their voices as they change.

The system functioned when tested in 10 indoor and outdoor settings. And in a 29-participant test, the users preferred the system over models that didn鈥檛 track speakers through space.

In a separate user test, most participants preferred a delay of 3-4 seconds, since the system made more errors when translating with a delay of 1-2 seconds. The team is working to reduce the speed of translation in future iterations. The system currently only works on commonplace speech, not specialized language such as technical jargon. For this paper, the team worked with Spanish, German and French 鈥� but previous work on translation models has shown they can be trained to translate around 100 languages.

鈥淭his is a step toward breaking down the language barriers between cultures,鈥� Chen said. 鈥淪o if I鈥檓 walking down the street in Mexico, even though I don鈥檛 speak Spanish, I can translate all the people鈥檚 voices and know who said what.鈥�

, a 91自拍 intern at HydroX AI and a UW undergraduate in the Allen School while completing this 91自拍, and , a UW doctoral student in the Allen School, are also co-authors on this paper. This 91自拍 was funded by a Moore Inventor Fellow award and a .

For more information, contact the 91自拍ers at babelfish@cs.washington.edu.听

Source

While smart devices in homes have grown to include speakers, security systems, lights and thermostats, the ways to control them have remained relatively stable. Users can interact with a phone, or talk to the tech, but these are frequently less convenient than the simple switches they replace: 鈥淭urn on the lamp鈥�. Not that one鈥�. Turn up the speaker volume鈥�. Not that loud!鈥�

91自拍 91自拍ers have developed IRIS, a smart ring that allows users to control smart devices by aiming the ring鈥檚 small camera at the device and clicking a built-in button. The prototype Bluetooth ring sends an image of the selected device to the user鈥檚 phone, which controls the device. The user can adjust the device with the button and 鈥� for devices with gradient controls, such as a speaker鈥檚 volume 鈥� by rotating their hand. IRIS, or Interactive Ring for Interfacing with Smart home devices, operates off a charge for 16-24 hours.

The team Oct. 16 at the 37th Annual ACM Symposium on User Interface Software and Technology in Pittsburgh. IRIS is not currently available to the public.

鈥淰oice commands can often be really cumbersome,鈥� said co-lead author , a UW doctoral student in the Paul G. Allen School of Computer Science & Engineering. 鈥淲e wanted to create something that鈥檚 as simple and intuitive as clicking on an icon on your computer desktop.鈥�

The team decided to put the system in a ring because they believed users would realistically wear that throughout the day. The challenge, then, was integrating a camera into a wireless smart ring with its size and power constraints. The system also had to toggle devices in under a second; otherwise, users tend to think it is not working.

To achieve this, 91自拍ers had the ring compress the images before sending them to a phone. Rather than streaming images all the time, the ring gets activated when the user clicks the button, then turns off after 3 seconds of inactivity.

In a study with 23 participants, twice as many users preferred IRIS over a voice command system alone (in this case, Apple鈥檚 Siri). On average, IRIS controlled home devices more than two seconds faster than voice commands.

鈥淚n the future, integrating the IRIS camera system into a health-tracking smart ring would be a transformative step for smart rings,鈥� Kim said. 鈥淚t鈥檇 let smart rings actually augment or improve human capability, rather than just telling you your step count or heart rate.鈥�

, 鈥斕齜oth UW doctoral students in the Allen School 鈥� were co-lead authors on the study, and , a UW professor in the Allen School, was the senior author. Additional co-authors include , a UW 91自拍 assistant in the Allen School; , a UW undergraduate in the Allen School; , a UW master鈥檚 student in the Allen School; and , a UW professor in the Allen School. This 91自拍 was funded by a Moore Inventor Fellow award and the National Science Foundation.

For more information, contact iris@cs.washington.edu.

Source

Imagine this: You鈥檙e at an office job, wearing noise-canceling headphones to dampen the ambient chatter. A co-worker arrives at your desk and asks a question, but rather than needing to remove the headphones and say, 鈥淲hat?鈥�, you hear the question clearly. Meanwhile the water-cooler chat across the room remains muted. Or imagine being in a busy restaurant and hearing everyone at your table, but reducing the other speakers and noise in the restaurant.

A team led by 91自拍ers at the 91自拍 has created a headphone prototype that allows listeners to create just such a 鈥渟ound bubble.鈥� The team鈥檚 artificial intelligence algorithms combined with a headphone prototype allow the wearer to hear people speaking within a bubble with a programmable radius of 3 to 6 feet. Voices and sounds outside the bubble are quieted an average of 49 decibels (approximately ), even if the distant sounds are louder than those inside the bubble.

The team Nov. 14 in Nature Electronics. The code for the proof-of-concept device is available for others to build on. The 91自拍ers are creating a startup to commercialize this technology.

鈥淗umans aren鈥檛 great at perceiving distances through sound, particularly when there are multiple sound sources around them,鈥� said senior author , a UW professor in the Paul G. Allen School of Computer Science & Engineering. 鈥淥ur abilities to focus on the people in our vicinity can be limited in places like loud restaurants, so creating sound bubbles on a has not been possible so far. Our AI system can actually learn the distance for each sound source in a room, and process this in real time, within 8 milliseconds, on the hearing device itself.鈥�

Researchers created the prototype with commercially available noise-canceling headphones. They affixed six small microphones across the headband. The team鈥檚 neural network 鈥� running on a small attached to the headphones 鈥� tracks when different sounds reach each microphone. The system then suppresses the sounds coming from outside the bubble, while playing back and slightly amplifying the sounds inside the bubble (because noise-canceling headphones physically let some sound through).

鈥淲e鈥檇 worked on a previous smart-speaker system where we spread the microphones across a table because we thought we needed significant distances between microphones to extract distance information about sounds,鈥� Gollakota said. 鈥淏ut then we started questioning our assumption. Do we need a big separation to create this 鈥榮ound bubble鈥�? What we showed here is that we don鈥檛. We were able to do it with just the microphones on the headphones, and in real-time, which was quite surprising.鈥�

To train the system to create sound bubbles in different environments, 91自拍ers needed a distance-based sound dataset collected in the real-world, which was not available. To gather such a dataset, they put the headphones on a mannequin head. A robotic platform rotated the head while a moving speaker played noises coming from different distances. The team collected data with the mannequin system as well as with human users in 22 different indoor environments, including offices and living spaces.

Researchers have determined that the system works for a couple of reasons. First, the wearer鈥檚 head reflects sounds, which helps the neural net distinguish sounds from various distances. Second, sounds (like human speech) have multiple frequencies, each of which goes through different phases as it travels from its source. The team鈥檚 AI algorithm, the 91自拍ers believe, is comparing the phases of each of these frequencies to determine the distance of any sound source (a person talking, for instance).

Headphones like Apple鈥檚 AirPods Pro 2 . But these features work by tracking head position and amplifying the sound coming from a specific direction, rather than gauging distance. This means the headphones can鈥檛 amplify multiple speakers at once, lose functionality if the wearer turns their head away from the target speaker, and aren鈥檛 as effective at reducing loud sounds from the speaker鈥檚 direction.

The system has been trained to work only indoors, because getting clean training audio is more difficult outdoors. Next, the team is working to make the technology function on hearing aids and noise-canceling earbuds, which requires a new strategy for positioning the microphones.

Additional co-authors are and , UW doctoral students in the Allen School; , a senior 91自拍er at Microsoft; and , director of 91自拍 at AssemblyAI. This 91自拍 was funded by a Moore Inventor Fellow award, a and the National Science Foundation.

For more information, contact soundbubble@cs.washington.edu.

Source

Noise-canceling headphones have gotten very good at creating an auditory blank slate. But allowing certain sounds from a wearer鈥檚 environment through the erasure still challenges 91自拍ers. The latest edition of Apple鈥檚 AirPods Pro, for instance, for wearers 鈥� sensing when they鈥檙e in conversation, for instance 鈥� but the user has little control over whom to listen to or when this happens.

A 91自拍 team has developed an artificial intelligence system that lets a user wearing headphones look at a person speaking for three to five seconds to 鈥渆nroll鈥� them. The system, called 鈥淭arget Speech Hearing,鈥� then cancels all other sounds in the environment and plays just the enrolled speaker鈥檚 voice in real time even as the listener moves around in noisy places and no longer faces the speaker.

The team presented May 14 in Honolulu at the ACM CHI Conference on Human Factors in Computing Systems. The is available for others to build on. The system is not commercially available.

鈥淲e tend to think of AI now as web-based chatbots that answer questions,鈥� said senior author , a UW professor in the Paul G. Allen School of Computer Science & Engineering. 鈥淏ut in this project, we develop AI to modify the auditory perception of anyone wearing headphones, given their preferences. With our devices you can now hear a single speaker clearly even if you are in a noisy environment with lots of other people talking.鈥�

To use the system, a person wearing off-the-shelf headphones fitted with microphones taps a button while directing their head at someone talking. The sound waves from that speaker鈥檚 voice then should reach the microphones on both sides of the headset simultaneously; there鈥檚 a 16-degree margin of error. The headphones send that signal to an , where the team鈥檚 machine learning software learns the desired speaker鈥檚 vocal patterns. The system latches onto that speaker鈥檚 voice and continues to play it back to the listener, even as the pair moves around. The system鈥檚 ability to focus on the enrolled voice improves as the speaker keeps talking, giving the system more training data.

The team tested its system on 21 subjects, who rated the clarity of the enrolled speaker鈥檚 voice nearly twice as high as the unfiltered audio on average.

This work builds on the team鈥檚 previous 鈥渟emantic hearing鈥� 91自拍, which allowed users to select specific sound classes 鈥� such as birds or voices 鈥� that they wanted to hear and canceled other sounds in the environment.

Currently the TSH system can enroll only one speaker at a time, and it鈥檚 only able to enroll a speaker when there is not another loud voice coming from the same direction as the target speaker鈥檚 voice. If a user isn鈥檛 happy with the sound quality, they can run another enrollment on the speaker to improve the clarity.

The team is working to expand the system to earbuds and hearing aids in the future.

Additional co-authors on the paper were , and , UW doctoral students in the Allen School, and , director of 91自拍 at AssemblyAI. This 91自拍 was funded by a Moore Inventor Fellow award, a and a .

For more information, contact tsh@cs.washington.edu.

Source

Most anyone who鈥檚 used noise-canceling headphones knows that hearing the right noise at the right time can be vital. Someone might want to erase car horns when working indoors, but not when walking along busy streets. Yet people can鈥檛 choose what sounds their headphones cancel.

Now, a team led by 91自拍ers at the 91自拍 has developed deep-learning algorithms that let users pick which sounds filter through their headphones in real time. The team is calling the system 鈥渟emantic hearing.鈥� Headphones stream captured audio to a connected smartphone, which cancels all environmental sounds. Either through voice commands or a smartphone app, headphone wearers can select which sounds they want to include from 20 classes, such as sirens, baby cries, speech, vacuum cleaners and bird chirps. Only the selected sounds will be played through the headphones.

The team presented Nov. 1 at in San Francisco. In the future, the 91自拍ers plan to release a commercial version of the system.

鈥淯nderstanding what a bird sounds like and extracting it from all other sounds in an environment requires real-time intelligence that today鈥檚 noise canceling headphones haven鈥檛 achieved,鈥� said senior author , a UW professor in the Paul G. Allen School of Computer Science & Engineering. 鈥淭he challenge is that the sounds headphone wearers hear need to sync with their visual senses. You can鈥檛 be hearing someone鈥檚 voice two seconds after they talk to you. This means the neural algorithms must process sounds in under a hundredth of a second.鈥�

Because of this time crunch, the semantic hearing system must process sounds on a device such as a connected smartphone, instead of on more robust cloud servers. Additionally, because sounds from different directions arrive in people鈥檚 ears at different times, the system must preserve these delays and other spatial cues so people can still meaningfully perceive sounds in their environment.

Tested in environments such as offices, streets and parks, the system was able to extract sirens, bird chirps, alarms and other target sounds, while removing all other real-world noise. When 22 participants rated the system鈥檚 audio output for the target sound, they said that on average the quality improved compared to the original recording.

In some cases, the system struggled to distinguish between sounds that share many properties, such as vocal music and human speech. The 91自拍ers note that training the models on more real-world data might improve these outcomes.

Additional co-authors on the paper were and , both UW doctoral students in the Allen School; , who completed this 91自拍 as a doctoral student in the Allen School and is now at Carnegie Mellon University; and , director of 91自拍 at AssemblyAI.

For more information, contact semantichearing@cs.washington.edu.

Source

In virtual meetings, it鈥檚 easy to keep people from talking over each other. Someone just hits mute. But for the most part, this ability doesn鈥檛 translate easily to recording in-person gatherings. In a bustling cafe, there are no buttons to silence the table beside you.

The ability to locate and control sound 鈥� isolating one person talking from a specific location in a crowded room, for instance 鈥� has , especially without visual cues from cameras.

A team led by 91自拍ers at the 91自拍 has developed a shape-changing smart speaker, which uses self-deploying microphones to divide rooms into speech zones and track the positions of individual speakers. With the help of the team鈥檚 deep-learning algorithms, the system lets users mute certain areas or separate simultaneous conversations, even if two adjacent people have similar voices. Like a fleet of Roombas, each about an inch in diameter, the microphones automatically deploy from, and then return to, a charging station. This allows the system to be moved between environments and set up automatically. In a conference room meeting, for instance, such a system might be deployed instead of a central microphone, allowing better control of in-room audio.

The team published Sept. 21 in Nature Communications.

鈥淚f I close my eyes and there are 10 people talking in a room, I have no idea who鈥檚 saying what and where they are in the room exactly. That鈥檚 extremely hard for the human brain to process. Until now, it鈥檚 also been difficult for technology,鈥� said co-lead author , a UW doctoral student in the Paul G. Allen School of Computer Science & Engineering. 鈥淔or the first time, using what we鈥檙e calling a robotic 鈥榓coustic swarm,鈥� we鈥檙e able to track the positions of multiple people talking in a room and separate their speech.鈥�

Previous 91自拍 on has required using overhead or on-device cameras, projectors or special surfaces. The UW team鈥檚 system is the first to accurately distribute a robot swarm using only sound.

The team鈥檚 prototype consists of seven small robots that spread themselves across tables of various sizes. As they move from their charger, each robot emits a high frequency sound, like a bat navigating, using this frequency and other sensors to avoid obstacles and move around without falling off the table. The automatic deployment allows the robots to place themselves for maximum accuracy, permitting greater sound control than if a person set them. The robots disperse as far from each other as possible since greater distances make differentiating and locating people speaking easier. Today鈥檚 consumer smart speakers have multiple microphones, but clustered on the same device, they鈥檙e too close to allow for this system鈥檚 mute and active zones.

鈥淚f I have one microphone a foot away from me, and another microphone two feet away, my voice will arrive at the microphone that鈥檚 a foot away first. If someone else is closer to the microphone that鈥檚 two feet away, their voice will arrive there first,鈥� said co-lead author , a UW doctoral student in the Allen School. 鈥淲e developed neural networks that use these time-delayed signals to separate what each person is saying and track their positions in a space. So you can have four people having two conversations and isolate any of the four voices and locate each of the voices in a room.鈥�

The team tested the robots in offices, living rooms and kitchens with groups of three to five people speaking. Across all these environments, the system could discern different voices within 1.6 feet (50 centimeters) of each other 90% of the time, without prior information about the number of speakers. The system was able to process three seconds of audio in 1.82 seconds on average 鈥� fast enough for live streaming, though a bit too long for real-time communications such as video calls.

As the technology progresses, 91自拍ers say, acoustic swarms might be deployed in smart homes to better differentiate people talking with smart speakers. That could potentially allow only people sitting on a couch, in an 鈥渁ctive zone,鈥� to vocally control a TV, for example.

Researchers plan to eventually make microphone robots that can move around rooms, instead of being limited to tables. The team is also investigating whether the speakers can emit sounds that allow for real-world mute and active zones, so people in different parts of a room can hear different audio. The current study is another step toward science fiction technologies, such as the 鈥渃one of silence鈥� in 鈥淕et Smart鈥� and 鈥淒une,鈥� the authors write.

Of course, any technology that evokes comparison to fictional spy tools will raise questions of privacy. Researchers acknowledge the potential for misuse, so they have included guards against this: The microphones navigate with sound, not an onboard camera like other similar systems. The robots are easily visible and their lights blink when they鈥檙e active. Instead of processing the audio in the cloud, as most smart speakers do, the acoustic swarms process all the audio locally, as a privacy constraint. And even though some people鈥檚 first thoughts may be about surveillance, the system can be used for the opposite, the team says.

鈥淚t has the potential to actually benefit privacy, beyond what current smart speakers allow,鈥� Itani said. 鈥淚 can say, 鈥楧on鈥檛 record anything around my desk,鈥� and our system will create a bubble 3 feet around me. Nothing in this bubble would be recorded. Or if two groups are speaking beside each other and one group is having a private conversation, while the other group is recording, one conversation can be in a mute zone, and it will remain private.鈥�

, formerly a principal 91自拍 manager at Microsoft, is a co-author on this paper, and , a professor in the Allen School, is a senior author. The 91自拍 was funded by a Moore Inventor Fellow award.

For more information, contact acousticswarm@cs.washington.edu.

Source

Even for scuba and snorkeling enthusiasts, the plunge into open water can be dislocating. Divers frequently swim with limited visibility, which can become a safety hazard for teams trying to find each other in an emergency. Yet even though many dive with smartwatches designed to go to depths of over 100 feet, accurately locating mobile devices underwater has confounded 91自拍ers.

Now, a team at the 91自拍 has developed the first underwater 3D-positioning app for smart devices. When at least three divers are within about 98 feet (30 meters) of each other, their devices鈥� existing speakers and microphones contact each other, and the app tracks each user鈥檚 location relative to the leader. This range can extend with more divers, if each is within 98 feet of another diver. The team will present in September at the in New York City.

鈥淢obile devices today can work nearly anywhere on Earth. You can be in a forest or on a plane and still get internet connectivity,鈥� said lead author , a UW doctoral student in the Paul G. Allen School of Computer Science & Engineering. 鈥淏ut the one place where we still hadn鈥檛 made mobile devices work was underwater. It鈥檚 kind of the final frontier.鈥�

Above water, GPS relies on a vast satellite network to locate mobile devices with radio signals. Underwater, these signals quickly fade. Sound, though, travels faster and farther in water than it does in air. Previous underwater positioning systems have relied on strategically placed buoys, but these systems are expensive and cumbersome to deploy, leading many divers to do without.

The UW team found that such buoys aren鈥檛 necessary. With the app, if the dive leader has at least one other diver visible, the group鈥檚 devices can send acoustic signals to each other through their microphones and speakers and use the timestamps to estimate each diver鈥檚 distance. Based on these distances, the app can estimate the group鈥檚 formation and each diver鈥檚 location. If a device also tracks depth, as sport monitors like the Apple Watch Ultra or the Garmin Descent do, the system can locate divers in 3D.

The app needs at least three devices in its network to function, and its accuracy improves as more devices are added. When tested with four to five devices in local lakes and a pool, the app estimated locations with an average error of about 5 feet (1.6 meters) 鈥� close enough for divers to see each other in most environments. To get actual GPS coordinates, instead of tracking locations relative to the dive leader, the leader needs to be wirelessly connected to a surface device on a boat with GPS capabilities.

The study builds on a , which allows divers to send messages to each other underwater.

鈥淭his and AquaApp can be used together,鈥� said author , a UW doctoral student in the Allen School. 鈥淔or example, if the dive leader finds someone going the wrong way, the leader can send an alert: 鈥楬ey, you鈥檙e going out of range. You need to come back.鈥� Or if a diver is running out of gas, an SOS can let the team find the person quickly even in murky water.鈥�

, a professor in the Allen School, is a senior author on this paper. This 91自拍 was funded by grants from the Gordon and Betty Moore Foundation and National Science Foundation.

For more information, contact underwaterGPS@cs.washington.edu.

Source