AssemblyAI Enhances Speaker Diarization with New Languages and Improved Accuracy

AssemblyAI
has
announced
significant
upgrades
to
its
Speaker
Diarization
service,
which
is
designed
to
identify
individual
speakers
within
a
conversation.
According
to
the
company,
these
improvements
have
led
to
enhanced
accuracy
and
expanded
language
support,
making
the
service
more
effective
and
versatile
for
end-users.

Speaker
Diarization
Improvements

The
updated
Speaker
Diarization
model
now
offers
up
to
13%
greater
accuracy
compared
to
its
predecessor.
The
enhancements
have
been
measured
across
various
industry
benchmarks,
including
a
10.1%
improvement
in
Diarization
Error
Rate
(DER)
and
a
13.2%
improvement
in
concatenated
minimum-permutation
word
error
rate
(cpWER).
These
metrics
are
critical
in
evaluating
the
performance
of
diarization
models,
with
lower
values
indicating
better
accuracy.

DER
measures
the
fraction
of
time
an
incorrect
speaker
is
attributed
to
the
audio,
while
cpWER
accounts
for
the
number
of
errors
made
by
the
speech
recognition
model,
including
those
due
to
incorrect
speaker
assignments.
AssemblyAI’s
improvements
in
both
metrics
highlight
the
model’s
enhanced
capability
in
accurately
identifying
speakers.

Speaker
Number
Accuracy

Another
significant
upgrade
is
the
85.4%
reduction
in
speaker
count
errors.
This
improvement
ensures
that
the
model
can
more
accurately
determine
the
number
of
unique
speakers
in
an
audio
file.
Accurate
speaker
count
is
essential
for
various
applications,
such
as
call
center
software
that
relies
on
identifying
the
correct
number
of
participants
in
a
conversation.

AssemblyAI’s
model
now
boasts
the
lowest
rate
of
speaker
count
errors
at
just
2.9%,
outperforming
several
other
providers
in
the
industry.

Increased
Language
Support

The
service
has
also
expanded
its
language
support,
now
available
in
five
additional
languages:
Chinese,
Hindi,
Japanese,
Korean,
and
Vietnamese.
This
brings
the
total
number
of
supported
languages
to
16,
covering
almost
all
languages
supported
by
AssemblyAI’s
Best
tier.

Technological
Advancements

The
improvements
to
Speaker
Diarization
stem
from
a
series
of
technological
upgrades:

1. 
   Universal-1
   Model:
   The
   new
   Speech
   Recognition
   model,
   Universal-1,
   has
   enhanced
   transcription
   accuracy
   and
   timestamp
   prediction,
   which
   are
   critical
   for
   aligning
   speaker
   labels
   with
   automatic
   speech
   recognition
   (ASR)
   outputs.
2. 
   Improved
   Embedding
   Model:
   Upgrades
   to
   the
   speaker-embedding
   model
   have
   improved
   the
   model’s
   ability
   to
   identify
   and
   differentiate
   between
   unique
   acoustical
   features
   of
   speakers.
3. 
   Increased
   Sampling
   Frequency:
   The
   input
   sampling
   frequency
   has
   been
   increased
   from
   8
   kHz
   to
   16
   kHz,
   providing
   higher-resolution
   input
   data
   and
   enabling
   the
   model
   to
   better
   distinguish
   between
   different
   speakers’
   voices.

Use
Cases
and
Applications

Speaker
Diarization
is
a
critical
feature
for
various
applications
across
industries:

Transcript
Readability

With
the
rise
of
remote
work
and
recorded
meetings,
accurate
and
readable
transcripts
are
more
important
than
ever.
Diarization
improves
the
readability
of
these
transcripts,
making
it
easier
for
users
to
digest
the
content.

Search
Experience

Many
conversation
intelligence
products
offer
search
features
that
allow
users
to
find
instances
where
specific
people
said
particular
things.
Accurate
diarization
is
essential
for
these
features
to
function
correctly.

Downstream
Analytics
and
LLMs

Many
analytical
features
and
large
language
models
(LLMs)
rely
on
knowing
who
said
what
to
extract
meaningful
information
from
recorded
speech.
This
is
crucial
for
applications
like
customer
service
software,
which
can
use
speaker
information
for
coaching
and
improving
agent
performance.

Creator
Tool
Features

Accurate
transcription
and
diarization
are
foundational
for
various
AI-powered
features
in
video
processing
and
content
creation,
such
as
automated
dubbing,
auto
speaker
focus,
and
AI-recommended
short
clips
from
long-form
content.

For
more
detailed
information,
you
can
visit
the
official

AssemblyAI
blog.

Image
source:
Shutterstock