Opus (audio format)


Opus is a free and open source lossy audio coding format developed by the Xiph.Org Foundation and standardized by the Internet Engineering Task Force, designed for efficient low-latency encoding of both speech and general audio. Due to its lower latency relative to other standard codecs, Opus finds specific use cases in real-time interactive communication for low-end embedded processors. Opus replaces both Vorbis and Speex for new applications.
Opus combines the speech-oriented LPC-based SILK algorithm and the lower-latency MDCT-based CELT algorithm, switching between or combining them as needed. Bitrate, audio bandwidth, complexity, and algorithm choice can be adjusted for each individual frame. Opus has low algorithmic delay ideal for use as part of a real-time communication link, networked music performances, and live lip sync; by trading off quality or bitrate, the delay can be further reduced down to 5 ms. Its delay thus is significantly lower compared to competing codecs, which require well over 100 ms. Opus remains competitive with these formats in terms of quality per bitrate.
As an open format standardized through RFC 6716, a reference implementation called libopus is available under the New BSD License. The reference has both fixed-point and floating-point optimizations for low- and high-end devices, with SIMD optimizations on platforms that support them. All known software patents that cover Opus are licensed under royalty-free terms. Opus is widely used as a voice over IP codec in applications such as Discord, WhatsApp, and the PlayStation 4. Listening tests have ranked it higher-quality than other standard audio formats at any given bitrate until transparency is reached, including MP3, AAC, and HE-AAC.

Features

Opus supports constant and variable bitrate encoding from 6 kbit/s to 510 kbit/s, frame sizes from 2.5 ms to 60 ms, and five sampling rates from 8 kHz to 48 kHz. An Opus stream can support up to 255 audio channels, and it allows channel coupling between channels in groups of two using mid-side coding.
Opus has very short latency, which makes it suitable for real-time applications such as telephony, voice over IP and videoconferencing; research by Xiph led to the CELT codec currently adopted by Opus, which preserved quality while maintaining low delay. In any Opus stream, the bitrate, bandwidth, and delay can be continually varied without introducing any distortion or discontinuity; mixing packets from different streams will cause a smooth non-intrusive change, rather than the distortion common in other codecs. Unlike Vorbis, Opus does not require large codebooks for each individual file, making it relatively more efficient and resilient for short clips of audio.
File:FSsongmetal2-Opus-exp7.20120823-sweep.png|thumb|400px|right|Spectrogram of Opus-encoded audio as bitrate rises clearly shows lowpass behavior and better preservation of the band energy with CELT.
The Opus format is based on a combination of the full-bandwidth CELT format and the speech-oriented SILK format, both heavily modified: CELT is based on the modified discrete cosine transform that most music codecs use, using CELP techniques in the frequency domain for better prediction, while SILK uses linear predictive coding and an optional long-term prediction filter to model speech. In Opus, both were modified to support more frame sizes, as well as further algorithmic improvements and integration, such as using CELT's range encoder for both types. To minimize overhead at low bitrates, if latency is not as pressing, SILK has support for packing multiple 20 ms frames together, sharing context and headers; SILK also allows Low Bitrate Redundancy frames, allowing low-quality packet loss recovery. CELT includes both spectral replication and noise generation, similar to AAC's SBR and PNS, and can further save bits by filtering out all harmonics of tonal sounds entirely, then replicating them in the decoder. Better tone detection is an ongoing project to improve quality.
The format has three different modes: speech, hybrid, and CELT. When compressing speech, SILK is used for audio frequencies up to 8 kHz. If wider bandwidth is desired, a hybrid mode uses CELT to encode the frequency range above 8 kHz. The third mode is pure-CELT, designed for general audio. SILK is inherently VBR and cannot hit a bitrate target, while CELT can always be encoded to any specific number of bytes, enabling hybrid and CELT mode when CBR is required.
SILK supports frame sizes of 10, 20, 40 and 60 ms. CELT supports frame sizes of 2.5, 5, 10 and 20 ms. Thus, hybrid mode only supports frame sizes of 10 and 20 ms; frames shorter than 10 ms will always use CELT mode. A typical Opus packet contains a single frame, but packets of up to 120 ms are produced by combining multiple frames per packet. Opus can transparently switch between modes, frame sizes, bandwidths, and channel counts on a per-packet basis, although specific applications may choose to limit this.
The reference implementation is written in C and compiles on hardware architectures with or without a floating-point unit, although floating-point is currently required for audio bandwidth detection and most speed optimizations.

Containers

Opus packets are not self-delimiting, but are designed to be used inside a container of some sort which supplies the decoder with each packet's length. Opus was originally specified for encapsulation in Ogg containers, specified as audio/ogg; codecs=opus, and for Ogg Opus files the .opus filename extension is recommended. Opus streams are also supported in Matroska, WebM, MPEG-TS, and MP4.
Alternatively, each Opus packet may be wrapped in a network packet which supplies the packet length. Opus packets may be sent over an ordered datagram protocol such as RTP.
An optional self-delimited packet format is defined in an appendix to the specification. This uses one or two additional bytes per packet to encode the packet length, allowing packets to be concatenated without encapsulation.

Bandwidth and sampling rate

Opus allows the following bandwidths during encoding. Opus compression does not depend on the input sample rate; timestamps are measured in 48 kHz units even if the full bandwidth is not used. Likewise, the output sample rate may be freely chosen. For example, audio can be input at 16 kHz yet be set to encode only narrowband audio. Version 1.6 introduced an experimental build flag that enables a feature known as Opus HD, allowing sampling rates of up to 96 kHz for non-audio purposes.
AbbreviationAudio
bandwidth		Effective
sample rate
NB 4 kHz8 kHz
MB 6 kHz12 kHz
WB 8 kHz16 kHz
SWB 12 kHz24 kHz
FB 20 kHz48 kHz

History

Opus was proposed for the standardization of a new audio format at the IETF, which was eventually accepted and granted by the codec working group. It is based on two initially separate standard proposals from the Xiph.Org Foundation and Skype Technologies S.A.. Its main developers are Jean-Marc Valin, Koen Vos, and Timothy B. Terriberry. Among others, Juin-Hwey Chen, Gregory Maxwell, and Chris Montgomery were also involved.
The development of the CELT part of the format originated from discussions on a successor for Vorbis under the working name Ghost. As a newer speech codec from the Xiph.Org Foundation, Opus replaces Xiph's older speech codec Speex, an earlier project of Jean-Marc Valin. CELT has been worked on since November 2007.
The SILK part has been under development at Skype since January 2007 as the successor of their SVOPC, an internal project to make the company independent from third-party codecs like iSAC and iLBC and respective license payments.
In March 2009, Skype suggested the development and standardization of a wideband audio format within the IETF. Nearly a year passed with much debate on the formation of an appropriate working group. Representatives of several companies which were taking part in the standardization of patent-encumbered competing format, including Polycom and Ericsson—the creators and licensors of G.719—as well as France Télécom, Huawei and the Orange Labs, which were involved in the creation of G.718, stated objections against the start of the standardization process for a royalty-free format. The working group finally formed in February 2010, and even the corresponding Study Group 16 from the ITU-T pledged to support its work.
In July 2010, a prototype of a hybrid format was presented that combined the two proposed format candidates SILK and CELT. In September 2010, Opus was submitted to the IETF as proposal for standardization. For a short time the format went under the name of Harmony before it got its present name in October 2010. At the beginning of February 2011, the bitstream format was tentatively frozen, subject to last changes. Near the end of July 2011, Jean-Marc Valin was hired by the Mozilla Corporation to continue working on Opus.

Finalization (1.0)

In November 2011, the working group issued the last call for changes on the bitstream format. The bitstream has been frozen since January 8, 2012. On July 2, 2012, Opus was approved by the IETF for standardization. The reference software entered release candidate state on August 8, 2012. The final specification was released as RFC 6716 on September 10, 2012. and versions 1.0 and 1.0.1 of the reference implementation libopus were released the day after.
On July 11, 2013, libopus 1.0.3 brought bug fixes and a new surround sound API that improves channel allocation and quality, especially for LFE.

1.1

On December 5, 2013, libopus 1.1 was released, incorporating overall speed improvements and significant encoder quality improvements: Tonality estimation boosts bitrate and quality for previously , like harpsichords; automated speech/music detection improves quality in mixed audio; mid-side stereo reduces the bitrate needs of many songs; band precision boosting for improved transients; and DC rejection below 3 Hz. Two new VBR modes were added: unconstrained for more consistent quality, and temporal VBR that boosts louder frames and generally improves quality.
libopus 1.1.1 was released on November 26, 2015, and 1.1.2 on January 12, 2016, both adding speed optimizations and bug fixes. July 15, 2016 saw the release of version 1.1.3 and includes bug fixes, optimizations, documentation updates and experimental Ambisonics work.