Shingled magnetic recording
Shingled magnetic recording is a magnetic storage data recording technology used in hard disk drives to increase storage density and overall per-drive storage capacity. Conventional hard disk drives record data by writing non-overlapping concentric magnetic tracks, while shingled recording writes new tracks that overlap part of the previously written magnetic track, leaving the previous track narrower and allowing higher track density. Thus, the tracks partially overlap similar to roof shingles. This approach was selected because, if the writing head is made too narrow, it cannot provide the very high fields required in the recording layer of the disk.
The overlapping-tracks architecture complicates the writing process, since writing to one track also overwrites an adjacent track. If adjacent tracks contain valid data, they must be rewritten as well. As a result, a single write request may trigger a cascade of rewrites over a sequence of adjacent tracks. To place an upper limit on the number of rewrites needed to serve a single write request, SMR drives are divided into many append-only zones of overlapping tracks that need to be rewritten entirely when full, resembling flash blocks in solid-state drives. Device-managed SMR devices hide this complexity by managing it in the firmware, presenting an interface like any other hard disk. Other SMR devices are host-managed and depend on the operating system to know how to handle the drive, and only write sequentially to certain regions of the drive. While SMR drives can use DRAM, flash memory, and even a portion of their own platter reserved for use with CMR instead of SMR as a cache or buffer to improve writing performance, continuous writing of large amounts of data is noticeably slower than with CMR drives.
History
Seagate started shipping device-managed SMR hard drives in September 2013, stating an increase in overall capacity of about 25% compared to non-shingled storage. In September 2014, HGST announced a 10 TB drive filled with helium that uses host-managed shingled magnetic recording, although in December 2015 it followed this with a 10 TB helium-filled drive that uses conventional non-shingled perpendicular recording.Heavily overlapped tracks also appeared earlier in the consumer helical scan video cassette recorders that were popular in the 1980s and 1990s. In Extended Play mode, both VHS and Betamax reduced the track pitch by a factor of three. The severe interference from the adjacent tracks was partially mitigated by the use of slant azimuth recording.
In 2001 the SuperDisk LS-240 drive could use SMR technology to format a regular 1.44 MB floppy disk to hold 32MB of data using its own FD32MB format.
Data management
There are three different ways that data can be managed on an SMR drive: device-managed, host-managed and host-aware.Device-managed
A device-managed or drive-managed drive appears to the host identically to a non-shingled drive. It is not necessary for the host to follow any special protocols. All handling of data, as it relates to the shingled nature of the storage, is managed by the device. Sequential writes are more efficient. In addition, the host is unaware that the storage is shingled.The disk controller in a device-managed drive internally handles any re-writing required by the special characteristics of a shingled drive, similar to the way a flash memory controller in a solid-state drive internally handles re-writing required by the special characteristics of flash media. SMR drives have append-only zones which operate like SSD blocks, where a sector in a zone cannot be modified without re-writing the entire contents of the zone, so writes are first sent to a CMR buffer and the disk moves these data to SMR parts when idle.
Until backlash against Western Digital in 2020, this type of SMR drive was often not labelled by the manufacturer, except in disks labelled as "archival".
RAID resilvering tends to overload the cache, sending SMR drives into minutes-long pauses. Faulty firmware may also cause the drive to return IDNF S.M.A.R.T. errors under intensive workloads. Both behaviors tend to be interpreted as drive failure by the RAID controller.
The zoned nature of SMR also means that the disk suffers from write amplification when garbage collecting, although for hard drives the main problem with writes is speed instead of longevity. Some SMR hard drives support the TRIM command for this reason. Although such SMR hard drives support the TRIM command, they still suffer the file system fragmentations.
Host-managed
A host-managed device requires strict adherence to a special protocol by the host. Since the host manages the shingled nature of the storage, it is required to write sequentially so as to not destroy existing data. The drive will refuse to execute commands which violate this protocol.Host-aware
Host-aware is a combination of drive-managed and host-managed. The drive is capable of managing the shingled nature of the storage and will execute any command the host gives it, regardless of if it is sequential or not. However, the host is aware that the drive is shingled, and able to query the drive for fill levels. This allows the host to optimize writes for the shingled nature, while also allowing the drive to be flexible and backwards-compatible.Protocol
SMR devices are considered zoned devices, as the storage is divided into zones of usually 256 MiB size. Two sets of specialized commands, ZBC for SCSI and ZAC for SATA are available for SMR devices. They tell the host about whether each zone is CMR or SMR and allow them to address these zones directly. Unless specifically mentioned, the commands are only available on host-aware/-managed devices. The specific commands are:- REPORT ZONES, for information on disk layout and zone status
- * SMR or similar zones are sequential required on host-managed drives, but sequential preferred on host-aware ones.
- RESET WRITE POINTER, for rewinding the write pointer so a sequential zone becomes empty
- OPEN ZONE, for explicitly declaring access to a zone and locking the associated firmware resources
- CLOSE ZONE, to unlock an opened zone
- FINISH ZONE, fill a zone full and make it readable
SMR devices identify themselves per the following:
- Host-aware or device-managed drives are marked as normal block devices, so they can be recognized as a normal hard drive.
- * A ZONED field shows whether the drive is device-managed, host-aware, or neither. This is found in the SCSI Block Device Characteristics VPD page and the ATA capabilities log page.
- Host-managed drives use a new device type. Only ZAC/ZBC-aware computers can detect and use them.
The zoned interface is also useful for flash storage. ZNS spec has been released by the NVM Express organization.
Software and application
The higher density of SMR drives, combined with its random-read nature, fills a niche between the sequential-access tape storage and the random-access conventional hard drive storage. They are suited to storing data that are unlikely to be modified, but need to be read from any point efficiently. One example of the use case is Dropbox's Magic Storage system, which runs the on-disk extents in an append-only way. Device-managed SMR disks have also been marketed as "Archive HDDs" due to this property.A number of file systems in Linux are or can be tuned for SMR drives:
- F2FS, originally designed for flash media, has a Zoned Block Device mode. It can be used on host-managed drives with conventional zones for metadata.
- Btrfs had zoned mode support added in Linux kernel 5.12, and it's compatible with host-managed drives. Btrfs already writes mostly sequentially due to the CoW nature.
- ext4 can be experimentally tuned to write more sequentially. Theodore Ts'o and Abutalib Aghayev gave a talk in 2017 on their ext4-lazy. Seagate also has a more radical "SMRFFS" extension from 2015 that makes use of the ZBC/ZAC commands.
- For other filesystems, the Linux device mapper has a dm-zoned target that maps a host-managed drive into a random-writable drive. Linux kernels since 4.10 can perform this task without dm. A from 2019 exposes the zones as files for easier access.
- SaunaFS is distributed file system that supports host-managed SMR drives natively with RAID-like erasure code and is optimized for sequential write workloads. It is actively developed by Leil Storage and released as open source.
Dynamic hybrid SMR
While for traditional SMR models each zone is assigned a type at manufacture time, dynamic hybrid SMR drives allow the customer to reconfigure the zone type from shingled to conventional and back. Adjusting the SMR/CMR setting helps suit the drive to the current workload of "hot" and "cold" data.Criticism
SMR drives have received criticism online for slow read/write speeds and low stability.Drive manufacturers such as Seagate, Toshiba, and Western Digital have faced criticism for attempting to conceal the use of SMR technology in certain drive models without labelling them as such, including offering models with the technology within high-end product lines that had otherwise used CMR. In May 2020, a class-action lawsuit was filed in the United States against WD, alleging that the company had misled and harmed consumers by offering products considered unfit for use in network-attached storage within a product line that was marketed for this purpose. The suit sought a permanent injunction preventing WD from advertising SMR hard drives as being fit for use in NAS and RAID configurations. In June 2020, WD announced that it would split its WD Red product line into sub-brands, with SMR-based drives being branded as "WD Red", and CMR-based drives being branded as "WD Red Plus" or "WD Red Pro". The suit was settled in 2021.
Specifications
- ZAC/ZBC version 1
- * T10,, 2014, Draft revision 1
- * T13,, Draft revision 5, 2015
- ZAC/ZBC version 2
- * T10,, 2020, Draft revision 04a
- * T13, ZAC-2, PDF unavailable